Twitter import failing
What happened? (Please give us a brief description of what happened.) Attempting to perform a twitter import, I start the import, an import folder shows up in drive, but no posts or media actually load.
Only thing I see on Bull dashboard is these two jobs, then nothing after.
What did you expect to happen? (Please give us a brief description of what you expected to happen.) Importing of twitter data
Version (What version of Sharkey is your instance running? You can find this by clicking your instance's logo at the top left and then clicking instance information.) 2024.3.1
Instance (What instance of Sharkey are you using?) https://sharktest.baka.social/
What type of issue is this? (If this happens on your device and has to do with the user interface, it's client-side. If this happens on either with the API or the backend, or you got a server-side error in the client, it's server-side.) Server-Side
What browser are you using? (Client-side issues only)
What operating system are you using? (Client-side issues only)
How do you deploy Sharkey on your server? (Server-side issues only) Docker
What operating system are you using? (Server-side issues only) Ubuntu 22.04 Relevant log output
2024-03-16 08:05:38
INFO 1 [queue import-notes] Starting note import of 9qwi10eedp87006a ...
2024-03-16 08:05:38
INFO 1 [queue import-notes] Temp dir is /tmp/tmp-122-u1T7CXLllvmA
2024-03-16 08:05:38
INFO 1 [download] Downloading https://files.baka.social/file/sharkeytest/shork/7760fba1-ed01-4ce5-a7e9-aa778238ce52 to /tmp/tmp-122-u1T7CXLllvmA/twitter.zip ...
2024-03-16 08:06:02
DONE 1 [download] Download finished: https://files.baka.social/file/sharkeytest/shork/7760fba1-ed01-4ce5-a7e9-aa778238ce52
2024-03-16 08:06:02
DONE 1 [queue import-notes] Unzipping to /tmp/tmp-122-u1T7CXLllvmA/twitter
2024-03-16 08:06:03
WARN 1 [queue db] failed(TypeError: removeFunction is not a function
2024-03-16 08:06:03
at _cleanupCallback (/sharkey/node_modules/.pnpm/tmp@0.2.2/node_modules/tmp/lib/tmp.js:358:16)
2024-03-16 08:06:03
at ImportNotesProcessorService.process (file:///sharkey/packages/backend/built/queue/processors/ImportNotesProcessorService.js:188:17)
2024-03-16 08:06:03
at async Worker.processJob (/sharkey/node_modules/.pnpm/bullmq@5.4.0/node_modules/bullmq/dist/cjs/classes/worker.js:395:28)
2024-03-16 08:06:03
at async Worker.retryIfFailed (/sharkey/node_modules/.pnpm/bullmq@5.4.0/node_modules/bullmq/dist/cjs/classes/worker.js:584:24)) id=1358 {
2024-03-16 08:06:03
job: Job {
2024-03-16 08:06:03
queue: Worker {
2024-03-16 08:06:03
_events: [Object: null prototype],
2024-03-16 08:06:03
_eventsCount: 5,
2024-03-16 08:06:03
_maxListeners: undefined,
2024-03-16 08:06:03
name: 'db',
2024-03-16 08:06:03
opts: [Object],
2024-03-16 08:06:03
closed: false,
2024-03-16 08:06:03
connection: [RedisConnection],
2024-03-16 08:06:03
qualifiedName: 'sharktest.baka.social:queue:db:db',
2024-03-16 08:06:03
keys: [Object],
2024-03-16 08:06:03
toKey: [Function (anonymous)],
2024-03-16 08:06:03
scripts: [Scripts],
2024-03-16 08:06:03
abortDelayController: null,
2024-03-16 08:06:03
blockUntil: 0,
2024-03-16 08:06:03
drained: false,
2024-03-16 08:06:03
extendLocksTimer: Timeout {
2024-03-16 08:06:03
_idleTimeout: 7500,
2024-03-16 08:06:03
_idlePrev: [TimersList],
2024-03-16 08:06:03
_idleNext: [Timeout],
2024-03-16 08:06:03
_idleStart: 183808384,
2024-03-16 08:06:03
_onTimeout: [AsyncFunction (anonymous)],
2024-03-16 08:06:03
_timerArgs: undefined,
2024-03-16 08:06:03
_repeat: null,
2024-03-16 08:06:03
_destroyed: false,
2024-03-16 08:06:03
[Symbol(refed)]: true,
2024-03-16 08:06:03
[Symbol(kHasPrimitive)]: false,
2024-03-16 08:06:03
[Symbol(asyncId)]: 2187935,
2024-03-16 08:06:03
[Symbol(triggerId)]: 0
2024-03-16 08:06:03
},
2024-03-16 08:06:03
limitUntil: 0,
2024-03-16 08:06:03
waiting: null,
2024-03-16 08:06:03
running: true,
2024-03-16 08:06:03
id: '5731b51d-d83e-46ef-bac5-6e9d360898cf',
2024-03-16 08:06:03
processFn: [Function (anonymous)],
2024-03-16 08:06:03
blockingConnection: [RedisConnection],
2024-03-16 08:06:03
stalledCheckTimer: Timeout {
2024-03-16 08:06:03
_idleTimeout: 30000,
2024-03-16 08:06:03
_idlePrev: [Timeout],
2024-03-16 08:06:03
_idleNext: [Timeout],
2024-03-16 08:06:03
_idleStart: 183800749,
2024-03-16 08:06:03
_onTimeout: [AsyncFunction (anonymous)],
2024-03-16 08:06:03
_timerArgs: undefined,
2024-03-16 08:06:03
_repeat: null,
2024-03-16 08:06:03
_destroyed: false,
2024-03-16 08:06:03
[Symbol(refed)]: true,
2024-03-16 08:06:03
[Symbol(kHasPrimitive)]: false,
2024-03-16 08:06:03
[Symbol(asyncId)]: 2130350,
2024-03-16 08:06:03
[Symbol(triggerId)]: 0
2024-03-16 08:06:03
},
2024-03-16 08:06:03
asyncFifoQueue: [AsyncFifoQueue],
2024-03-16 08:06:03
[Symbol(kCapture)]: false
2024-03-16 08:06:03
},
2024-03-16 08:06:03
name: 'importNotes',
2024-03-16 08:06:03
data: { user: [Object], fileId: '9qx5l9wmdp87008s', type: 'Twitter' },
2024-03-16 08:06:03
opts: {
2024-03-16 08:06:03
attempts: 0,
2024-03-16 08:06:03
delay: 0,
2024-03-16 08:06:03
removeOnFail: true,
2024-03-16 08:06:03
removeOnComplete: true,
2024-03-16 08:06:03
backoff: undefined
2024-03-16 08:06:03
},
2024-03-16 08:06:03
id: '1358',
2024-03-16 08:06:03
progress: 0,
2024-03-16 08:06:03
returnvalue: null,
2024-03-16 08:06:03
stacktrace: [
2024-03-16 08:06:03
'TypeError: removeFunction is not a function\n' +
2024-03-16 08:06:03
' at _cleanupCallback (/sharkey/node_modules/.pnpm/tmp@0.2.2/node_modules/tmp/lib/tmp.js:358:16)\n' +
2024-03-16 08:06:03
' at ImportNotesProcessorService.process (file:///sharkey/packages/backend/built/queue/processors/ImportNotesProcessorService.js:188:17)\n' +
2024-03-16 08:06:03
' at async Worker.processJob (/sharkey/node_modules/.pnpm/bullmq@5.4.0/node_modules/bullmq/dist/cjs/classes/worker.js:395:28)\n' +
2024-03-16 08:06:03
' at async Worker.retryIfFailed (/sharkey/node_modules/.pnpm/bullmq@5.4.0/node_modules/bullmq/dist/cjs/classes/worker.js:584:24)'
2024-03-16 08:06:03
],
2024-03-16 08:06:03
attemptsStarted: 1,
2024-03-16 08:06:03
attemptsMade: 1,
2024-03-16 08:06:03
delay: 0,
2024-03-16 08:06:03
repeatJobKey: undefined,
2024-03-16 08:06:03
timestamp: 1710576338572,
2024-03-16 08:06:03
parentKey: undefined,
2024-03-16 08:06:03
parent: undefined,
2024-03-16 08:06:03
toKey: [Function: bound ],
2024-03-16 08:06:03
scripts: Scripts { queue: [Worker], moveToFinishedKeys: [Array] },
2024-03-16 08:06:03
queueQualifiedName: 'sharktest.baka.social:queue:db:db',
2024-03-16 08:06:03
processedOn: 1710576338573,
2024-03-16 08:06:03
failedReason: 'removeFunction is not a function',
2024-03-16 08:06:03
token: '5731b51d-d83e-46ef-bac5-6e9d360898cf:36721',
2024-03-16 08:06:03
finishedOn: 1710576363233
2024-03-16 08:06:03
},
2024-03-16 08:06:03
e: {
2024-03-16 08:06:03
stack: 'TypeError: removeFunction is not a function\n' +
2024-03-16 08:06:03
' at _cleanupCallback (/sharkey/node_modules/.pnpm/tmp@0.2.2/node_modules/tmp/lib/tmp.js:358:16)\n' +
2024-03-16 08:06:03
' at ImportNotesProcessorService.process (file:///sharkey/packages/backend/built/queue/processors/ImportNotesProcessorService.js:188:17)\n' +
2024-03-16 08:06:03
' at async Worker.processJob (/sharkey/node_modules/.pnpm/bullmq@5.4.0/node_modules/bullmq/dist/cjs/classes/worker.js:395:28)\n' +
2024-03-16 08:06:03
' at async Worker.retryIfFailed (/sharkey/node_modules/.pnpm/bullmq@5.4.0/node_modules/bullmq/dist/cjs/classes/worker.js:584:24)',
2024-03-16 08:06:03
message: 'removeFunction is not a function',
2024-03-16 08:06:03
name: 'TypeError'
2024-03-16 08:06:03
}
2024-03-16 08:06:03
}Contribution Guidelines By submitting this issue, you agree to follow our Contribution Guidelines
- I agree to follow this project's Contribution Guidelines
- I have searched the issue tracker for similar issues, and this is not a duplicate.
Activity
that looks like a bug in the https://www.npmjs.com/package/tmp library…
Welcome to Node.js v20.11.0. Type ".help" for more information. > const tmp = await import('tmp'); undefined > tmp.dir({unsafeCleanup: true},(e,path,cleanup) => { cleanup() }); undefined > Uncaught TypeError: removeFunction is not a function at _cleanupCallback (/home/dakkar/src/Sharkey/node_modules/.pnpm/tmp@0.2.2/node_modules/tmp/lib/tmp.js:358:16) at REPL6:1:53 at _dirCreated (/home/dakkar/src/Sharkey/node_modules/.pnpm/tmp@0.2.2/node_modules/tmp/lib/tmp.js:207:7)seems fixed in 0.2.3, we should bump that dependency
mentioned in commit f4e89f2e
mentioned in merge request !475 (merged)
assigned to @dakkar
added bugbackend in progress in review labels
removed in progress label
mentioned in commit cfc8081c
closed with merge request !475 (merged)
closed with commit f4e89f2e
removed in review label
Please register or sign in to reply