Skip to content

Implement new Background Task queue

What does this MR do?

Implements a new primary queue type: "Background Task", which handles logic previously relegated to "fire-and-forget" async promises. This new implementation improves performance, is more reliable, and maintains system stability during bursts of heavy load. To complement the new queue, this PR also overhauls the "Collapsed Queue" implementation - making it faster and safe to use in multi-process deployments.

Background Task types:

  • update-user: updates a remote user over AP.
  • update-featured: updates a remote user's featured notes over AP.
  • update-instance: updates a remote instance over AP.
  • update-user-tags: updates rankings for a user's featured hashtags.
  • update-note-tags: updates rankings for hashtags used in a note.
  • update-latest-note: updates "latest note" index after a note is created, edited, or deleted.
  • post-deliver: updates stats and metrics after attempting to deliver an activity.
  • post-inbox: updates stats and metrics after attempting to process an inbox activity.
  • post-note: complete slow "post processing" tasks after a local or remote note is created or edited.
  • post-suspend: pauses federation, notifications, and other automatic behavior after a user is suspended.
  • post-unsuspend: restores federation, notifications, and other automatic behavior after a suspension is revoked.
  • delete-file: removes a drive file, including the physical file and/or S3 blob.
  • delete-ap-logs: removes all AP logs associated with a given URI(s).

Collapsed Queue types:

  • updateInstanceQueue: aggregates incremental federation stats and policy changes to instance entities.
  • updateUserQueue: aggregates incremental stats to user entities.
  • updateNoteQueue: aggregates incremental stats to note entities.
  • updateAccessTokenQueue: accumulates activity for access_token entities.
  • updateAntennaQueue: accumulates activity for antenna entities.

Additional performance improvements and micro-optimizations:

  • Use CacheService to fetch users in many more places.
  • Limit concurrency when performing parallel async map operations.
  • Add caching to reduce database load of get-online-users-count.
  • Remove unused API and websocket calls from admin dashboard
  • Opportunistically use note.user relation in PollService.deliverQuestionUpdate
  • Slightly reduce stalls when updating MiInstance.

Additional stability and reliability improvements:

  • Six cache sync after populating a user's banner.
  • Synchronize collapsed queues across the cluster to avoid data races.
  • Migrate to InternalEventService in ApPersonService.
  • Update userProfileCache from updatePerson.
  • More efficient algorithm for identifying cascaded notes.
  • Make NoteDeleteService.delete safe to call inside a loop.
  • Fix account deletion not triggering background tasks for cascaded note deletions.
  • await more promises to improve stack traces.
  • Track federation-related promises to avoid data loss during restart.

Additional bug fixes:

  • Fix incorrect counts for self-renotes and renotes by bots.
  • Fix cascaded note deletion missing some notes.
  • Fix stall in EnableInstanceHOTUpdates1750217001651 migration.

Additional technical changes:

  • Add immediate option to NoteDeleteService.
  • Add MiUser.lastFetchedFeaturedAt.
  • Fix typings of resolveCollection and resolveCollectionItems.
  • Fix typings of Serialized<T>.
  • Add CollapsedQueue.onPerform callback.
  • Add CollapsedQueue.delete method.
  • Return results from CollapsedQueue.performAllNow.
  • Fix API schema for admin/queue/jobs endpoint.
  • Add name and error logging to CollapsedQueue.
  • Rename noteEditRepository to noteEditsRepository, matching other repository names.
  • Remove useless async/await from charts.

Contribution Guidelines

By submitting this merge request, you agree to follow our Contribution Guidelines

  • I agree to follow this project's Contribution Guidelines
  • I have made sure to test this merge request

Merge request reports

Loading