Removing completed jobs automatically & atomically

Created by: bradvogel

It's our desire to remove jobs automatically upon completion. My understanding of the Bull API is that we should do this by listening to the completed event on the queue and call job.remove(), like this:

this.on('completed', job => job.remove());

However, we're finding that some completed jobs aren't getting removed. The likely reason for this is that the server is occasionally being killed right after the job is moved to "completed", but before job.remove() can be called. Since events are broadcast locally, there is no second server to pick up the message, the job stays around forever. Our current workaround is to run a cron job every hour that calls clean(10 * 1000, 'completed') on the queue.

I wanted to get your thoughts on the following possible approaches to clean up completed jobs:

1. Add a removeOnCompleted option to the job and have the moveToCompleted script atomically remove the job upon completion.
2. Use the new distributed events architecture to broadcast the 'completed' event to all workers, so one can remove the job.