Queue close unexpected behavior with multiple named job processors

Created by: ErikMartensson

Description

I experiencing problems while trying to gracefully shutdown my queue(s). My code works (or should work) a bit like this:

• I have multiple queues, with named jobs
• I have one or more processes per queue (one process for each named job obviously)
• I have some logic that listens for interrupt signals so that I know when to try and gracefully shut down my queues
• I loop through my queues and run .close() on each of the queues
• I want every processes to finish their current job (if they have any) and then close the queue to prevent any new jobs from being processed
• Once all currently processing jobs have finished I want the Node process to exit

The problem I'm experiencing is that whenever I run .close() on a queue, the queue will close even if there's a job currently being processed.

However, this only happens when I have multiple processes for a queue. Which is necessary since I'm using named jobs.

Minimal, Working Test code to reproduce the issue.

Here's a repo with some code that reproduces this behaviour and (hopefully) explains my problem better. https://github.com/ErikMartensson/bull-multiple-processes-named-jobs

Whenever the second process is removed or commented out, the queue will wait for the current job to end processing and then close as expected.

(An easy to reproduce test case will dramatically decrease the resolution time.)

const testQueue = new Queue('test');

testQueue.process('jobA', async (job) => {
    console.log(`Process started for Job ${job.id}`);
    return new Promise(resolve => {
        setTimeout(() => {
            console.log(`Process done for Job ${job.id}!`);
            resolve();
        }, 10000); // 10 seconds
    });
});

// Comment this process out to see the difference.
testQueue.process('jobB', async () => {});

// Add a job to the queue
testQueue.add('jobA', {});

process.on('SIGINT', async () => {
    console.log('Closing Queue now...');
    await testQueue.close();
    console.log('Queue closed');
    process.exit(0);
});

Run this code and press Ctrl + C to exit the process, within 10 seconds. Notice how the queue closes before reaching the end of the process.

Bull version

3.12.1

Additional information

At one point while testing this, there was two jobs being processed at the same time, and then the queue actually did close as expected. Even though there was two processes defined, which kind of contradicts my problem. Unfortunately I can't figure out how to reproduce that scenario.