Incompatibility between bulk enqueue and concurrency

[Alexander-Blair]

Hey Ben,

Just wanted to check - is bulk enqueue intended to work alongside concurrency controls? It seems like concurrency control ignores the bulk enqueue and inserts one at a time. I can imagine how these two modes may be incompatible, though is it possible for the bulk enqueue to complain if a job with concurrency controls is enqueued within its block?

If it's helpful, I took some screenshots that show the Rails debug output to demonstrate:

Before

After

Thanks,
Alex

[bensheldon]

With Bulk Enqueuing, when a job has enqueue-level concurrency controls, the jobs are inserted one at a time to ensure the concurrency limits are respected:

good_job/lib/good_job/bulk.rb

Lines 94 to 108 in 07c4cf1

	if adapter.respond_to?(:enqueue_all)
	unbulkable_jobs, bulkable_jobs = jobs.partition do |job|
	job.respond_to?(:good_job_concurrency_key) && job.good_job_concurrency_key &&
	(job.class.good_job_concurrency_config[:enqueue_limit] || job.class.good_job_concurrency_config[:total_limit])
	end
	adapter.enqueue_all(bulkable_jobs) if bulkable_jobs.any?
	else
	unbulkable_jobs = jobs
	end
	unbulkable_jobs.each do |job|
	job.enqueue
	rescue GoodJob::ActiveJobExtensions::Concurrency::ConcurrencyExceededError
	# ignore
	end

Could you share what the warning message might be? I'm not sure why it would be a problem: the jobs will still be enqueued and if you want the concurrency limits, there's not an alternative to enqueuing them.

[Alexander-Blair]

One problem is that, if the developer assumes that the operation is atomic (e.g. all jobs get queued, or none do), that assumption would be incorrect. Granted that jobs should be idempotent anyway, but it's still a useful tool.

Message could be something along the lines of "WARNING: jobs with concurrency controls cannot be bulk enqueued. At least one job with concurrency controls was added within GoodJob::Bulk.enqueue"

[Alexander-Blair]

On a separate note, I'm not sure if you have thoughts on this @bensheldon, but I'm trying to improve how we operate our background jobs queue, and one issue we run into is fixing / handling discarded jobs. Current setup is:

• By default, jobs retry ~8 times, and then move to the discard pile (infinite retries = infinite error tracking noise)
• We retain discarded jobs, then manually delete jobs we can't do anything with, and retry jobs we've fixed

A couple of things we struggle with:

• Is someone already looking at or addressing a failed job? (I would be tempted to build an extension to GoodJob that hooks into our users table)
• Having some jobs in the discard queue that are already being "worked on", and some that need attention. Perhaps the assignment extension + a filter would actually address this

Let me know if this question should be fielded elsewhere / in another issue if anybody else has mentioned it!