-
-
Notifications
You must be signed in to change notification settings - Fork 230
Description
Rails 8.0.2, good_job 4.11.1
Seeing behavior where job hash parameters do not maintain order in async execution -- post serde.
ActiveStorage variants rely heavily on the order of variant arguments to generate deterministic variant keys. My first theory is that the issue may be rooted in good_job storing job params in a jsonb column. We use Postgres which does not guarantee maintained key order.
Given a variant defined like so:
has_one_attached :file, dependent: :purge do |attachable|
attachable.variant :preview,
loader: { n: -1 },
saver: {
quality: 80,
strip: true,
density: 72
},
resize_to_limit: [ 750, 400 ],
preprocessed: true
end
I notice that the order of these variant arguments are expressed differently once in the ActiveStorage::VariantJob, when running with good_job as the async executor. (Logging here is from my own temp monkeypatch).
I, [2026-02-09T10:40:08.422723 #86143] INFO -- : Performing ActiveStorage::TransformJob (Job ID: 461f229a-f000-41c0-8540-844be095561d) from GoodJob(transform) enqueued at 2026-02-09T17:40:07.417676000Z with arguments: #<GlobalID:0x0000000139a88ee8 @uri=#<URI::GID gid://mc-rails-api/ActiveStorage::Blob/6235850c-bead-4f2d-879c-6d20a58ff19e>>, {saver: {strip: true, density: 72, quality: 80}, loader: {n: -1}, resize_to_limit: [600, nil]}
I, [2026-02-09T10:40:08.423516 #86143] INFO -- : [AS::TransformJob PATCH] PID=86143 transformations={saver: {strip: true, density: 72, quality: 80}, loader: {n: -1}, resize_to_limit: [600, nil]}
This results in the variant that is generated by the ActiveStorage::TransformJob having a different variant digest key than what would otherwise be generated by calling record.file.representation(:preview).processed in any other context.
I was able to work around this to achieve uniformity in all contexts with a crude monkeypatch (acknowledging that monkeypatches are inherently brittle).
Rails.application.config.after_initialize do
require "active_storage"
class ActiveStorage::Variation
alias_method :__initialize, :initialize
def initialize(transformations)
canon = canonicalize_transformations(transformations)
Rails.logger.debug(
"[AS::Variation.encode PATCH] PID=#{Process.pid} " \
"before=#{transformations.inspect} after=#{canon.inspect}"
)
__initialize(canon)
end
private
def canonicalize_transformations(obj)
# Use recursion to deterministically sort hash keys
case obj
when Hash
obj
.to_h { |k, v| [ k.to_sym, canonicalize_transformations(v) ] }
.sort_by { |k, _| k.to_s }
.to_h
when Array
obj.map { |v| canonicalize_transformations(v) }
else
obj
end
end
end
end
This however prevents the ability to affect intentional operational ordering with the variant args.
Unsure what good_job should/could do to mitigate this, but raising the issue as it can result in major performance ramifications once systems start to use active storage representations / variants.
Metadata
Metadata
Assignees
Labels
Projects
Status