Skip to content

Job arguments key-order serialization issue (ActiveStorage::TransformJob results in mismatched variant digest keys) #1715

@ajmanlove

Description

@ajmanlove

Rails 8.0.2, good_job 4.11.1

Seeing behavior where job hash parameters do not maintain order in async execution -- post serde.

ActiveStorage variants rely heavily on the order of variant arguments to generate deterministic variant keys. My first theory is that the issue may be rooted in good_job storing job params in a jsonb column. We use Postgres which does not guarantee maintained key order.

Given a variant defined like so:

  has_one_attached :file, dependent: :purge do |attachable|
    attachable.variant :preview,
       loader: { n: -1 },
       saver: {
         quality: 80,
         strip: true,
         density: 72
       },
       resize_to_limit: [ 750, 400 ],
       preprocessed: true
  end

I notice that the order of these variant arguments are expressed differently once in the ActiveStorage::VariantJob, when running with good_job as the async executor. (Logging here is from my own temp monkeypatch).

I, [2026-02-09T10:40:08.422723 #86143]  INFO -- : Performing ActiveStorage::TransformJob (Job ID: 461f229a-f000-41c0-8540-844be095561d) from GoodJob(transform) enqueued at 2026-02-09T17:40:07.417676000Z with arguments: #<GlobalID:0x0000000139a88ee8 @uri=#<URI::GID gid://mc-rails-api/ActiveStorage::Blob/6235850c-bead-4f2d-879c-6d20a58ff19e>>, {saver: {strip: true, density: 72, quality: 80}, loader: {n: -1}, resize_to_limit: [600, nil]}
I, [2026-02-09T10:40:08.423516 #86143]  INFO -- : [AS::TransformJob PATCH] PID=86143 transformations={saver: {strip: true, density: 72, quality: 80}, loader: {n: -1}, resize_to_limit: [600, nil]}

This results in the variant that is generated by the ActiveStorage::TransformJob having a different variant digest key than what would otherwise be generated by calling record.file.representation(:preview).processed in any other context.

I was able to work around this to achieve uniformity in all contexts with a crude monkeypatch (acknowledging that monkeypatches are inherently brittle).

Rails.application.config.after_initialize do
  require "active_storage"

  class ActiveStorage::Variation
    alias_method :__initialize, :initialize

    def initialize(transformations)
      canon = canonicalize_transformations(transformations)

      Rails.logger.debug(
        "[AS::Variation.encode PATCH] PID=#{Process.pid} " \
          "before=#{transformations.inspect} after=#{canon.inspect}"
      )
      __initialize(canon)
    end

    private

    def canonicalize_transformations(obj)
      # Use recursion to deterministically sort hash keys
      case obj
      when Hash
        obj
          .to_h { |k, v| [ k.to_sym, canonicalize_transformations(v) ] }
          .sort_by { |k, _| k.to_s }
          .to_h
      when Array
        obj.map { |v| canonicalize_transformations(v) }
      else
        obj
      end
    end
  end
end

This however prevents the ability to affect intentional operational ordering with the variant args.

Unsure what good_job should/could do to mitigate this, but raising the issue as it can result in major performance ramifications once systems start to use active storage representations / variants.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    Status

    Inbox

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions