[Triage] dirty protocol fails with binary data

[DeoLeung]

Type

Bug Report

Description

rapidocr-1  |   File "/usr/local/lib/python3.13/dist-packages/gunicorn/dirty/client.py", line 257, in execute_async
rapidocr-1  |     await DirtyProtocol.write_message_async(self._writer, request)
rapidocr-1  |   File "/usr/local/lib/python3.13/dist-packages/gunicorn/dirty/protocol.py", line 159, in write_message_async
rapidocr-1  |     data = DirtyProtocol.encode(message)
rapidocr-1  |   File "/usr/local/lib/python3.13/dist-packages/gunicorn/dirty/protocol.py", line 68, in encode
rapidocr-1  |     raise DirtyProtocolError(f"Failed to encode message: {e}")
rapidocr-1  | gunicorn.dirty.errors.DirtyProtocolError: Failed to encode message: Object of type bytes is not JSON serializable

Steps to Reproduce (for bugs)

send bytes to dirty app
document indicate it supports https://gunicorn.org/dirty/#example-image-processing
but looks like it does a json.dumps before, which makes it fail

Configuration

No response

Logs / Error Output

No response

Gunicorn Version

25.0.1

Python Version

3.13

Worker Class

custom

Operating System

linux

Additional Context

No response

Checklist

• I have searched existing discussions and issues for duplicates
• I have checked the documentation and FAQ

[benoitc]

Yes this is expected. Look at the example it is encoding the binary as base64 to ensure it pass. It's a limitation in using json: https://gunicorn.org//dirty/#best-practices_1

It's possible to replace change the protocol to not use json, but this would makes it slightly more complicated. Thoughts?

[DeoLeung]

argh, i saw the number 5 :) , i was reading the code and saw it using BytesIO :(

if external deps is ok, i think could be using msgpack or msgspec for binary ipc

btw, how to integrate with pytest that testing sending request to api and it calls the app, i did monkeypatch, but it pass my input directly to the app worker(which bypass the json se/deserialzation, hence pass in test but find it fails in real)

[marcinkuzminski]

I think for stream responsens using bytes would be better ootb, you don't need to encode/decode between streaming to clients, and communication between main process and dirty workers.

ideally the protocol could be pluggable, i'd also use msgpack for this to keep all things binary

[benoitc]

not sure about having the protocol pluggable, that would make support difficult. I took JSON as the easy path, but it's scertainly doable to make the protocol fully binary and mayber emove some extra steps in the process. I will create an issue from it

[benoitc]

Can you test #3500 which implements it ? Let me know.

[benoitc]

merged in main :)