Repository navigation
Support for chunked transfer encoding #1911
Description
Hi, I have a server that is responding with a http chunked response to load data as it's available (also it reduces the consumption of memory on the server):
https://en.wikipedia.org/wiki/Chunked_transfer_encoding
When I make a direct call with my browser I can see how the results pop-up as they are flushed from the server (as expected) however if I use hx-get I don't see anything in the screen until the full response is sent (not the expected behavior)
I see there is support for SSE and Web Sockets, is there a plan to support this feature also?
Reacted by Matthew Brandman, fnio5, Bradley Kemp, Rune K. Svendsen, Vladimir Shchur, Rian Dolphin, Shritesh Bhattarai, Woojong Koh, Evangelos, maciekkreft and 3 more
Activity
nickchomey commentedon Oct 20, 2023
This doesn't answer your question, but have you considered using the newer Streams API instead of chunked transfer encoding?
https://developer.mozilla.org/en-US/docs/Web/API/Streams_API
You might also consider using a service worker to fix this problem more immediately than htmx might get around to it - it'll intercept all network requests and you could deal with it as needed (as well as do LOTS more)
carlos-verdes commentedon Oct 23, 2023
I use that API to read from the server when doing SPA development, that's why I'm asking if HTMX will support this feature or not
The good thing about it is you don't need to change the protocol and most of HTTP and actually HTMX just works, the only problem is behavior is not as expected (HTMX waits for full response to be sent before rendering anything back to the user)
joetifa2003 commentedon Nov 6, 2023
I really need this feature (http streaming).
sjc5 commentedon Nov 14, 2023
It would be lovely to see this supported
jtoppine commentedon Nov 14, 2023
+1
Example use case: Streaming search results. Instead of complex "infinite scroll" or other types of convoluted result delivery schemes, we could just start pushing results to the client as soon as we get first hits from the database - client could start loading and rendering images etc for the first entries in listing right away. Ohh, it would be so straightforward and beautiful, so old school in the bestest of ways.
carlos-verdes commentedon Nov 16, 2023
I send all collections from my backend using streams to avoid also memory pressure on big collections so for me is a natural thing to do
alexpetros commentedon Dec 20, 2023
Had to close the above PR but I think the existing extension mechanism should be more than sufficient to implement this. Would love if someone wanted to take that on.
douglasduteil commentedon Dec 26, 2023
@alexpetros I'm on it ;)
carlos-verdes commentedon Dec 26, 2023
Are you doing extension for this @douglasduteil ?
Can you share link to the PR when ready?
douglasduteil commentedon Dec 26, 2023
🎉 https://github.com/douglasduteil/htmx.ext...chunked-transfer 🎉
\to @carlos-verdes it's christmas time again 🎄
Install
Usage
mattbrandman commentedon Feb 4, 2024
I don't know if there is a plan to add this into the base of HTMX or rely on extensions but I thought this is an example of the functionality folks are looking for https://livewire.laravel.com/docs/wire-stream. The ability to append vs replace is a nice addition as well.
I would bet a good deal of this is around streaming back AI based content. While using SSE or webhooks is an option it adds complexity to infra depending on the surrounding infra. The chunked transfer encoding feels clean because once all the data is sent the connection is closed vs SSE needing to be replaced to be closed with no real "polite close" or having to deal with websockets in general.
The other simplicity comes from not having to deal with channels for SSE or websockets on the server for multiple client, when you want to send back to only the sender the websocket / sse solutions feel heavyweight.
JEBailey commentedon Apr 25, 2024
I have an ES5 version of an extension that supports the chunked encoding that I've been using internally at the company I work for.
https://github.com/JEBailey/htmx/blob/master/src/ext/chunked.js
alexpetros commentedon Aug 9, 2024
So I originally closed #2101 because 2.0 was coming up and we weren't going to make that happen in time. I'm seeing some compelling use-cases and @douglasduteil's extension looks like it's been working. Are people using it? What's the case for including this in core?
fabge commentedon Aug 9, 2024
one usecase would be a fairly simple chatbot app which supports streaming.
no need for websockets, no need for server-sent events, no need for keeping a connection to the server.
you could simply make a request to the server, the server sends a
Transfer-Encoding: chunkedresponse, which will then incrementally be swapped/added into the respective chat bubble.runeksvendsen commentedon Aug 10, 2024
Hi @alexpetros,
Yes, I'm using @douglasduteil's extension in https://github.com/runeksvendsen/haskell-function-graph and it's working for me. Thank you @douglasduteil!
The case for including it in core is that it solves a very generic problem: you don't want the user to wait for the very last part of your page to be received until the first part of the page is shown. The larger the time difference between the backend having the first and last result available the worse this problem is.
In my case, I have a page that includes in the following order: (1) a list of results, where the first result is usually available to the backend very quickly (within a few milliseconds) and the last result can take an additional ~second to become available; followed by (2) an SVG graph that's slow to generate, because it calls out to a CLI executable. Without this extension, the user has to wait around ~two seconds to see the first results, even though they're available to the backend (and sent to the client) almost immediately.
22 remaining items
alarbada commentedon Dec 10, 2024
@carlos-verdes man I changed my stacks back and forth this year, the current thing I've settled with is go + my own thing that compiles handlers into ts sdk + astro with htmx and solidjs.
I know it looks like a pile of random tools but in my pet project works like wonders :D
htmx is my spa router, solidjs is for client side interactivity, astro is the backbone of all, go is there because I hate backend js with a passion.
If I was to program this streaming thing with this stack I would just stream jsons in go and manually parse them in solidjs in this setup, and because htmx doesn't reload the page my solidjs state is still there, even if the solidjs html is replaced by htmx.
wjkoh commentedon Jan 7, 2025
Hi @1cg, I just read The future of htmx and thanks for your insights! I agree with many points, and "No New Features as a Feature" resonates deeply with me. By any chance, do you think it's worth having HTMX support Chunked Transfer Encoding before it goes into the stable state? Websocket and SSE are not always the best choice, and HTTP/2 and its streaming feature seem too far out at the moment. Chunked Transfer Encoding might be the best standard bet we have for ChatGPT-like unidirectional text streaming. Would you mind sharing your opinion? Best, Woojong
wjkoh commentedon Jan 12, 2025
Just FYI, gRPC-gateway also uses Chunked Transfer Encoding to implement the server-to-client streaming. Check out https://github.com/grpc-ecosystem/grpc-gateway/blob/4aa9147d64085aabcd2af3dc59c02c60cf5e365d/runtime/handler.go#L31 .
alexpetros commentedon Jan 13, 2025
Nothing in the future of htmx essay precludes using chunked transfers (since it would basically be an invisible upgrade). The thing that keeps chunked transfers from happening is that that htmx is that, as others have pointed out, htmx waits for the whole response to come in before performing the swap, which is simpler to code than a buffered stream.
I don't think we have any ideological objection to it, other than needing it to work in a way that doesn't cause regressions or add major complexity to the code. That having been said... I haven't seen any compelling reasons why the @douglasduteil extension isn't sufficient. Maybe we just need to promote it alongside some of the other major ones.
runeksvendsen commentedon Jan 14, 2025
@alexpetros the most compelling argument I've seen is that it breaks progressive enhancement, which HTMX claims to support. Ie. boosting a link changes the behavior of a web page if the endpoint delivers the response using chunked transfer encoding. See my examples here #2789.
alexpetros commentedon Jan 15, 2025
The server is something you control, not a fixed entity. htmx doesn't support chunked transfer encoding, so if you're using it for non-boosted links, then you should turn it off it you detect a boosted one. It doesn't "break" anything, it's just something that htmx doesn't currently support.
This also isn't an argument for why adding
hx-extisn't sufficient. It seems like that does work fine for everyone.wjkoh commentedon Nov 5, 2025
Now that htmx 4.0 switches to fetch(), can we get htmx to support chunked transfer encoding as well? https://four.htmx.org/
carlos-verdes commentedon Nov 5, 2025
@wjkoh from the release notes I see this will be supported:
Lanayx commentedon Apr 21, 2026
@1cg Can chunked encoding support make it to fourth version or become default extension?
1cg commentedon Apr 21, 2026
1cg commentedon Sep 7, 2026
OK folks, we now, finally, have a multi-part core extension for this use case:
https://four.htmx.org/extensions/hx-multipart
By @scriptogre
Lanayx commentedon Sep 7, 2026
@1cg I don't think it's a replacement. Chunked encoding is just a regular html response, but without specifying content length, so server continues sending until the terminal chunk, it doesn't require multipart content type and special response encoding.
https://dev.to/sahan/deep-dive-how-chunked-transfer-encoding-works-4o9n
carlos-verdes commentedon Sep 7, 2026
I agree with @Lanayx, actually this solution is the opposite of the problem I was trying to solve.
One example is to send candles for trading from the server, I don't want to send the full set of data in a single shot that's why I use chunked transfer, this helps both backend and frontend to avoid memory pressure on big data sets (as we just hold the next chunk in memory).
Still this extension can be useful for some use cases and it's welcome.
1cg commentedon Sep 7, 2026
heya carlos, you can do exactly that with multi-part: stream down new content after an initial bit
i understand wanting to support chunked too and if someone wants to pick up and port the extension linked above (should be much cleaner in 4) I'd be happy to list it as a community extension for four, but multi-part, Web Socket & SSE are our core extensions for streaming HTML.