Text/Event Stream response doesnt work through Nginx

[cohen1280]

Select Topic Area

Question

Body

Hello, guys!
I am building the chatbot and built the backend as fastAPI. with /chat route, it returns streaming response. I deployed it on Ubuntu instance and configured nginx and connected to subdomain. But /chat route doenst work as streaming response. after finish all responses, it returned all responses at one time. This problem causes from nginx, when I was using it as IP address, it worked well as streaming response. I tried with several referenced related with nginx configuration, but I couldn't . Please help me with nginx configuration for Text/Event streaming

Thanks

[yusofy]

Hi @spyhunter1280 ,

Have you been able to resolve this? Running into the same issue.

[cohen1280]

I couldn't solve that problem, so I didn't use nginx.

[yusofy]

Figured it out with the help of colleagues . Running it in Kubernetes, I translated it back to nginx config:

location / {
    if ($request_method = 'OPTIONS') {
        add_header 'Content-Type' 'text/plain charset=UTF-8';
        add_header 'Content-Length' 0;
        return 204;
    }

    proxy_buffering off;
    proxy_http_version 1.1;
    proxy_max_temp_file_size 0;

    # Cors Preflight methods needs additional options and different Return Code
    chunked_transfer_encoding on;
    add_header 'Access-Control-Allow-Origin' $http_origin;
    add_header 'Access-Control-Allow-Credentials' 'true';
    add_header 'Access-Control-Allow-Methods' 'GET, PUT, POST, DELETE, PATCH, OPTIONS';
    add_header 'Access-Control-Allow-Headers' 'Authorization, Cache-Control, Content-Type, Keep-Alive, x-feature-events, responseType, stream';
    add_header 'Access-Control-Max-Age' 1728000;
}

Here are the annotations used:

  annotations:
    nginx.ingress.kubernetes.io/proxy-buffering: "off"
    nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
    nginx.ingress.kubernetes.io/proxy-max-temp-file-size: "0"
    nginx.ingress.kubernetes.io/configuration-snippet: |
       # Cors Preflight methods needs additional options and different Return Code
       chunked_transfer_encoding "on";
       more_set_headers 'Access-Control-Allow-Origin: $http_origin';
       more_set_headers 'Access-Control-Allow-Credentials: true';
       more_set_headers 'Access-Control-Allow-Methods: GET, PUT, POST, DELETE, PATCH, OPTIONS';
       more_set_headers 'Access-Control-Allow-Headers: Authorization, Cache-Control, Content-Type, Keep-Alive, x-feature-events, responseType, stream';
       more_set_headers 'Access-Control-Max-Age: 1728000';
       if ($request_method = 'OPTIONS') {
           more_set_headers 'Content-Type: text/plain charset=UTF-8';
           more_set_headers 'Content-Length: 0';
           return 204;
       }

The issue I had was that there was an extra nginx server in front I wasn't aware of that was blocking it.

The following had to be put in:

location / {
       chunked_transfer_encoding on;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-port 443;
        proxy_buffering off;
        proxy_http_version 1.1;
        proxy_set_header Connection "";

        proxy_pass XXXXX
    }

[asrorbekh]

Hello! I applied to my conf but not helped. I use fastify nodejs 20+. I got network error while connecting js client fetch but when i open it directly it send sse events.

[dellvolk]

Thanks, it helped!

chunked_transfer_encoding on;
proxy_buffering off;

[bertiejohnson]

Thanks a lot, sorted.

I was running a nuxt app and was thinking it was something to do with running the production build then twigged that under the ip address the stream worked but not under the domain ... thus the nginx set up was the obvious problem ..

chunked_transfer_encoding on;
proxy_buffering off;