Error: Premature close in streaming mode, seems it is related to timeout

[Istiopaxx]

Confirm this is a Node library issue and not an underlying OpenAI API issue

• This is an issue with the Node library

Describe the bug

My code use chat completion api create func with stream:true, and takes so long time: over 10 minutes(default timeout setting).

const stream = await this.openaiApi.chat.completions.create(
  {
    model: 'gpt-3.5-turbo-16k',
    messages: history, // an array of chat history
    stream: true,
  },
);

This request makes long -over 10 minutes-stream and after 10 minutes the request fails with error:

Error: Premature close
    at IncomingMessage.<anonymous> (/home/dev/refri_be/data-process/node_modules/node-fetch/lib/index.js:1748:18)
    at Object.onceWrapper (node:events:631:28)
    at IncomingMessage.emit (node:events:517:28)
    at emitCloseNT (node:internal/streams/destroy:132:10)
    at processTicksAndRejections (node:internal/process/task_queues:81:21) {
  code: 'ERR_STREAM_PREMATURE_CLOSE'
}

I tried to chage timeout setting by passing timeout: 20 * 60 * 1000 both in create function and OpenAi constructor, but not worked. I checked #294 but nothing was helpful..

1. Timeout setting not working for streaming seems bug.
2. If node-fetch's Premature close error is due to streaming timeout, this error should be handled by this library and transfer to proper error, e.g. APIConnectionTimeoutError.

To Reproduce

1. prepare so long, long time-taking prompt. It seems languages other than english takes so long times for response 🥲
2. call OpenAI.chat.completions.create with stream:true and wait for 10 minutes
3. Then Error: premature close thrown

Code snippets

No response

OS

macOS sonoma

Node version

18.18.2

Library version

4.14.2

[rattrayalex]

Thanks for reporting this. Can you paste an error that is thrown after you set timeout to 20 minutes?

Can you share a prompt that takes this amount of time for repro?

[Istiopaxx]

Thanks for reporting this. Can you paste an error that is thrown after you set timeout to 20 minutes?

It also Premature Close error. This error thrown after 10 minutes, which means timeout setting(20 minutes) passed is not working.

Error: Premature close
    at IncomingMessage.<anonymous> (/home/dev/refri_be/data-process/node_modules/node-fetch/lib/index.js:1748:18)
    at Object.onceWrapper (node:events:631:28)
    at IncomingMessage.emit (node:events:517:28)
    at emitCloseNT (node:internal/streams/destroy:132:10)
    at processTicksAndRejections (node:internal/process/task_queues:81:21) {
  code: 'ERR_STREAM_PREMATURE_CLOSE'
}

Can you share a prompt that takes this amount of time for repro?

This is minimal reproduction: try this at api-busy time.
I tried this just few minutes ago but takes 9 minutes.. Maybe now is non busy time, I guess.

https://github.com/Istiopaxx/openai-node-bug-report

[rattrayalex]

Interesting, that script runs for me in 72s. I may have a hard time reproducing this from where I sit in the USA.

Can you confirm that the same requests made over curl don't suffer from the same problem? I think this might be a server-or-gateway-side timeout rather than a client timeout.

[Istiopaxx]

Can you confirm that the same requests made over curl don't suffer from the same problem? I think this might be a server-or-gateway-side timeout rather than a client timeout.

I tried curl, and it also stopped after 600s. I checked my home gateway timeout setting but i found no timeout setting related.
Certainly, there is no problem for this library.. I'll close this issue.

I searched for this topic so much but could not find any helpful solution. I guess openai api's tier system makes response of my api calls slower(cause i'm in tier 2), and probably 600s timeout can be related to that. Nothing is certain about this 😢

Searched pages:
https://community.openai.com/t/anyone-facing-gpt-3-5-turbo-api-delays/460467/39
https://community.openai.com/t/gpt-3-5-api-is-30x-slower-than-chatgpt-equivalent-prompt/423902
https://community.openai.com/t/openai-why-are-the-api-calls-so-slow-when-will-it-be-fixed/148339/39

[rattrayalex]

Thank you @Istiopaxx, and sorry to hear about your difficulties! I might recommend opening a new thread at community.openai.com about the 10min API timeout affecting you.