Async generators support for server actions

[remorses]

Goals

1. Make streaming easier, currently you need to use a ReadableStream to stream data to the client, using a server action and yield you could just call for await of

Non-Goals

No response

Background

An example usage

export default function Streaming({ }) {
    async function* streamItems() {
        'use server'
        yield 'one'
        yield 'two'
    }

    return (
        <div>
            <button
                onClick={async () => {
                    const generator = streamItems()
                    for await (const item of generator) {
                        console.log(item)
                    }
                }}
            >
                Stream items
            </button>
        </div>
    )
}

Proposal

Async generators get converted to ReadableSteam of ndjson data

[Ektaros]

@remorses You can actually make this work with a little bit of wrapping.
Then you return a Promise inside some object from a Server action it return as Chunk (some internal type extending from Promise)
If you resolve this Chunk with another object with some data and another Promise you get another Chunk and can basically mimic an async generator.

Here is the server snippet with the wrapper:

'use server'

const sleep = (t: number) => new Promise(r => setTimeout(r, t))

export type StreamResponseChunk<T> = {
  iteratorResult: IteratorResult<T>
  next?: Promise<StreamResponseChunk<T>>
}

async function streamChunk<T>(generator: AsyncGenerator<T>) {
  const next = generator.next()
  return new Promise<StreamResponseChunk<T>>((resolve, reject) => {
    next.then(res => { 
      if (res.done) resolve({ iteratorResult: res })
      else resolve({ iteratorResult: res, next: streamChunk(generator) })
    })
    next.catch(error => reject(error))
  })
}

function streamResponse<T, P extends any[]>(createGenerator: (...args: P) => AsyncGenerator<T>) {
  return (...args: Parameters<typeof createGenerator>) => {
    const generator = createGenerator(...args)
    return streamChunk<T>(generator)
  }
}

export const generateData = streamResponse(async function* (a: number, b: string) {
  for (let i = 0; i < 10; i++) {
    await sleep(1000)
    yield a + b + i
  }
})

Here is the client side with 'unwrapper':

'use client'
import { useCallback } from 'react'
import * as api from './api/test'

function iterateStreamResponse<T>(streamResponse: Promise<api.StreamResponseChunk<T>>) {
  return {
    [Symbol.asyncIterator]: function () {
      return {
        current: streamResponse,
        async next() { 
          const { iteratorResult, next } = await this.current

          if (next) this.current = next
          else iteratorResult.done = true

          return iteratorResult
        }
      };
    }
  };
}

function Streams() {
  const run = useCallback(async () => {
    try {
      const start = Date.now()
      for await (const value of iterateStreamResponse(api.generateData(1, '1'))) {
        console.log('Elapsed: ', Date.now() - start, ' Got: ', value)
      }
    } catch (error) {
      console.error(error)
    }
  }, [])

  return (
    <button onClick={run}>
      Start the show!
    </button>
  )
}

export default Streams

[KareemMAX]

For some reason NextJS kept spitting "Error: A "use server" file can only export async functions.". I fixed that by adding async on streamResponse.

function streamResponse<T, P extends any[]>(createGenerator: (...args: P) => AsyncGenerator<T>) {
-  return (...args: Parameters<typeof createGenerator>) => {
+  return async (...args: Parameters<typeof createGenerator>) => {
    const generator = createGenerator(...args)
    return streamChunk<T>(generator)
  }
}

[mehdinourollah]

Even with above it's giving error on generateData "function" ...any idea?

Error sending message: Error: A "use server" file can only export async functions, found object.

[mehdinourollah]

NVM I found a better solution ...for anyone who is interested
https://github.com/jherr/streaming-sa-spotify/blob/main/src/app/action.tsx

[worldsayshi]

Even with above it's giving error on generateData "function" ...any idea?

Error sending message: Error: A "use server" file can only export async functions, found object.

This issue seems to specifically arise when exporting some of the utility functions, streamResponse and streamChunk. When exporting only functions that have return type Promise<StreamResponseChunk<string>> it works. So the utility functions can't be placed in separate file and has to be copied to any file that uses them. It seems like a bug in Next.js?

As hinted above and below there are some alternative solutions in the vercel ai sdk but they have changed and are still labeled as experimental and may change again.

Also, here's a mostly comprehensive example doing it the vercel way.

[Christopher-Hayes]

This worked great, tyvm.

[drxc00]

Still a goated response. Thanks!

[remorses]

Async genreators seem to work in server actions now! Just return the generator inside an async function

[mnahinkhan]

Could you please further elaborate on this? Is there a specific version that supports this?