How to stream.pipe response in hapi v17

[toddhickerson]

In hapi 16, we were doing this:

stream.pipe(request.raw.res)

In hapi 17, this works, but is waiting until the end to send the data:

  return h.response(stream)
    .type('application/pdf')
    .header('Content-type', 'application/pdf')
    .header('Content-length', stream.length);

Any ideas on how to make this work well in hapi v17?

Much appreciated.

[devinivy]

Can you explain a little more about what you mean when you say "waiting until the end to send the data"?

[toddhickerson]

Under hapi 16, we were able to pipe the stream on the response: stream.pipe(request.raw.res)

When sending large pdf documents, this allowed the first page to display to the user long before the rest of the document was completed rendering and sending.

Now with hapi 17, we are using a different syntax (see above), which does not behave the same way to the user. They don't see any pages until the whole pdf document has been sent by the server and received.

So, essentially, I'm looking for a way to "pipe" a stream on a hapi response. Does "h.response(stream)..." do that (if so, I'm not seeing it), or is there some way to adapt the hapi 16 technique we were using to hapi 17 with async/await?

[devinivy]

To my knowledge that should work just fine as-is. How can you tell that hapi's behavior has changed? If you need to take control of the response yourself you can always return h.abandon.

[yereby]

Hi,
I have the same issue. I want to stream via the EventSource api.

I receive the error and the open events. I cannot have the messages.
I see the logs "write data..." but don't get the messages. It looks like the response is not send

In my front js :

let source = new EventSource('/builds')
source.onmessage = event => { console.log('mess', event.data) }
source.onerror = event => { console.log('err', event) }
source.onopen = event => { console.log('open', event) }

In my server for the route /builds :

const channel = new stream.PassThrough

setInterval(() => {
  channel.write('event: message\n')
  channel.write('data: abcdef\n\n')
  console.log('write data...')
}, 1000)

return h
  .response(channel)
  .type('text/event-stream')
  .header('Connection', 'keep-alive')
  .header('Cache-Control', 'no-cache')

[kanongil]

@yereby Are you sure it is not because of missing gzip flushing? See hapijs/hapi#3599 for details.

[yereby]

@kanongil It is possible but i don't know how to cut it in v17.

A little step forward i added channel.end() in my code above just after the channel.write() and i manage to receive the message events.
But i got an error at each message, and the connection is closed and reopened each time.

It is somehow due to the sending response but i don't know how

[toddhickerson]

Does anyone have any ideas for the original issue of this post?

In Hapi 17, how do you do a stream.pipe(request.raw.res); ?

[isaiahtaylor]

My guess is that h.response "reads" a stream (converts to a buffer) rather than actually listening to and piping. Perhaps implemented as a fallback? So h.response is probably not the correct way to do it. Using request.raw.res is advised against in the docs, maybe there should be a feature request for a writeable.

[Marsup]

With all due respect, I'd say you were doing it wrong on hapi 16, trying to do it the same way again would be an error. @kanongil referenced an issue that was closed by a commit, have a look at its tests and try it this way.

[yereby]

I can confirm the other issue helped.
I added this line before the return of internals.transmit function in transmit.js :

request.raw.res.compressor = compressor;

And i can call it in my interval like this :

handler: (request, h) => {
  try {
    const channel = new stream.PassThrough

    setInterval(() => {
      channel.write('event: message\n')
      channel.write('data: abcdef ' + it.i + '\n\n')
      console.log('write data...', it.i)
      request.raw.res.compressor.flush()
    }, 1000)

    return h
      .response(channel)
      .type('text/event-stream')
  } catch(err) { throw err }
}

Indeed the response is flushed and i got my stream send.
@toddhickerson maybe this can help you.

But now i am wondering how to implement this the good way oO

[yereby]

Hello,

By defining the compression option in my server const server = new Hapi.Server({ port: process.env.PORT || 1337, compression: false }) it works, the response is not gzipped and flushed for the stream as i wanted.

But it is defined for the entire server.
I would like to deactivate the compression only for some routes. So i added a "content-encoding" header for my route like that

return h
  .response(channel)
  .type('text/event-stream; charset=utf-8')
  .header('Content-Encoding', 'none')

I am not sure that's the good way to do it. Somebody has some good practice ?

Thanks.

[johnparn]

@yereby Would it be possible to use the server.options.mime to override the compression for example for just the stream mimetype?

There is an example here https://hapijs.com/api how to override.

[yereby]

@johnparn I believe that would serve to override all mimetype, and not for juste one route.
But it is quite interesting, thanks.

[gitawego]

I've exactly the same prob. any ideal how to resolve it ? tried with compression = false, not working

[brzez]

Here\s a working / work in progress example for hapi16
https://gist.github.com/brzez/7ccb625f7f431b95f16f13b6c54c6b0e

The key is the SSEStream class - it flushes the compressor.