streaming interface for get

[RichardWright]

Hi

Would it be possible to add support for a nodejs streaming interface? (https://nodejs.org/api/stream.html)

Thanks

[leibale]

This is impossible with RESP2, and will be possible only when Redis implements RESP3 "Streamed String"

[karankraina]

@leibale While it is not possible currently with RESP2, can we add a helper function in the library that uses range command to get chunked values from redis and return a nodejs readable stream?

I tried to implement this in ioredis, but that is no longer actively maintained. I can open a similar PR here, if it looks okay. Thoughts?

Reference PR - redis/ioredis#1922

[leibale]

@karankraina That's an interesting idea! :) One concern I have is whether we can detect if a key changes "mid-stream" and handle it appropriately. For instance, if we have a key with a length of 1024 and we're reading in chunks of 64, what happens if someone runs a SET command after we've already read two chunks? Will we be able to detect the change and abort the stream?

(Also, just to note: streamed strings aren't currently implemented in Redis [the server], so even after RESP3 support is available on the client side, we'll still need to wait for the server to implement streamed strings before we can fully support a readable stream.)

[karankraina]

@leibale I initially thought about that too. But I'm afraid I dont have any clue if we can do that currently. I think, for now, we can maybe have a warning message on getStream and let end users handle this, no ?

[karankraina]

Hi @leibale ! Should we go ahead with this ?

[leibale]

@karankraina if we can't detect/handle it:

1. how do we expect the "user" to handle it?
2. the warning will always be there (as we can't detect it)

[karankraina]

@leibale By handling I mean, users should own this part about their redis key. As there is no way for the lib to know if a key value has changed in redis, users must only use this method if they are sure that the data wont be updated. Like for example, a static cache.