No easy way to support async stream

[vadixidav]

In the last two years, I have encountered this issue at least three times and I have worked for days and never came up with an answer, including after talking with the team here. My current issue is that I need to be able to support a serial port that is transformed into an async stream of data by an API, which I then graph in iced. I have a dynamic number of these serial ports, and each may have a different protocol associated with it. They can be spawned dynamically in the UI. Let me explain why it is so difficult with the current API to support this use case without resorting to polling using time::every.

The first thing that is suggested is to support this with a Subscription. The subscription has a hash associated with it which allows it to continue running so long as the hash does not change. If it does change, then the previous subscription is removed and replaced with the new one. However, an issue arises. If I create a Subscription in any way, such as with subscription::Run, then I may also create the underlying Stream without knowing whether or not the Stream will be used. I have a stateful serial port which talks to embedded devices, so if I try to create a Stream, I will have to open up a serial stream and several intermediary things that perform the communication back and forth and mediate the creation of messages via a protocol. These things cannot be instantiated multiple times, and it will fail if I attempt to create a second stream. Even if I could create a second stream, it is inherently stateful, with multiple intermediary state machines. Note that this could still be implemented with a Recipe so long as the Recipe didn't form the Stream until it was asked for.

My suggestion to the team for how to make it simpler for people to use Subscription would be to add a new API. The iced_native::subscription module should contain a new function that takes three arguments, the first some data, the second a closure that produces the hash ID, and the third a closure that produces the BoxStream. Ideally, if this API could be added directly to Subscription, I think it would solve a lot of headaches. You could add a non-stream Subscription method that just uses a future that returns a tuple with the message and some other arbitrary type that could be the remainder of the stream (via Stream::into_future). I am not sure why the base subscription module doesn't have the run API on it, but as mentioned, even the run API is not sufficient for many use cases, including every use case I have used it for thus far.

Another way would be if you could just add subscriptions (as simple async Stream) statefully and remove them using an ID when they are no longer needed inside of update(). Since this isn't in the spirit of iced I assume this won't be done, but it would make implementation substantially easier for me.

The second thing that may be suggested is to handle this in update() with Command. Unfortunately, it seems that Clone and Debug is a requirement for the Application::Message. This prevents me from using Command::perform and putting the output of the stream and the rest of the stream with StreamExt::into_future into the message. If the Application::Message could be relaxed to not have this Clone + Debug requirement, this would resolve this issue.

iced already has full control over my application, so I can create the stream at any point that iced would like, but I have been unsuccessful in the last day in finding a way to make this work, and I have only narrowly come to a solution. None of the examples covered this common case. I have referred back to my previous conversations with @hecrj, but none of the solutions work for this particular application. To be clear, it took me quite a long time to figure out how to make this work despite substantial effort. Ultimately, many resort to simply polling a synchronous channel with infinite bound, as I have in my last 6 iced projects, and moving the statefulness out of iced. At the very least, I think there must be a better way to construct this API. Based on what I am seeing in the discussions and what I have seen in the issue tracker, it seems many don't leave with a solution on how to use Subscription. The built-in subscriptions (like every) do work, but they are simple in nature. Many other applications depend on stateful elements of servers, connections, IPC, files, and other things. Nonetheless, UIs do still need to be created for these things. As it stands, I haven't found a UI framework that works nearly as well and easily as iced, and so, despite the difficulty, I still find it best to use iced to create my applications even when it doesn't fit as well. I think changing the Subscription API to support this use case would be of great value for people like me so that we don't need to implement the Recipe trait or resort to channels.

This would also resolve #1310.

[vadixidav]

I also will add that I ran into an additional issue in my case. I need Recipe::stream to be an async fn or at least allow me to return a closure that returns a Future which can be .await on by the underlying implementation. I can work around this, but it further complicates the usage.

[hecrj]

subscription::run is meant for leveraging the Stream API directly.

We don't need a broad Subscription API when Stream already has one.

[vadixidav]

The problem is that I would like it if run didn't create the Stream until after it has verified that it does need to create it. Otherwise, I can't create the Stream.

That being said, what I ultimately did was create a task with tokio, receive things from one stream, send them in an mpsc through another stream, and then pass that final stream out the other side. I will have to do it this way (though inefficient) until there is a good way to make it so that the stream doesn't need to be created except when it doesn't already exist.

[hecrj]

You can create a Stream without doing anything until it is actually polled (i.e. spawned by the runtime).

The echo module in the websocket example is a great example of this. The Subscription and underlying Stream is created without connecting the websocket first. Instead, the stream starts in a disconnected state and is only connected when/if the Stream is run.

You should already be doing something like this if your serial connection can fail or be closed unexpectedly, so you can recover and reconnect inside the Subscription.

Also notice that unfold is just a helper that leverages run internally, which is why I said:

We don't need a broad Subscription API when Stream already has one.

[vadixidav]

The first thing I would say is that it would be much easier if the run API took a future (could be a BoxFuture) that resolves into a BoxStream. Otherwise, it is necessary to implement Recipe for my use case.

Secondly, I have had other issues. For instance, the way COM ports work on Windows is very odd, so it isn't possible to reconnect to the device immediately. Additionally, there is a sending side of the COM port, which is independent of the incoming side, and I need to be careful to manage both of them to statefully interact with the device such that they don't act out of sync so that they both have to run in a task that is spawned with tokio when the stream is created.

I want to add though that my experience, even despite these challenges, has been that iced is really powerful, and I do appreciate the fact that iced does try to facilitate things like re-connections. While it has challenges with weird stateful things, those challenges have to be dealt with at some point, and iced is just asking me to deal with them up-front. Overall, the composability of the UI is great.

My only challenge thus far has been that I had a UI element that existed on a per-connection basis. But I had trouble adding buttons to the per-connection UI since they have the iced::button::State which must be borrowed mutably. It seems to me that upcoming versions will use interior mutability, so I am excited for those changes (presumably 0.5?).

Also, thank you for listening, and I hope this can be improved.

[hecrj]

The first thing I would say is that it would be much easier if the run API took a future (could be a BoxFuture) that resolves into a BoxStream. Otherwise, it is necessary to implement Recipe for my use case.

Secondly, I have had other issues. For instance, the way COM ports work on Windows is very odd, so it isn't possible to reconnect to the device immediately. Additionally, there is a sending side of the COM port, which is independent of the incoming side, and I need to be careful to manage both of them to statefully interact with the device such that they don't act out of sync so that they both have to run in a task that is spawned with tokio when the stream is created.

You can use unfold and create a Stream state machine for all of this.