Hot reloading of layout

[PvdBerg1998]

Like discussed on Discord - perhaps it's nice to provide some sort of "quick iteration" possibility, for example using Lua.

[krooq]

I think this is an essential feature for designing any kind of complex UI.
It takes me 30 seconds to recompile my 90 line main file when I make a small change to the view which is just far far too long for iterative development.

EDIT: I have done a lot of work with Flutter and it's hot reload is amazing. I think nothing that extensive is needed but just reloading the view on the fly would be great.

[clawoflight]

It might be worth thinking about clever imgui integration instead, at least in the short run?

[ethanpailes]

#303

[tangmi]

I'm taking a stab at this and I'll upload what I have once I get a bit more working, but I'll try and enumerate what I've learned so far.

I think this requires two main components:

1. Markup/templating language: Something like XAML, JSX, Handlebars + HTML, etc. I think this is a very useful component for iced to have, regardless of hot swap capabilities.
   • The template engine needs to support some control flow (if, for each, etc) based on user state.
   • The runtime component of the markup language needs to:
     • Resolve custom widgets into iced::Elements.
       • This might need widgets to have some declarative metadata describing attributes and events, which would allow custom and "builtin" widgets to be treated the same
     • Store/reference widget state. (Note: This could happen automatically! It'd be an awesome gain for iced user producivity!)
     • Reference message construction functions for callbacks.
       • This probably needs reflection (codegen or runtime).
         • I'm currently playing around with a HashMap<String, &dyn Any> that the app fills on initialization and downcasting to the exact function pointer type, but am running into an issue with Any::downcast to a function pointer.
         • Also going to look into leveraging serde's codegen to generate messages.
   • This could probably be built as a library, given changes in iced for support.
2. Hot reload runtime component: This will actually do the hotswapping of the resolved markup and ends up being pretty trivial (file watcher with an optional code recompiler) once we have the markup language.
   • This can probably be done as a library, with minimal or no support needed in iced itself.
   • I can imagine a setup where the layout itself is reloaded from the markup language (almost) instantly and when state/callbacks change the entire app could be recompiled and the binary hotswapped in, preserving app state.

Some additional notes:

• My current prototype uses ron as the markup "language" and can currently generate/reload non-interactive stateless views. Most features I described above are not implemented.
• It'd be nice to eventually include the markup language and reloading as "blessed" crates.
• I'm currently ignoring the issue of how to make this zero/low overhead in release builds.

@hecrj What are your thoughts on an iced layout markup language? I'll happily file an issue with all the details/prototype I have if you're into it.

[tangmi]

https://github.com/tangmi/iced-hotswap-prototype

Here's a working proof-of-concept of the following features:

• Ron-based markup language templated with handlebars
  • currently supports what's in iml::Element: Column, Row, Text, Button, Slider (with minimal parameters)
  • resolves templates with user-provided state
• automatically creates, owns, and maintains widget "ephemeral" state (i.e. button::State)
• resolves callbacks with user-provided "message" struct
• can read values from user-provided state

Some caveats:

• I'm heavily leveraging serde's code generation to e.g. discover the variants of a user-provided messages enum.
• A lot of features are missing
• no errors are handled gracefully (if you spell your markup file wrong, it'll immediately crash).
• I'm abusing std::mem::transmute in a couple places to avoid "solving" where to put lifetime annotations
• After saving the file, one needs to interact with the app (mouseover) to get it to reload. This could be fixed by implementing a Subscription that watches the file system I've added a Subscription that just wakes up the app every half second to force a rerender. (there's definitely a smarter way to do this).

[hecrj]

@tangmi Hey! That's really cool!

I really like how it keeps update code type-safe. I imagine we could build our own Application trait wrapping iced::Application to hide the Hotswappo struct and view and provide the state directly to update. This should also help us use a different strategy for release mode (ideally generating Rust view code from the template and using the iced::Application directly for no runtime cost).

I would also probably rename the various callback_name to the actual widget method names (i.e. on_press for a Button, etc.), but I imagine the details of the language are not very important right now.

Overall this is very, very promising! It could ease development process a lot and help us build view editors!

[hecrj]

automatically creates, owns, and maintains widget "ephemeral" state (i.e. button::State)

It will be very tricky to do this well without an important performance impact and logic errors (i.e. diffing).

At least for the time being, I would require the user to define widget state in the application state and provide it in the markup. Then, if the user adds a new widget in the template and the state field doesn't exist, we can fall back to the current naive tracker for hot reloading. However, I would make it a requirement for release builds (i.e. compilation error if the state fields do not exist).

[tangmi]

Glad you like it! Thanks for the feedback!

I like the idea of hiding Hotswapo behind a separate Application trait opening up the possibility for transparent hotswapping in debug builds for the user.

For the markup side, I was imagining two "runtimes":

• Dynamic: one with a similar lineage to the prototype--intended to load dynamically and have additional hot-swapping abilities at the cost of performance depredations.
  • This is the one hotswapping would use.
• Static: a compile-time affair that generates rust code for the view (that includes the branching as rust control flow) and a struct that contains the ephemeral state (please correct me if there's a better name for this) of exactly the widgets that appear in the markup file.
  • In this case, I don't think the user needs to provide widget state, which is an otherwise very mechanical action (every widget one adds needs its associated state, it's very unlikely that two separate widgets want to share the same state, etc).
  • This is the one non-hotswapping (release builds?) would use.

In both cases, the result would be a function something like fn view(&loaded_layout, &mut user_state) -> Element<'_, M>. I might be handwaving the difficulty of implementation/maintaining cost this dual-action runtime, but I believe some combo dynamic/static layout file solution gets us both perf and flexibility.

[aevyrie]

@tangmi I was about to try something similar, nice work! For my needs, I was thinking of a different, possibly simpler, approach. I'll expand on that here in the hopes that it adds to this discussion.

Instead of generating any rust layout code, my priority lies in hot-reloading the styling of existing widgets. The plan was to define all styles and sizes in a single serializable object, I'll call StyleSheet for now. I would then reference this StyleSheet whenever I define a widget in an iced fn view(). Similar to CSS classes.

The hot swapping logic would live in an async loop constantly looking for edits to a file, then triggering an iced Command when an edit is detected. This Command would use serde to import this (TOML?) markup file containing the serialized StyleSheet object. This would allow me to rapidly iterate on the styling (widths, spacing, coloring, borders, etc) of my framed out UI. Once I'm happy with it, I can just drop this tweaked StyleSheet object into my rust code to remove the need for the hot-reloading runtime.

This approach obviously doesn't allow for generating view code on the fly, but I haven't found that to be an issue so far. Compartmentalizing groupings of reusable widgets into "components" that I can then drop into the top level UI has been painless, and makes the top-level code very readable when the entire gui isn't expressed in a giant nest of .push()s. I've also found that iced's Elmish way of defining layout makes it dead simple to get the layout, if not the styling, where I want it to be very rapidly.

I'm also skeptical of the utility of generated view code, because it quickly becomes customized - especially views that hold lists of things - with .filter()s and .fold()s. Once I've prototyped a gui and hook up all this logic, the next time I want to tweak my view code via hot-reloading, I'd have to manually reconfigure all this logic, which seems to defeat the purpose!

In my mind, that's like generating HTML, when what I really want is to just hot-reload my CSS.

With all that said, maybe what I have in mind isn't the right approach, or representative of most users' needs! I think this is really important work for iced and I'm glad you're tackling it. 🙂

[tangmi]

@hecrj I'll keep poking at this when I can find time. I think the main takeaway from my investigation is that both a markup language and hotswapping can probably be developed as a library with no* changes in iced needed (great work on library design!).

*I think some mechanically consistent declaration of widgets and their state/attributes (not sure what this looks like!) can help simplify the markup language implementation and allow for custom widgets.

[hecrj]

I think some mechanically consistent declaration of widgets and their state/attributes (not sure what this looks like!) can help simplify the markup language implementation and allow for custom widgets.

Could you elaborate? What do you mean by this?