How to Get imageSampler of Widgets Stacked Under the Builder

[Ansh-Rathod]

Description

I'm encountering a challenge with obtaining an imageSampler of widgets that are stacked under a builder in Flutter. Specifically, I need guidance on how to capture and use the imageSampler for the widgets that are positioned below the builder widget in the widget tree.

Context

In my Flutter application, I am using shaders for custom rendering effects. I have a setup where multiple widgets are stacked using a Stack widget. Within this stack, I utilize a builder to manage the shader rendering. The goal is to access an imageSampler that includes the visual representation of all the widgets underneath the builder, so the shader can apply effects based on this combined image.

Example Scenario

Consider the following simplified widget structure:

Stack(
  children: [
    Positioned(
      child: SomeWidget(), // Widget that I want to include in the imageSampler
    ),
    Positioned(
      child: AnotherWidget(), // Another widget to include
    ),
    ShaderBuilder(
      builder: (context, shader, child) {
        // Shader rendering logic
      },
    ),
  ],
)

In this scenario, I need a way for the ShaderBuilder to capture an image of SomeWidget and AnotherWidget so that the shader can use this image as an imageSampler.

Expected Behavior
The shader should be able to access an imageSampler that represents the combined visuals of all widgets stacked below it. This would allow the shader to apply effects based on the overall image composition of these widgets.

Actual Behavior
Currently, I am unsure of the best approach to achieve this. I need guidance or examples on how to capture the visual output of the stacked widgets as an imageSampler.

Steps to Reproduce

1. Create a Flutter application with a stack of widgets.
2. Add a ShaderBuilder to the stack.
3. Attempt to capture an imageSampler for the widgets below the ShaderBuilder.

[Ansh-Rathod]

i think if we can add BackdropFilter logic here we can maybe get expected result. tried myself but I maybe wrong.

  ui.Image _buildChildScene(Rect bounds, double pixelRatio) {
    final ui.SceneBuilder builder = ui.SceneBuilder();
    final Matrix4 transform =
        Matrix4.diagonal3Values(pixelRatio, pixelRatio, 1);
    builder.pushTransform(transform.storage);
    addChildrenToScene(builder);
    builder.pop();
    return builder.build().toImageSync(
          (pixelRatio * bounds.width).ceil(),
          (pixelRatio * bounds.height).ceil(),
        );
  }```

[jonahwilliams]

You can't use custom shaders as a backdrop filter yet, see: flutter/flutter#132099. (No ETA, lots of other work).

The only way to make this work would be to make the stack a child of the ShaderBuilder

[Ansh-Rathod]

yet, but if I try to explore the logic of the backdrop filter and get the image sampler of the back area would it be possible? i tried but couldn't figure it out on my own.

[jonahwilliams]

No, it won't be possible to do without engine support.

[Ansh-Rathod]

oh alright! i should look for the other options.

[jonahwilliams]

Support for this was added to the engine, but you need to use the new ImageFilter.shader and not this package.

[Ansh-Rathod]

this is cool