PoC: SVG canvas #38602

yjbanov · 2023-01-02T23:27:16Z

A proof-of-concept for a canvas that uses SVG exclusively to render all pictures. Reuses HTML renderer's layer tree and text layout. Only replaces the PersistedPicture implementation. Does not implement things that are not implementable in SVG with expected correctness and performance characteristics: drawVertices and drawPoints.

This is currently in a state where it can render the gallery 99% correctly, but probably contains a long tail of small issues that would need to be fixed before this can turn into anything serious. In particular, this needs a higher coverage of Paint features, such as gradients.

Notes on performance

Tested on M1 MacBook Pro, iPhone 7, and Pixel 6 Pro.

Good news:

Eliminates virtually all 2d canvases (required [web] Fix canvas2d leaks in text measurement #38640).
Uses less GPU memory compared to HTML (on the Gallery drops from ~40 MB down to ~30 MB).
Just eyeballing the frame rate, feels competitive with the HTML renderer on Firefox and Chromium-based browsers.

Bad news:

In Safari there are cases where performance has severely regressed. One example is the Reply demo in the gallery. The transition when expanding and collapsing emails is super-janky.
The material text fields demo severely regressed on Android.

Next steps

I don't think this is ready for anything close to production. We should at least eliminate the obvious regressions. We could land this experimentally without enabling it, so we can iterate on it. All changes outside lib/src/engine/html/svg directory are meant to be non-breaking. They just generalize existing code so it can be used both in the HTML mode and SVG mode. If we decide to abandon this experiment, we can simply delete lib/src/engine/html/svg and a couple of lines that refer to it, and forget this ever happened. The actual "SVG canvas" part is ~450 LoC (svg_canvas.dart + svg_picture.dart).

The other thing is that this should not become yet another canvas, but a replacement for both dom_canvas.dart and bitmap_canvas.dart. If we can't get it to a point that it fully replaces the old canvases, we should drop it.

mdebbar

This is amazing! Thanks for hacking on this, @yjbanov!

mdebbar · 2023-01-03T17:45:57Z

lib/web_ui/lib/src/engine/dom.dart

+
+@JS()
+@staticInterop
+class DOMMatrix extends DOMMatrixReadOnly {}


Not specific to this PR, but can we use this and delete our vector_math.dart implementation? Seems to be widely available in modern browsers, and implements most (all?) of our Matrix4.

mdebbar · 2023-01-03T17:46:46Z

lib/web_ui/lib/src/engine/embedder.dart

+      // DO_NOT_SUBMIT: revert this before submitting.
+      // ..style.pointerEvents = 'none';


REMINDER TO NOT SUBMIT THIS.

fwiw, one trick for this kind of thing is to use // TODO: revert before submitting because then the todo style lint will make the analyzer shard red, preventing you from landing.

mdebbar · 2023-01-03T17:51:57Z

lib/web_ui/lib/src/engine/html/path_to_svg_clip.dart

-/// clip is set to "svgClip${_clipIdCounter}", e.g. "svgClip123".
-SVGSVGElement pathToSvgClipPath(ui.Path path,
-    {double offsetX = 0,
+class SvgClip {


Cleaner as a class. Me like it!

mdebbar · 2023-01-03T19:35:13Z

lib/web_ui/lib/src/engine/html/svg/svg_canvas.dart

+
+  @override
+  void drawRRect(ui.RRect rrect, SurfacePaintData paint) {
+    // TODO(yjbanov): this could be faster if specialized for rrect.


One optimization we could do is use an SVGRectElement and change its rx and ry. Flutter's RRect is more flexible (e.g. each corner can have its own radius), so this optimization can only be applied in certain cases. But I would argue that in most cases, rrects are symmetric, which makes it hit this optimization case.

mdebbar · 2023-01-03T19:42:49Z

lib/web_ui/lib/src/engine/html/svg/svg_canvas.dart

+      imageElement.setAttribute('clip-path', clip.url);
+      imageElement.style
+        ..transformOrigin = '${dst.left}px ${dst.top}px 0'
+        ..transform = 'scale(calc(${dst.width / src.width}), calc(${dst.height / src.height})) '


I don't think calc(..) is necessary here? You aren't asking the CSS engine to calculate anything. You are giving it a number.

Or maybe you wanted to do this:

..transform = 'scale(calc(${dst.width}/${src.width}), calc(${dst.height}/${src.height})) '

mdebbar · 2023-01-03T20:14:14Z

lib/web_ui/lib/src/engine/text/canvas_paragraph.dart

+        // the Y coordinate. However, Flutter positions the top edge of the text
+        // line at the Y coordinate. For the SVG text to match what Flutter
+        // expects, the baseline is added to the Y coordinate.
+        spanElement.addY(boxRect.top + line.baseline);


(I think you already did this but haven't pushed it yet)

Suggested change

spanElement.addY(boxRect.top + line.baseline);

spanElement.addY(line.baseline);

ditman · 2023-01-03T20:53:32Z

lib/web_ui/lib/src/engine/embedder.dart

-    _sceneHostElement = domDocument.createElement('flt-scene-host')
-      ..style.pointerEvents = 'none';
+    _sceneHostElement = domDocument.createElement('flt-scene-host');
+      // DO_NOT_SUBMIT: revert this before submitting.


(Just to make the PR "red" :P)

ditman · 2023-01-03T23:01:08Z

lib/web_ui/lib/src/engine/dom.dart

+  external factory DOMPointInit({
+    double x = 0,
+    double y = 0,
+    double z = 0,
+    double w = 1,
+  });


I normally don't separate the external factory from the staticInterop class that it creates. I think you should be able to do this:

@JS() @anonymous @staticInterop class DOMPoint { external factory DOMPoint({ double x = 0, double y = 0, double z = 0, double w = 1, }); } extension DOMPointExtension on DOMPoint { external double get x; external double get y; external double get z; external double get w; }

(And then the types of stuff elsewhere can receive a DOMPoint and a DOMMatrix rather than the ...Init versions?)

I might remove this. I was confused by MDN docs saying that SVGMatrix is deprecated and we must use DOMMatrix, but when I tried using it the browser didn't accept it. So I changed it to SVGMatrix.

yjbanov · 2023-04-16T23:33:25Z

Couldn't schedule time to investigate this further. In particular, it's still unclear how to deal with the poor Safari performance until Safari itself optimizes SVG filters (there are bugs filed in WebKit dating back to 2012 on this, but no movement so far). Finally, the recent improvements in the size and performance of CanvasKit are getting closer to removing the need for an HTML renderer altogether. So spending time on this has dubious value. Closing for now.

POC: SVG canvas

ac679af

flutter-dashboard bot added the platform-web Code specifically for the web engine label Jan 2, 2023

yjbanov marked this pull request as draft January 2, 2023 23:27

fix clip transform; use <image>; fix drawImageRect

e2d3fc2

mdebbar reviewed Jan 3, 2023

View reviewed changes

ditman suggested changes Jan 3, 2023

View reviewed changes

ditman reviewed Jan 3, 2023

View reviewed changes

mdebbar mentioned this pull request Jan 11, 2023

Material 3 Demo App is Crashing on some iOS devices flutter/flutter#117164

Closed

yjbanov mentioned this pull request Jan 13, 2023

[web] Consider (again) using SVG as the main way to render instead of HTML+Canvas2D flutter/flutter#118468

Closed

mdebbar mentioned this pull request Jan 13, 2023

[web] Improve canvas2d allocation heuristics flutter/flutter#118471

Open

yjbanov closed this Apr 16, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PoC: SVG canvas #38602

PoC: SVG canvas #38602

yjbanov commented Jan 2, 2023 •

edited

Loading

mdebbar left a comment

mdebbar Jan 3, 2023

mdebbar Jan 3, 2023

Hixie Jan 19, 2023

mdebbar Jan 3, 2023

mdebbar Jan 3, 2023

mdebbar Jan 3, 2023

mdebbar Jan 3, 2023

ditman Jan 3, 2023 •

edited

Loading

ditman Jan 3, 2023 •

edited

Loading

yjbanov Jan 11, 2023

yjbanov commented Apr 16, 2023

		// DO_NOT_SUBMIT: revert this before submitting.
		// ..style.pointerEvents = 'none';

	spanElement.addY(boxRect.top + line.baseline);
	spanElement.addY(line.baseline);

PoC: SVG canvas #38602

PoC: SVG canvas #38602

Conversation

yjbanov commented Jan 2, 2023 • edited Loading

Notes on performance

Next steps

mdebbar left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ditman Jan 3, 2023 • edited Loading

Choose a reason for hiding this comment

ditman Jan 3, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yjbanov commented Apr 16, 2023

yjbanov commented Jan 2, 2023 •

edited

Loading

ditman Jan 3, 2023 •

edited

Loading

ditman Jan 3, 2023 •

edited

Loading