The dart2js transformer runs out of memory compiling multiple entrypoints

[DartBot]

<img src="https://avatars.githubusercontent.com/u/46275?v=3" align="left" width="96" height="96"hspace="10"> Issue by munificent
Originally opened as dart-lang/sdk#14730

---

Siggy (and others) can consistently repro an issue where pub build will crash with out of memory. To do so, just duplicate the index.html file in web/ and then add it as an entrypoint:

transformers:

• polymer:
      entry_points: [web/index.html, web/index2.html]

The issue seems to be that dart2js does some of its work asynchronously, which means we get multiple compiles going at least a little bit in parallel, and there isn't enough room for it.

We should investigate this more and have a clearer picture of what's going on. We can work around it by making the transformer force itself to be sequential, but a cleaner fix would be better.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/46275?v=3" align="left" width="48" height="48"hspace="10"> Comment by munificent

---

Added this to the Later milestone.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/2049220?v=3" align="left" width="48" height="48"hspace="10"> Comment by sigmundch

---

Issue #722 has been merged into this issue.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/405837?v=3" align="left" width="48" height="48"hspace="10"> Comment by zoechi

---

Attachment:
pub_build_out_of_memory.txt (305.51 KB)

[DartBot]

<img src="https://avatars.githubusercontent.com/u/405837?v=3" align="left" width="48" height="48"hspace="10"> Comment by zoechi

---

Attachment:
build.zip (103.54 KB)

[DartBot]

<img src="https://avatars.githubusercontent.com/u/46275?v=3" align="left" width="48" height="48"hspace="10"> Comment by munificent

---

On my mac laptop, I was able to get it to build completely, but it did peak at around 600MB usage.

If I disable dart2js, I see it still eating about 260MB even before it prints "Building polymer_elements_examples....", so it looks like we're using a decent amount of memory just in barback. This is for a build that has 474 source assets, which is not a huge number.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/405837?v=3" align="left" width="48" height="48"hspace="10"> Comment by zoechi

---

For me it still fails with 'out of memory' on a 12GB Debian linux system with or without --mode=debug

[DartBot]

<img src="https://avatars.githubusercontent.com/u/46275?v=3" align="left" width="48" height="48"hspace="10"> Comment by munificent

---

Removed Priority-Unassigned label.
Added Priority-High label.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/46275?v=3" align="left" width="48" height="48"hspace="10"> Comment by munificent

---

Marked this as blocking dart-lang/sdk#15829.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/46275?v=3" align="left" width="48" height="48"hspace="10"> Comment by munificent

---

Marked this as being blocked by dart-lang/sdk#8355.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/46275?v=3" align="left" width="48" height="48"hspace="10"> Comment by munificent

---

Added DevStory label.

[DartBot]

<img src="https://avatars.githubusercontent.com/u/46275?v=3" align="left" width="48" height="48"hspace="10"> Comment by munificent

---

With latest code, it seems like we're doing OK. I just ran the following test:

• A package containing 50 copies of the TodoMVC web directory.
• 50 polymer entrypoints
• 1,101 source files (1.93 MB)
• 417 output files (72.0 MB)

Although it took a while, it completed successfully on both my Mac and Windows machine. Memory usage quickly went to about 450 MB and then hovered around there, gradually creeping up to around 500 MB before completing.

---

Added AssumedStale label.