feature: performance improvements

[DmitryEfimenko]

I stumbled accross an article which describes how performance of angular unit tests can be improved (sometimes by 300%). The author created a package based on the technicks described in the article. Here's its source code.

I'm wondering if these technicks could be baked into spectator. What do you think?

[NetanelBasal]

Do you want to create a POC?

[DmitryEfimenko]

I was hoping someone else would look into it :D
I am quite swamped at work right now (aren't we all). I'll take a stab at it when I get a bit of a break (if no one else tries it before me)

[dirkluijk]

I'm interested. I'm gonna take a look and let you know.

[dirkluijk]

I did a POC and tested it on our project. It looks very promising. No tests break and it is really a lot faster (we use Jest, but should not matter).

The differences:

app 1 (before)

=============================== Coverage summary ===============================
Statements   : 80.7% ( 1535/1902 )
Branches     : 61.7% ( 501/812 )
Functions    : 69.13% ( 365/528 )
Lines        : 79.74% ( 1248/1565 )
================================================================================

Test Suites: 1 skipped, 48 passed, 48 of 49 total
Tests:       9 skipped, 197 passed, 206 total
Snapshots:   0 total
Time:        72.012s
Ran all test suites.

library 1 (before)

=============================== Coverage summary ===============================
Statements   : 75.32% ( 1968/2613 )
Branches     : 48.09% ( 340/707 )
Functions    : 55.17% ( 315/571 )
Lines        : 73.77% ( 1631/2211 )
================================================================================

Test Suites: 36 passed, 36 total
Tests:       1 skipped, 177 passed, 178 total
Snapshots:   0 total
Time:        20.504s, estimated 26s

library 2 (before)

=============================== Coverage summary ===============================
Statements   : 72.21% ( 2816/3900 )
Branches     : 44.7% ( 595/1331 )
Functions    : 54.8% ( 519/947 )
Lines        : 70.82% ( 2444/3451 )
================================================================================

Test Suites: 44 passed, 44 total
Tests:       224 passed, 224 total
Snapshots:   0 total
Time:        19.937s, estimated 48s

app 1 (after)

=============================== Coverage summary ===============================
Statements   : 80.65% ( 1517/1881 )
Branches     : 61.66% ( 497/806 )
Functions    : 69.1% ( 360/521 )
Lines        : 79.74% ( 1236/1550 )
================================================================================

Test Suites: 1 skipped, 48 passed, 48 of 49 total
Tests:       9 skipped, 197 passed, 206 total
Snapshots:   0 total
Time:        34.835s, estimated 55s

library 1 (after)

=============================== Coverage summary ===============================
Statements   : 75.28% ( 1967/2613 )
Branches     : 48.09% ( 340/707 )
Functions    : 54.99% ( 314/571 )
Lines        : 73.72% ( 1630/2211 )
================================================================================

Test Suites: 36 passed, 36 total
Tests:       1 skipped, 177 passed, 178 total
Snapshots:   0 total
Time:        13.688s, estimated 18s

library 2 (after)

=============================== Coverage summary ===============================
Statements   : 72.21% ( 2816/3900 )
Branches     : 44.7% ( 595/1331 )
Functions    : 54.8% ( 519/947 )
Lines        : 70.82% ( 2444/3451 )
================================================================================

Test Suites: 44 passed, 44 total
Tests:       224 passed, 224 total
Snapshots:   0 total
Time:        14.229s, estimated 17s

Summary

From 72.01s to 36.76s is 1.96 times faster
From 20.50s to 13.69s is 1.50 times faster
From 19.94s to 14.22s is 1.40 times faster

Some notes

I could only improve Spectator, not SpectatorWithHost as it does not use compileComponents(). I tried to use ng-bullet for that as well, but that does not make it faster but even slightly slower.

That also matches the test results, because we use SpectatorWithHost more in the libraries and less in the apps.

Last note, the coverage drops a very little bit for our app. Not sure why. Not I problem imho.

@NetanelBasal if you would release a snapshot or beta for #100 I could also test it on our CI build.

ping @topnotch48

[DmitryEfimenko]

@dirkluijk Wow, thanks for testing around! Really awesome to see this! Looking forward for where it gets us!

[dirkluijk]

Weird. I suddenly cannot measure any difference anymore. As if my laptop was just slow the first test run. How weird.

Can someone else test as well?

[IlCallo]

If you are using Jest via Nx and jest-angular-preset, there're some known performance problems with the first run after last update to jest 24.
Check their repo issue tracker to find out more

[dirkluijk]

Closed as the POC in #100 did not deliver the expected results.

[kleky]

I did a POC and tested it on our project. It looks very promising. No tests break and it is really a lot faster (we use Jest, but should not matter).

The differences:

app 1 (before)

=============================== Coverage summary ===============================
Statements   : 80.7% ( 1535/1902 )
Branches     : 61.7% ( 501/812 )
Functions    : 69.13% ( 365/528 )
Lines        : 79.74% ( 1248/1565 )
================================================================================

Test Suites: 1 skipped, 48 passed, 48 of 49 total
Tests:       9 skipped, 197 passed, 206 total
Snapshots:   0 total
Time:        72.012s
Ran all test suites.

library 1 (before)

=============================== Coverage summary ===============================
Statements   : 75.32% ( 1968/2613 )
Branches     : 48.09% ( 340/707 )
Functions    : 55.17% ( 315/571 )
Lines        : 73.77% ( 1631/2211 )
================================================================================

Test Suites: 36 passed, 36 total
Tests:       1 skipped, 177 passed, 178 total
Snapshots:   0 total
Time:        20.504s, estimated 26s

library 2 (before)

=============================== Coverage summary ===============================
Statements   : 72.21% ( 2816/3900 )
Branches     : 44.7% ( 595/1331 )
Functions    : 54.8% ( 519/947 )
Lines        : 70.82% ( 2444/3451 )
================================================================================

Test Suites: 44 passed, 44 total
Tests:       224 passed, 224 total
Snapshots:   0 total
Time:        19.937s, estimated 48s

app 1 (after)

=============================== Coverage summary ===============================
Statements   : 80.65% ( 1517/1881 )
Branches     : 61.66% ( 497/806 )
Functions    : 69.1% ( 360/521 )
Lines        : 79.74% ( 1236/1550 )
================================================================================

Test Suites: 1 skipped, 48 passed, 48 of 49 total
Tests:       9 skipped, 197 passed, 206 total
Snapshots:   0 total
Time:        34.835s, estimated 55s

library 1 (after)

=============================== Coverage summary ===============================
Statements   : 75.28% ( 1967/2613 )
Branches     : 48.09% ( 340/707 )
Functions    : 54.99% ( 314/571 )
Lines        : 73.72% ( 1630/2211 )
================================================================================

Test Suites: 36 passed, 36 total
Tests:       1 skipped, 177 passed, 178 total
Snapshots:   0 total
Time:        13.688s, estimated 18s

library 2 (after)

=============================== Coverage summary ===============================
Statements   : 72.21% ( 2816/3900 )
Branches     : 44.7% ( 595/1331 )
Functions    : 54.8% ( 519/947 )
Lines        : 70.82% ( 2444/3451 )
================================================================================

Test Suites: 44 passed, 44 total
Tests:       224 passed, 224 total
Snapshots:   0 total
Time:        14.229s, estimated 17s

Summary

From 72.01s to 36.76s is 1.96 times faster
From 20.50s to 13.69s is 1.50 times faster
From 19.94s to 14.22s is 1.40 times faster

Some notes

I could only improve Spectator, not SpectatorWithHost as it does not use compileComponents(). I tried to use ng-bullet for that as well, but that does not make it faster but even slightly slower.

That also matches the test results, because we use SpectatorWithHost more in the libraries and less in the apps.

Last note, the coverage drops a very little bit for our app. Not sure why. Not I problem imho.

@NetanelBasal if you would release a snapshot or beta for #100 I could also test it on our CI build.

ping @topnotch48

It's a shame this was ditched. We've started to use ng-bullet in our regular angular tests and seen 300% speed improvement. Are you able to share your work here? I think I may be able to help on this. Never helped with open source before, but there's a first for everything.

[NetanelBasal]

@kleky you can do a POC on your project.