Empty PR to test the benchmarks again

[smarter]

I'm suspicious.

[smarter]

test performance please

[dottybot]

performance test scheduled: 1 job(s) in queue, 0 running.

[liufengyun]

This morning I changed the forks and warmups for some noisy lines (mainly Vector and exhaustivity check tests), that might explain some of the changes. Le 17 sept. 2017 5:47 PM, "Guillaume Martres" <notifications@github.com> a écrit :

…

test performance please — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#3137 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAuDycaC-lnsoNDSwgzJWddOcRcUufxpks5sjT8EgaJpZM4PaL5T> .

[dottybot]

performance test scheduled: 1 job(s) in queue, 1 running.

[smarter]

I think the bot decided to run the benchmarks again because you quoted the command :).

[liufengyun]

Thanks for the catch @smarter . The duplicate job removed and system updated. Now commands have to start from a line, so quoted commands are excluded.

[dottybot]

Performance test finished successfully:

Visit http://dotty-bench.epfl.ch/3137 to see the changes.

Benchmarks is based on merge(s) with master

[smarter]

Looks like the benchmark changes made a big difference, invalidating the results of #3133 and #3136. @liufengyun I suggest reverting the changes for now. In the future I guess we'll need to rerun the benchmarks for all points when we make changes like this.

[liufengyun]

@smarter I've temporarily reverted the changes in liufengyun/bench@19cd60c on the server, which affects 5 lines. Now you can try again.

Note that re-run all benchmarks is both computationally expensive and laborious, thus re-run is only justified when there's a compelling reason.

1. Each point takes about 1 hour
2. The dependencies/structure of the compiler is changing, which requires updating the infrastructure to fit the compiler every time it changes.

[smarter]

test performance please

[dottybot]

performance test scheduled: 1 job(s) in queue, 0 running.

[dottybot]

Performance test finished successfully:

Visit http://dotty-bench.epfl.ch/3137/ to see the changes.

Benchmarks is based on merge(s) with master

[smarter]

This looks more reasonable to me.