Add build_even/odd types to split Arduino tests #4737
Conversation
Arduino builds are now taking 49 minutes or more, and failing due to Travis timeouts. Split the Arduino build task into even and odd half, where each job will build every other test. This will reduce any test's runtime by 50% and speed up checks to avoid the timeout.
|
Just verified that the combination of both _odd and _even run the same tests as the single combined run (96 in total now). |
|
Added @igrr to get his input, too. Right now it's looking like jobs fail almost all the time due to timeouts, so if we want to get any merges in, this (or some other solution) has probably got to be merged first. |
|
You can use travis build stages, all stages are executed consecutively, and each stage can run up to one hour. |
|
@Juppit interesting idea, I didn't know about them. However, that's not quite the problem we need to solve here. Everything can be run in parallel with these tests, the problem is that the Arduino "build-all-examples" job itself takes ~ 50 mins to complete, leading to a CI timeout. With the proposed change, the individual jobs are cut in half to ~25 mins, and all 7 CI jobs are run in parallel (assuming enough CI slots). |
| @@ -239,7 +249,13 @@ fi | |||
|
|
|||
| if [ "$BUILD_TYPE" = "build" ]; then | |||
| install_arduino | |||
| build_sketches_with_arduino | |||
| build_sketches_with_arduino 1 0 | |||
| elif [ "$BUILD_TYPE" = "build_even" ]; then | |||
There was a problem hiding this comment.
Could you please rename build/build_odd/build_even to e.g. arduino/arduino_odd/arduino_even? It's not critical for the pr, but every time I read through this code it takes me a few minutes to remember that "build" really means arduino and is not common to everything.
| build_sketches_with_arduino 2 0 | ||
| elif [ "$BUILD_TYPE" = "build_odd" ]; then | ||
| install_arduino | ||
| build_sketches_with_arduino 2 1 | ||
| elif [ "$BUILD_TYPE" = "platformio" ]; then |
There was a problem hiding this comment.
Shouldn't a similar even/odd scheme be implemented for platformio builds?
There was a problem hiding this comment.
It's not needed. PlatformIO builds the examples 2x (yes, two times) as fast as Arduino. So it takes ~28mins today vs 50+mins for Ardunio. Should we go from 96 to 200 examples, I'll revisit this statement, but for now we're OK. 😄
There was a problem hiding this comment.
Looking at the.times, you're right. Even with load I see the platformio runtime doesn't go above 40mins.
|
Merging this now in the interest of moving forward. |
Arduino builds are now taking 49 minutes or more, and failing due to
Travis timeouts.
Split the Arduino build task into even and odd halves, where each job
will build every other test. This will reduce any test's runtime
by 50% and speed up checks to avoid the timeout.