Separate build status for each job in a build matrix

[chuckatkins]

Currently, the status reported to GitHub for a build is the combined pass/fail of all builds. It would be a great feature to have each configuration in the build matrix post a separate status so that instead of a single continuous-integration/appveyor/pr status, you would get separate statuses for ci/appveyor: vs2015 and ci/appveyor: vs2017, for example. This is a really useful feature available in CircleCI now where each independent "job" get's it's own status message.

[preed]

As discussed in this question/support ticket https://help.appveyor.com/discussions/questions/16443-status-badge-per-matrix-build, it sounds like this is the feature we're describing a desire for as well (with slightly different language).

[chuckatkins]

Yes, I believe they are one in the same. I used "job" in this context to reference circle CI, which is how they refer to the individual build matrix entries (at least in circle CI 2.0), each of which gets its own GitHub status entry.

[IlyaFinkelshteyn]

@chuckatkins can you point to some existing open source commit status to see how this look like?

[chuckatkins]

Certainly! An easy example to see this in is the ADIOS2 project (I'm one of the core developers). You can see from one of the currently open PRs: ornladios/ADIOS2#382 we're using travis to run code format tests with clang-format, circle-ci for linux tests where we use docker containers to run two jobs, one for gcc 4.8 and another for gcc7 + openmpi, and then appveyor for vs2015 and vs2017 tests. You can see here how the two circle-ci builds show up as separate status entries for each while the appveyor builds only show up as a single combined status.

Unrelated, but if you're wondering what's up with the "cdash" status, we have one of our circle builds use curl to manually post an "always success" status message with a link to the CDash dashboard entries for the CI results. That way we can easily get to the CDash result page for the PR from the same place we get to the other CI results.

[IlyaFinkelshteyn]

Another comment from customer email:
Not sure whether it's possible at all, but it would be neat if each job in the matrix (or, better yet, some to-be-defined groups of jobs) could be shown as separate checks in GitHub pull requests; e.g. in case of our project, it would be neat if the Windows builds would show up as one check and the Unix build would show up as another one. (This is a feature we currently have because we're using a different CI service for the Unix builds.)

[IlyaFinkelshteyn]

Related issue: #1960

[spyoungtech]

This has a lot of value for us.

We have a repo with multiple solution files e.g. src/aSolutionDir and src\anotherSolution. Following the advice here we do this via a build matrix.

However, we'd like a badge to report on the build status of each solution independently. To accomplish this, we have tried to configure multiple projects, which works*. The consequence, however, is that every commit to the repr triggers builds for all the projects, which is untenable.

Allowing status badges per job would alleviate this.

Alternatively (and somewhat unrelated to this issue) the ability to only run builds when changes to subdirectories are detected would also resolve our current frustrations. That way only relevant projects will build.

[IlyaFinkelshteyn]

@spyoungtech you resolve your current frustrations, you can use commit filtering with custom YAML file name or location for each project.

[spyoungtech]

@IlyaFinkelshteyn Thanks! This should work beautifully. Didn't notice the configuration was available in the UI because we had ignore appveyor.yml checked.