[live tests] Improve test matrix generation capabilities

[benbp]

Our latest model for generating a test matrix uses a combination of parameters and azure pipelines template expressions to define a matrix object that gets passed to the test job definition. This has been an improvement for our ability to define multiple cloud and platform configurations independent of the pipeline test logic (with plans to split the two).

There are still many constraints to this approach, however:

1. Expanding the matrix to support even more dimensions and filters, based on generic scenarios, increases in difficulty and/or becomes impossible to express within the azure pipelines templating language. For example, allowing a user to manually run a pipeline only on linux, or testing a subset of service versions in a standardized way.
2. Adding new functionality, and expanding it across the language repos, makes the pipeline templates more complex and harder to understand and consume for developers.
3. We've had to introduce non-ideal workarounds to support scenarios like test recording. This is due to the underlying logic in azure pipelines regarding where and when parameters and variables are available in the execution context for template compilation and job condition evaluation.

At the moment, it takes a non-trivial amount of effort to support new matrix configuration scenarios via azure pipelines yaml that would be trivial to implement in a script or program.

Recently azure pipelines added public documentation for what has been a powerful "undocumented" scenario: dynamic matrix generation. Taking advantage of this capability has many benefits:

1. We can dynamically generate a matrix definition at runtime, meaning we will have all pipeline parameters and variables in scope for the generation logic.
2. We can use powershell or another language to generate the matrix, giving us much more capability.
   1. We can develop and test matrix generation locally, independently of the azure pipelines server.
3. We can use our own configuration schema for defining test matrixes, beyond what has been possible using azure pipeline parameters.
   1. The test matrix config can be stored in a separate file independent of the pipeline definitions, de-coupling language and service test matrix definitions from our core test pipeline.
   2. The matrix can be defined in a way that prioritizes readability, as opposed to a format that is constrained by what azure pipelines can parse.