Skip to content

Test strategy for our providers on the CI #69

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
hug-dev opened this issue Nov 15, 2019 · 2 comments
Closed

Test strategy for our providers on the CI #69

hug-dev opened this issue Nov 15, 2019 · 2 comments
Labels
enhancement New feature or request

Comments

@hug-dev
Copy link
Member

hug-dev commented Nov 15, 2019

I think it could be tested by changing nothing but having a different configuration file which points to the SoftHSM library. There is an available Dockerfile that installs the library that we can use.

I think we also have to think how we are going to scale testing of multiple providers on the CI.

I was thinking of having single provider tests (our current tests) and multiple providers tests that test more the configuration as opposed to a provider implementation. For that purpose we should have multiple configuration files for all of those cases: one per provider and another one including them all.

@hug-dev hug-dev added the enhancement New feature or request label Nov 15, 2019
@hug-dev
Copy link
Member Author

hug-dev commented Nov 18, 2019

We could add an optional argument when launching Parsec to specify a configuration file path and use this one. That way we can have in-tree one configuration file per provider and test them independently 😄

And one more with all of them 😈

@ionut-arm ionut-arm reopened this Nov 19, 2019
@hug-dev hug-dev changed the title Test the PKCS 11 provider on the CI Test strategy for our providers on the CI Dec 13, 2019
@hug-dev
Copy link
Member Author

hug-dev commented Dec 13, 2019

We would have the following test structure (in tests):

all.sh -> Executing all of the tests below + rustfmt + clippy (will work only if everything is installed)
per_provider/ -> Test executed only with a single provider
    normal/
    stress/
    fuzz/
    persistence/
    PROVIDER_NAME/ -> Folder containing the Dockerfile and config.toml file per provider
        Dockerfile
        config.toml
all_providers/ -> Tests using all providers
    Dockerfile -> Would contain all providers (not published)
    config.toml
    mod.rs
cross_compilation/ -> Testing cross compilation of Parsec
    Dockerfile
    Cross.toml

The following Dockerfile would be needed:

  1. one for the TPM provider + tss-esapi crate (there is one in parallaxsecond/rust-tss-esapi repo already)
  2. one for the PKCS 11 provider
  3. one for the Mbed Crypto provider + crate (cf Create a PSA Crypto Rust wrapper crate #62)
  4. one combining 1, 2 and 3 for the all_providers tests
  5. one for cross-compilation, that already exists

Exactly the same tests (except cross_compilation) could be done on an Arm platform, on the CI, check #55 to also have coverage for Arm.

@hug-dev hug-dev added this to the Parsec production ready milestone Dec 13, 2019
hug-dev added a commit to hug-dev/parsec that referenced this issue Dec 20, 2019
In order to execute the tests under different single providers, under
all providers and for cross-compilation, split the tests directory into
sub-directories. Each one contains a specific configuration file to run
the tests under.
Please check issue parallaxsecond#69 for details.

Signed-off-by: Hugues de Valon <[email protected]>
hug-dev added a commit to hug-dev/parsec that referenced this issue Dec 20, 2019
In order to execute the tests under different single providers, under
all providers and for cross-compilation, split the tests directory into
sub-directories. Each one contains a specific configuration file to run
the tests under.
Please check issue parallaxsecond#69 for details.

Signed-off-by: Hugues de Valon <[email protected]>
hug-dev added a commit to hug-dev/parsec that referenced this issue Jan 3, 2020
In order to execute the tests under different single providers, under
all providers and for cross-compilation, split the tests directory into
sub-directories. Each one contains a specific configuration file to run
the tests under.
Please check issue parallaxsecond#69 for details.

Signed-off-by: Hugues de Valon <[email protected]>
@hug-dev hug-dev closed this as completed Jan 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants