Skip to content

When running with --test-randomize-ordering-seed, logging of seed is not in JSON when reporter is a JSON reporter. #1545

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
gspencergoog opened this issue Jun 24, 2021 · 2 comments · Fixed by #1548
Closed

When running with --test-randomize-ordering-seed, logging of seed is not in JSON when reporter is a JSON reporter. #1545

gspencergoog opened this issue Jun 24, 2021 · 2 comments · Fixed by #1548
Labels
type-bug Incorrect behavior (everything from a crash to more subtle misbehavior)

Comments

@gspencergoog
Copy link

gspencergoog commented Jun 24, 2021 •
edited
Loading

When running the test executable with -r json and --test-randomize-ordering-seed=random together, the test outputs the line "Shuffling test order with --test-randomize-ordering-seed=2995889359" at the start of the test, instead of printing valid JSON. This can cause issues if the output is being captured in order to parse it by machine.

Suggest outputting a JSON dict that includes the seed if using the json reporter, or converting the message to print to stderr instead of stdout, or just not printing the seed when using a JSON reporter, and adding it to the metadata for each suite.

/cc @jakemac53 @natebosch
@gspencergoog
Copy link
Author

Nevermind, this was in my code, not the test library.

@gspencergoog gspencergoog reopened this Jun 24, 2021
@gspencergoog
Copy link
Author

Actually, no, I also did this in my code, but it is in the test code too.

@natebosch natebosch added the type-bug Incorrect behavior (everything from a crash to more subtle misbehavior) label Jun 24, 2021
This was referenced Jun 29, 2021
Open
Merged
jakemac53 added a commit that referenced this issue Jun 29, 2021
@jakemac53
dont log the --test-randomization-ordering-seed if using the json rep…
b570ede 
…orter (#1548)

Fixes #1545

This is hacky but we don't have a good solution right now, added a TODO and filed an issue though.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type-bug Incorrect behavior (everything from a crash to more subtle misbehavior)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

dont log the --test-randomization-ordering-seed if using the json reporter dart-lang/test
2 participants
@natebosch @gspencergoog