Skip to content

Expose current test information from Invoker #1068

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
smaifullerton-wk opened this issue Sep 3, 2019 · 8 comments
Open

Expose current test information from Invoker #1068

smaifullerton-wk opened this issue Sep 3, 2019 · 8 comments

Comments

@smaifullerton-wk
Copy link

We use source imports to access the Invoker directly to collection information about tests we run with a web driver in CI, such as:

Invoker.current.liveTest.test.name;
Invoker.current.liveTest.state.result.isPassing;
Invoker.current.liveTest.suite.group.name;

It'd be great if this information were publicly exposed so the API changing won't break us.

Related to #107

@natebosch
Copy link
Member

@grouma - we had discussed something similar recently.

@smaifullerton-wk - can you expand on your use case? I think we had also considered an API like onFailure(void Function() callback) to run some code after a test fails - I'm curious if that covers your use case for isPassing. I'm also curious what your use case is for needing the group name.

@smaifullerton-wk
Copy link
Author

@natebosch Thanks for the fast response. We use the test status in our custom web driver wrapper to report to Sauce Labs whether or not a test passed (so we depend on knowing passing vs. failing -- the onFailure callback seems like it may not cover that directly. That said, if we can make it work, it's better than a src import 😄).

The group name is a bit hacky and lowest priority, but looks like it's used to report to Sauce Labs that all of the tests in a particular group have failed if one of them does. This may be unnecessary as the few usages we have were written 3 years ago.

With all usages, the Invoker fields are retrieved in a driver class for functional tests completely separate from the test file.

@natebosch
Copy link
Member

It sounds like the best thing for this use case might be #12

Having access to this info in a reporter is a little less risky than having it in the tests themselves, or in setup or teardown...

@RKushnir
Copy link

RKushnir commented Oct 28, 2019

I'm doing something hacky where I would like to use this. In a Flutter integration test I want to send a message to the app under test with the name of the current test, so it can initialize the http mocks specific to the current test.

This is the thing from Ruby I'm trying to replicate in Dart https://github.com/vcr/vcr

@travissanderson-wf
Copy link

We're still stuck using a hacky workaround to access this via a src import - any update on making this available via a public API?

@natebosch
Copy link
Member

No updates, we are hesitant to commit to a public API given that we don't believe it would be the best solution - it's what has come up as the hack because it's the cheapest option available. There isn't likely to be movement in either direction on this (either making it public or breaking existing imports) for a while.

@evanweible-wf
Copy link
Contributor

What about the related issue #12? That would solve the majority of our uses

@natebosch
Copy link
Member

#12 is something we'd like to push on, but it's not on our immediate roadmap. It's really hard to have an idea of how much work that would take. We don't have solid patterns for "injecting" user code into the test runner - the fact that we build with user code is a big reason for the complexity and some of the slowness in build_runner.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants