New integration tests for @sentry/browser and core.

[onurtemizkan]

Currently, the integration tests for the browser package are flaky and so are disabled on CI.

Having a simpler set of integration tests that can be run on main browser cores (Chromium/WebKit/Firefox) could be very useful for catching issues related to stack trace formats or breadcrumbs.

The options I was thinking that could be fit for that are:

• jest + puppeteer (chrome + firefox)
• jest + playwright (chrome + webkit + firefox)
• webdriver-io (simply selenium)

What would you think can be the best path to take?

[kamilogorek]

I'm all up for it, but only as long as they will be reliable and won't take hours to run. We should discuss it during next planing @getsentry/team-webplatform

[AbhiPrasad]

We've discussed this with the team internally and we figured it might be a good idea to explore playwright e2e tests for @sentry/browser. As Kamil mentioned above, the priority is that they are reliable and run relatively quickly.

@onurtemizkan do you mind exploring to see what this would look like? Once we get a grasp on how it feels, we can make a decision if we want to pursue this more indepth.

[onurtemizkan]

@AbhiPrasad, sure thing! I'll open a draft PR with a little PoC then. 👍

[github-actions]

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you label it Status: Backlog or Status: In Progress, I will leave it alone ... forever!

---

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

[rhcarvalho]

For the record, documenting our current state:

Currently, the integration tests for the browser package are flaky and so are disabled on CI.

I think the most flaky part was running the tests in BrowserStack.

Running the same tests on a local headless browser seems fine and the original tests have been re-enabled in #4226.

tests that can be run on main browser cores (Chromium/WebKit/Firefox)

We've also started running the original tests on those three browsers (the same browsers that come bundled with Playwright) in #4232.

[AbhiPrasad]

One thing I'd like to note is that the existing karma integration tests are pretty hard to debug. I went through this while working on: #4265

Karma has issues with console.log debugging: karma-runner/karma-mocha#47, and setting up breakpoints is also a pain. In addition, it's quite hard to tell what scenarios are associated with what html + assertions at an initial glance.

[AbhiPrasad]

Closing this as the playwright tests are in a pretty good spot now!