Use single lakebase instance for all integration tests and add retry logic

[mwojtyczka]

Changes

• Create a single lakebase instance for all tests.
• Implement retry logic in case workspace quota limit for the number of lakebase instances is exceeded.
• Bumped pytester version.
• Bumped hatch version to avoid click compatibility issues.

Tests

• manually tested
• added unit tests
• added integration tests
• added end-to-end tests
• added performance tests

[Copilot]

Pull Request Overview

This PR adds retry logic to handle workspace quota limits when creating lakebase database instances during integration tests. The change introduces a timeout-based retry mechanism that specifically handles BadRequest exceptions, which commonly occur when the workspace quota for lakebase instances is exceeded during concurrent test runs.

• Added retry logic with 6-minute timeout for lakebase database instance creation
• Refactored database creation into a separate helper function to support retry mechanism
• Removed unused RETRY_INTERVAL_SECONDS constants from test files

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
tests/conftest.py	Added retry logic for lakebase database creation and refactored creation into helper function
tests/e2e/test_run_demos.py	Removed unused RETRY_INTERVAL_SECONDS constant
tests/e2e/conftest.py	Removed unused RETRY_INTERVAL_SECONDS constant

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

Pull Request Overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[github-actions]

✅ 403/403 passed, 2 flaky, 1 skipped, 4h24m17s total

Flaky tests:

• 🤪 test_quality_checker (3.783s)
• 🤪 test_e2e_workflow_with_custom_install_folder (7m17.119s)

_{Running from acceptance #2864}