Testing: Demonstrate Python test layer implementations

The patch demonstrates the whole matrix of building test harnesses with
Python, using both unittest and pytest, with and without "Testcontainers
for Python".

It only exercises basic features of the corresponding test adapters.
Demonstrating advanced features like test layer parameterization can be
added on behalf of subsequent iterations.

Author: amotl

.github/dependabot.yml

@@ -103,3 +103,13 @@ updates:
     package-ecosystem: "gradle"
     schedule:
       interval: "weekly"
+
+  - directory: "/testing/testcontainers/python-pytest"
+    package-ecosystem: "pip"
+    schedule:
+      interval: "weekly"
+
+  - directory: "/testing/testcontainers/python-unittest"
+    package-ecosystem: "pip"
+    schedule:
+      interval: "weekly"

.github/workflows/testing-native-python.yml

@@ -0,0 +1,66 @@
+name: Native Testing with Python
+
+on:
+  pull_request:
+    branches: ~
+    paths:
+    - '.github/workflows/testing-native-python.yml'
+    - 'testing/native/python**'
+    - 'requirements.txt'
+  push:
+    branches: [ main ]
+    paths:
+    - '.github/workflows/testing-native-python.yml'
+    - 'testing/native/python**'
+    - 'requirements.txt'
+
+  # Allow job to be triggered manually.
+  workflow_dispatch:
+
+  # Run job each night after CrateDB nightly has been published.
+  schedule:
+    - cron: '0 3 * * *'
+
+# Cancel in-progress jobs when pushing to the same branch.
+concurrency:
+  cancel-in-progress: true
+  group: ${{ github.workflow }}-${{ github.ref }}
+
+jobs:
+  test:
+    name: "
+     Python: ${{ matrix.python-version }}
+     CrateDB: ${{ matrix.cratedb-version }}
+     on ${{ matrix.os }}"
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ 'ubuntu-latest', 'macos-latest' ]
+        python-version: [ '3.7', '3.11' ]
+        cratedb-version: [ 'nightly' ]
+
+    steps:
+
+      - name: Acquire sources
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+          architecture: x64
+          cache: 'pip'
+          cache-dependency-path: |
+            requirements.txt
+            testing/native/python-pytest/requirements.txt
+            testing/native/python-unittest/requirements.txt
+
+      - name: Install utilities
+        run: |
+          pip install -r requirements.txt
+
+      - name: Validate testing/native/python-{pytest,unittest}
+        run: |
+          ngr test --accept-no-venv testing/native/python-pytest
+          ngr test --accept-no-venv testing/native/python-unittest

.github/workflows/testing-testcontainers-python.yml

@@ -0,0 +1,66 @@
+name: Testcontainers for Python
+
+on:
+  pull_request:
+    branches: ~
+    paths:
+    - '.github/workflows/testing-testcontainers-python.yml'
+    - 'testing/testcontainers/python**'
+    - 'requirements.txt'
+  push:
+    branches: [ main ]
+    paths:
+    - '.github/workflows/testing-testcontainers-python.yml'
+    - 'testing/testcontainers/python**'
+    - 'requirements.txt'
+
+  # Allow job to be triggered manually.
+  workflow_dispatch:
+
+  # Run job each night after CrateDB nightly has been published.
+  schedule:
+    - cron: '0 3 * * *'
+
+# Cancel in-progress jobs when pushing to the same branch.
+concurrency:
+  cancel-in-progress: true
+  group: ${{ github.workflow }}-${{ github.ref }}
+
+jobs:
+  test:
+    name: "
+     Python: ${{ matrix.python-version }}
+     CrateDB: ${{ matrix.cratedb-version }}
+     on ${{ matrix.os }}"
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ 'ubuntu-latest' ]
+        python-version: [ '3.7', '3.12' ]
+        cratedb-version: [ 'nightly' ]
+
+    steps:
+
+      - name: Acquire sources
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+          architecture: x64
+          cache: 'pip'
+          cache-dependency-path: |
+            requirements.txt
+            testing/testcontainers/python-pytest/requirements.txt
+            testing/testcontainers/python-unittest/requirements.txt
+
+      - name: Install utilities
+        run: |
+          pip install -r requirements.txt
+
+      - name: Validate testing/testcontainers/python-{pytest,unittest}
+        run: |
+          ngr test --accept-no-venv testing/testcontainers/python-pytest
+          ngr test --accept-no-venv testing/testcontainers/python-unittest

testing/native/python-pytest/README.md

@@ -0,0 +1,43 @@
+# Using "pytest-crate" with CrateDB and pytest
+
+[pytest-crate] wraps the CrateDB test layer from [cr8], and provides
+a few pytest fixtures to conveniently make it accessible for test
+cases based on pytest.
+
+This folder contains example test cases demonstrating how to use the
+`crate`, `crate_cursor`, and `crate_cursor` pytest fixtures exported
+by [pytest-crate].
+
+> [!TIP]
+> Please also refer to the header sections of each of the provided
+> example programs, to learn more about what's exactly inside.
+
+
+## Run Tests
+
+Acquire the `cratedb-examples` repository, and install sandbox and
+prerequisites.
+```shell
+git clone https://github.com/crate/cratedb-examples
+cd cratedb-examples
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -r requirements.txt
+```
+
+Then, invoke the integration test cases.
+```shell
+export TC_KEEPALIVE=true
+ngr test testing/native/python-pytest
+```
+
+Alternatively, you can change your working directory to the selected
+test case folder, and run `pytest` inside there.
+```shell
+cd testing/native/python-pytest
+pytest
+```
+
+
+[cr8]: https://pypi.org/project/cr8/
+[pytest-crate]: https://pypi.org/project/pytest-crate/

testing/native/python-pytest/pyproject.toml

@@ -0,0 +1,10 @@
+[tool.pytest.ini_options]
+minversion = "2.0"
+addopts = """
+  -rsfEX -p pytester --strict-markers --verbosity=3
+  --capture=no
+  """
+log_level = "DEBUG"
+log_cli_level = "DEBUG"
+testpaths = ["*.py"]
+xfail_strict = true

testing/native/python-pytest/requirements.txt

@@ -0,0 +1,3 @@
+crash==0.31.2
+pytest<9
+pytest-crate==0.3.0

testing/native/python-pytest/test_pytest.py

@@ -0,0 +1,51 @@
+"""
+Using "pytest-crate" with CrateDB and pytest
+
+Build test harnesses around CrateDB using the `crate` pytest fixture
+exported by `pytest-crate`. In turn, this is using `CrateNode`
+exported by `cr8`.
+
+https://pypi.org/project/pytest-crate/
+https://pypi.org/project/cr8/
+"""
+import subprocess
+
+SQL_STATEMENT = "SELECT * FROM sys.summits ORDER BY height DESC LIMIT 3;"
+
+
+def test_crash(crate):
+    """
+    After provisioning a test instance of CrateDB, invoke `crash`.
+    """
+    http_url = crate.dsn()
+    command = f"time crash --hosts '{http_url}' --command 'SELECT * FROM sys.summits ORDER BY height DESC LIMIT 3;'"
+    subprocess.check_call(command, shell=True)
+
+
+def test_crate(crate):
+    assert crate.dsn().startswith("http://127.0.0.1:42")
+    assert "http" in crate.addresses
+    assert crate.addresses["http"].host == "127.0.0.1"
+    assert 4300 > crate.addresses["http"].port >= 4200
+    assert "psql" in crate.addresses
+    assert crate.addresses["psql"].host == "127.0.0.1"
+    assert 5500 > crate.addresses["psql"].port >= 5432
+    assert "transport" in crate.addresses
+    assert crate.addresses["transport"].host == "127.0.0.1"
+    assert 4400 > crate.addresses["transport"].port >= 4300
+
+
+def test_cursor(crate_cursor):
+    crate_cursor.execute("SELECT 1")
+    assert crate_cursor.fetchone() == [1]
+
+
+def test_execute(crate_execute, crate_cursor):
+    for stmt in [
+        "CREATE TABLE pytest (name STRING, version INT)",
+        "INSERT INTO pytest (name, version) VALUES ('test_execute', 1)",
+        "REFRESH TABLE pytest",
+    ]:
+        crate_execute(stmt)
+    crate_cursor.execute("SELECT name, version FROM pytest")
+    assert crate_cursor.fetchall() == [["test_execute", 1]]

testing/native/python-unittest/.ngr-type

@@ -0,0 +1 @@
+python-unittest

testing/native/python-unittest/README.md

@@ -0,0 +1,41 @@
+# Using "cr8" test layers with CrateDB and unittest
+
+[cr8] provides a subsystem to invoke throwaway instances of CrateDB
+for testing purposes.
+
+This folder contains example test cases demonstrating how to use the
+`create_node` utility function and the `CrateNode` class, exported
+by [cr8], within your Python unittest-based test cases.
+
+> [!TIP]
+> Please also refer to the header sections of each of the provided
+> example programs, to learn more about what's exactly inside.
+
+
+## Run Tests
+
+Acquire the `cratedb-examples` repository, and install sandbox and
+prerequisites.
+```shell
+git clone https://github.com/crate/cratedb-examples
+cd cratedb-examples
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -r requirements.txt
+```
+
+Then, invoke the integration test cases.
+```shell
+export TC_KEEPALIVE=true
+ngr test testing/native/python-unittest
+```
+
+Alternatively, you can change your working directory to the selected
+test case folder, and run `unittest` inside there.
+```shell
+cd testing/native/python-unittest
+python -m unittest -vvv
+```
+
+
+[cr8]: https://pypi.org/project/cr8/

testing/native/python-unittest/requirements.txt

@@ -0,0 +1,2 @@
+cr8==0.25.0
+crash==0.31.2

testing/native/python-unittest/test_unittest.py

@@ -0,0 +1,37 @@
+"""
+Using "cr8" test layers with CrateDB and unittest
+
+Build test harnesses around CrateDB using the `create_node`
+primitive exported by `cr8`.
+
+https://pypi.org/project/cr8/
+"""
+import subprocess
+from unittest import TestCase
+
+from cr8.run_crate import create_node
+
+# Run a testing instance of CrateDB.
+# TODO: How to select a nightly release?
+cratedb_layer = create_node(version="latest-testing")
+
+SQL_STATEMENT = "SELECT * FROM sys.summits ORDER BY height DESC LIMIT 3;"
+
+
+def setUpModule():
+    cratedb_layer.start()
+
+
+def tearDownModule():
+    cratedb_layer.stop()
+
+
+class CrashTest(TestCase):
+
+    def test_crash(self):
+        """
+        After provisioning a test instance of CrateDB, invoke `crash`.
+        """
+        http_url = cratedb_layer.http_url
+        command = f"time crash --hosts '{http_url}' --command '{SQL_STATEMENT}'"
+        subprocess.check_call(command, shell=True)

testing/testcontainers/python-pytest/README.md

@@ -0,0 +1,42 @@
+# Using "Testcontainers for Python" with CrateDB and pytest
+
+[Testcontainers] is an open source framework for providing throwaway,
+lightweight instances of databases, message brokers, web browsers, or
+just about anything that can run in a Docker container.
+
+This folder contains example test cases demonstrating how to use the
+`cratedb_service` pytest fixture exported by [cratedb-toolkit].
+
+> [!TIP]
+> Please also refer to the header sections of each of the provided
+> example programs, to learn more about what's exactly inside.
+
+
+## Run Tests
+
+Acquire the `cratedb-examples` repository, and install sandbox and
+prerequisites.
+```shell
+git clone https://github.com/crate/cratedb-examples
+cd cratedb-examples
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -r requirements.txt
+```
+
+Then, invoke the integration test cases.
+```shell
+export TC_KEEPALIVE=true
+ngr test testing/testcontainers/python-pytest
+```
+
+Alternatively, you can change your working directory to the selected
+test case folder, and run `pytest` inside there.
+```shell
+cd testing/testcontainers/python-pytest
+pytest
+```
+
+
+[cratedb-toolkit]: https://pypi.org/project/cratedb-toolkit/
+[Testcontainers]: https://testcontainers.org/

testing/testcontainers/python-pytest/pyproject.toml

@@ -0,0 +1,10 @@
+[tool.pytest.ini_options]
+minversion = "2.0"
+addopts = """
+  -rsfEX -p pytester --strict-markers --verbosity=3
+  --capture=no
+  """
+log_level = "DEBUG"
+log_cli_level = "DEBUG"
+testpaths = ["*.py"]
+xfail_strict = true