UPdate release branch? #3206

sgoggins · 2025-07-02T00:21:05Z

Description

We think the release branch would get updated before doing the release? Or, is mergify already activated?

Signed-off-by: Adrian Edwards <[email protected]>

- Replaces setup.py with pyproject.toml - Creates .python-version file that will control the python version that uv uses in the virtualenv Signed-off-by: John Strunk <[email protected]>

Signed-off-by: John Strunk <[email protected]>

Update GitHub workflow to include a new job that checks the uv lockfile, ensuring it is up to date by installing uv and running the lock check command. Signed-off-by: John Strunk <[email protected]>

Added post_create_environment job to generate requirements.txt using uv, and updated python install method to use the generated requirements file. Signed-off-by: John Strunk <[email protected]>

Signed-off-by: John Strunk <[email protected]>

Add keyman image and remove conditional login to GitHub Container Registry Signed-off-by: John Strunk <[email protected]>

This adds a new test job to run on macOS. It installs the package dependencies and runs the worker setup script. Due to the interactive nature of `make install`, we can't use that directly. Signed-off-by: John Strunk <[email protected]>

This updates ugboost to fix the build on macos Signed-off-by: John Strunk <[email protected]>

graphql-server-core 1.1.1 is incompatible w/ Python 3.10 due to changes in the standard library. This ensures that a newer version gets installed. Signed-off-by: John Strunk <[email protected]>

Update build rules

We fixed some documentation around how `uv run` gets executed in sequence with `nohup` Signed-off-by: Sean P. Goggins <[email protected]>

Changed `git checkout dev` to `git checkout main` in docs file. Signed-off-by: Sean P. Goggins <[email protected]>

Fixing `nohup` and `uv` order in docs. Signed-off-by: Sean P. Goggins <[email protected]>

Fixing `nohup` sequencing using `uv run` as tested. Signed-off-by: Sean P. Goggins <[email protected]>

Fixed references to the `housekeeper` Signed-off-by: Sean P. Goggins <[email protected]>

`nohup` ordering with `uv run` Signed-off-by: Sean P. Goggins <[email protected]>

fixing `uv run` and `nohup` order in docs. Signed-off-by: Sean P. Goggins <[email protected]>

Try 2 - Switch from venv/pip to uv

Push the keyman container image

.github/workflows/checks.yml

+    name: runner / uv-lock
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v4
+      - name: Install uv
+        uses: astral-sh/setup-uv@v6
+      - name: Ensure uv lockfile is up to date
+        run: uv lock --check


The fix involves adding a permissions block to the uv-lock job and assigning the least privileges required for its execution. Since the job steps seem to involve checking out code and running commands locally without modifying repository content, the contents: read permission is sufficient. This ensures the GITHUB_TOKEN has minimal access while keeping the workflow functional.

Signed-off-by: Adrian Edwards <[email protected]>

…retry Signed-off-by: Adrian Edwards <[email protected]>

Signed-off-by: Adrian Edwards <[email protected]>

determine whether to reset logs via new AUGUR_RESET_LOGS environment variable There are some linting issues revealed by this PR, but they are not a problem for this PR. They should be dealt with separately.

github-actions · 2025-07-11T00:31:29Z

augur/application/cli/backend.py

@@ -32,7 +32,9 @@

 from keyman.KeyClient import KeyClient, KeyPublisher


[pylint] _{reported by reviewdog 🐶}
W0611: Unused KeyClient imported from keyman.KeyClient (unused-import)

MoralCode

At this point I'm convinced it's just github weirdness.

We should merge a PR to bump the version number and edit the changelogs and other metadata fixes at the time we want to cut this release, but I'll stop being the squeaky wheel

Signed-off-by: Adrian Edwards <[email protected]>

Removed Files Not Needed

This changes the test for the DB connection components to not exit with failure if AUGUR_DB can't be constructed. Signed-off-by: John Strunk <[email protected]>

Address warnings from linter

Update entrypoint.sh construction of AUGUR_DB from env vars

fix missing docker -> podman in podman CI job

Handle 410 gone when querying github

github-actions · 2025-07-14T20:24:42Z

augur/tasks/github/util/github_data_access.py

@@ -124,6 +135,13 @@ def make_request(self, url, method="GET", timeout=100):
            if response.status_code == 401:
                raise NotAuthorizedException(f"Could not authorize with the github api")

+            if response.status_code == 410:
+                response_msg = response.json().get("message")
+                if response_msg is not None:


[pylint] _{reported by reviewdog 🐶}
R1720: Unnecessary "else" after "raise", remove the "else" and de-indent the code inside it (no-else-raise)

github-actions · 2025-07-14T20:24:42Z

augur/tasks/github/util/github_data_access.py

@@ -145,13 +163,21 @@
            return self.__make_request_with_retries(url, method, timeout)
        except RetryError as e:
            raise e.last_attempt.exception()
+
+    def _decide_retry_policy(exception: Exception) -> bool:


[pylint] _{reported by reviewdog 🐶}
E0213: Method '_decide_retry_policy' should have "self" as first argument (no-self-argument)

github-actions · 2025-07-14T20:24:42Z

augur/tasks/github/util/github_data_access.py

+        Returns:
+            bool: Boolean describing whether or not the request should be retried
+        """
+        return not isinstance(num, (UrlNotFoundException, ResourceGoneException))


[pylint] _{reported by reviewdog 🐶}
E0602: Undefined variable 'num' (undefined-variable)

Signed-off-by: Adrian Edwards <[email protected]>

use check_output for git log fetching so that it can fail more easily

correct docs links in error messages relating to keys

Signed-off-by: Adrian Edwards <[email protected]>

sgoggins

@MoralCode / @JohnStrunk / @Ulincsys : The main branch is leaking database connections. I have been through this PR to attempt to identify what change could have this side effect. Nothing is jumping out at me .. I made some comments, some of which are surely pointing a flashlight in the wrong corner ... If you have a chance to look this over and identify other possible places where this leak is introduced, THAT WOULD BE AWESOME!

NOTE: I have verified the database connection leak issue does NOT exist on the release branch.

sgoggins · 2025-07-23T18:41:15Z

augur/tasks/github/util/github_data_access.py


-    @retry(stop=stop_after_attempt(10), wait=wait_fixed(5), retry=retry_if_exception(lambda exc: not isinstance(exc, UrlNotFoundException)))
+    @retry(stop=stop_after_attempt(10), wait=wait_fixed(5), retry=retry_if_exception(_decide_retry_policy))


I think this may be causing an issue where not found URL's are causing breaking errors.

fixed in #3235

sgoggins · 2025-07-23T19:17:10Z

augur/api/gunicorn_conf.py

@@ -19,7 +20,22 @@
 workers = multiprocessing.cpu_count() * 2 + 1
 umask = 0o007
 reload = True
-reload_extra_files = glob(str(Path.cwd() / '**/*.j2'), recursive=True)
+


Maybe some issues here ... related to the disposal of database connections.

Can you elaborate on the issues you are seeing? This is a gunicorn config, so it shouldn't be affecting DB connections

@MoralCode : In short, I have my POstgresql instance set to 3,000 connections .. and within one hour of starting Augur, they are all consumed. Which is bad in myriad ways, right? For comparison, the version of Augur in release uses max 200 connections over weeks.

sgoggins · 2025-07-23T19:38:36Z

augur/tasks/github/util/github_data_access.py

@@ -22,6 +22,17 @@ class UrlNotFoundException(Exception):
 class NotAuthorizedException(Exception):
    pass

+class ResourceGoneException(Exception):


This is generating errors in testing. @MoralCode .. basically we are getting errors when these resource exceptions are hit and the job fails. Perhaps that is intended. Just checking.

I suspect this is fixed by #3235 (TL;DR my fault).

If a resource returns 410 Gone from github it is handled similarly to a 404 (i.e. bail out of the retry loop and keep going with collection rather than propagating the exception up to the worker and stopping everything)

I think you are right.

sgoggins · 2025-07-23T19:56:58Z

keyman/KeyClient.py

    def __init__(self) -> None:
        # Load channel names and IDs from the spec
        for channel in spec["channels"]:
            # IE: self.ANNOUNCE = "augur-oauth-announce"
            setattr(self, channel["name"], channel["id"])
-            
+        self.conn = get_redis_connection()


This seems possibly wrong. @MoralCode / @Ulincsys ... we have changed the import from as conn to simply be the library. I don't see where conn exists as an import any longer. If this fails, it seems it could be the source of the database connection leaks on main right now.

That sounds plausible to me. I remember seeing that this keyclient file got refactored, but maybe it needs a deeper look

Later on in my review it seemed like the object was getting used and reassigned in the new class made specifically for that ... I am suspicious of Redis in general, perhaps being used in a way that creates a database connection but doesn't let go of it ... or the process loses it ... Like I said, there is no change that appears to directly alter the way we are connecting to the database ... which is everyone's favorite type of bug, right?

sgoggins · 2025-07-23T19:57:45Z

keyman/KeyClient.py

@@ -172,7 +175,7 @@ def publish(self, key: str, platform: str):
            "key_platform": platform
        }

-        conn.publish(self.ANNOUNCE, json.dumps(message))
+        self.conn.publish(self.ANNOUNCE, json.dumps(message))


@MoralCode / @Ulincsys : Here as well, conn is referenced but I don't think it exists any longer.

sgoggins · 2025-07-23T19:58:57Z

keyman/KeyClient.py

@@ -1,4 +1,4 @@
-from augur.tasks.init.redis_connection import redis_connection as conn
+from augur.tasks.init.redis_connection import get_redis_connection


here is where the as conn is removed, but is still in use through the rest of the file.

sgoggins · 2025-07-23T19:59:37Z

keyman/KeyClient.py

@@ -215,7 +218,7 @@ def wait(self, timeout_seconds = 30, republish = False):
        }

        listen_delta = 0.1
-        conn.publish(self.ANNOUNCE, json.dumps(message))
+        self.conn.publish(self.ANNOUNCE, json.dumps(message))


sgoggins · 2025-07-23T20:00:18Z

keyman/Orchestrator.py

    from augur.application.logs import AugurLogger

    logger = AugurLogger("KeyOrchestrator").get_logger()
+    conn = get_redis_connection()


ok .. here conn exists as a call to a method.

sgoggins · 2025-07-23T20:01:45Z

keyman/KeyClient.py

@@ -29,8 +29,10 @@ def __init__(self, platform: str, logger: Logger):
        if not platform:
            raise ValueError("Platform must not be empty")

-        self.stdout = conn
-        self.stdin: PubSub = conn.pubsub(ignore_subscribe_messages = True)
+        self.conn = get_redis_connection()


The change here is moving the refernce from the library to this call to the get_redis_connection() method.

Fix variable name in _decide_retry_policy

Signed-off-by: Ulincsys <[email protected]>

cache redis connection on first retrieval; this fixes our open database connection infinite growth issue! I will talk with @ulincys before putting this to release to make sure I understand *why* this work. I have a conceptual grasp of how Redis is the issue, but less of a grasp of why this particular fix works. Mostly, before we release I want to understand if there are any possible side effects of this patch we need to look for.

MoralCode and others added 25 commits June 25, 2025 16:17

update container build automation for new branching model

422af93

Signed-off-by: Adrian Edwards <[email protected]>

remove dev, add release as branches to run jobs on push

ccfc861

Signed-off-by: Adrian Edwards <[email protected]>

remove automerge workflow

fe56e69

Signed-off-by: Adrian Edwards <[email protected]>

update branch name in checks.yml

bb57a7a

Signed-off-by: Adrian Edwards <[email protected]>

update comments in build_docker workflow

5e5f70f

Signed-off-by: Adrian Edwards <[email protected]>

Initial conversion of main augur package to uv

641db32

- Replaces setup.py with pyproject.toml - Creates .python-version file that will control the python version that uv uses in the virtualenv Signed-off-by: John Strunk <[email protected]>

Add dependencies for sub-packages into main pyproject.toml

ba8f342

Signed-off-by: John Strunk <[email protected]>

Update docker build for uv

36fc563

Signed-off-by: John Strunk <[email protected]>

Add uv-lock job

1fd74e5

Update GitHub workflow to include a new job that checks the uv lockfile, ensuring it is up to date by installing uv and running the lock check command. Signed-off-by: John Strunk <[email protected]>

Fix documention build on RTD

fbb61ab

Added post_create_environment job to generate requirements.txt using uv, and updated python install method to use the generated requirements file. Signed-off-by: John Strunk <[email protected]>

Update documentation for using uv

9c3087f

Signed-off-by: John Strunk <[email protected]>

Push the keyman container image

a45dff9

Add keyman image and remove conditional login to GitHub Container Registry Signed-off-by: John Strunk <[email protected]>

Add macOS test job

bc5bb74

This adds a new test job to run on macOS. It installs the package dependencies and runs the worker setup script. Due to the interactive nature of `make install`, we can't use that directly. Signed-off-by: John Strunk <[email protected]>

Upgrade xgboost to 3.0.2

68431f1

This updates ugboost to fix the build on macos Signed-off-by: John Strunk <[email protected]>

Constrain graphql-server-core>1.1.1

0f12451

graphql-server-core 1.1.1 is incompatible w/ Python 3.10 due to changes in the standard library. This ensures that a newer version gets installed. Signed-off-by: John Strunk <[email protected]>

Merge pull request #3200 from MoralCode/update-build-rules

6974aa5

Update build rules

Update nginx-configuration.rst

1f683ee

We fixed some documentation around how `uv run` gets executed in sequence with `nohup` Signed-off-by: Sean P. Goggins <[email protected]>

Update server-deployment.rst

bc51a81

Changed `git checkout dev` to `git checkout main` in docs file. Signed-off-by: Sean P. Goggins <[email protected]>

Update collecting-data.rst

9aa9b43

Fixing `nohup` and `uv` order in docs. Signed-off-by: Sean P. Goggins <[email protected]>

Update backend.rst

f8adc95

Fixing `nohup` sequencing using `uv run` as tested. Signed-off-by: Sean P. Goggins <[email protected]>

Update backend.rst

195a7c0

Fixed references to the `housekeeper` Signed-off-by: Sean P. Goggins <[email protected]>

Update dev-osx-install.rst

4a12816

`nohup` ordering with `uv run` Signed-off-by: Sean P. Goggins <[email protected]>

Update new-install.rst

6e4ab16

fixing `uv run` and `nohup` order in docs. Signed-off-by: Sean P. Goggins <[email protected]>

Merge pull request #3202 from JohnStrunk/uv-try2

3ab3029

Try 2 - Switch from venv/pip to uv

Merge pull request #3204 from JohnStrunk/push-keyman2

cf69b9e

Push the keyman container image

github-advanced-security bot found potential problems Jul 2, 2025

View reviewed changes

sgoggins requested a review from JohnStrunk July 2, 2025 00:21

MoralCode added 3 commits July 2, 2025 09:06

update docs refs in CONTRIBUTING

cb449d2

Signed-off-by: Adrian Edwards <[email protected]>

update badges

43f07d2

Signed-off-by: Adrian Edwards <[email protected]>

update some branch names

b3656b7

Signed-off-by: Adrian Edwards <[email protected]>

MoralCode and others added 3 commits July 10, 2025 20:28

include ResourceGoneException as one that should be ignored during a …

04ef1ef

…retry Signed-off-by: Adrian Edwards <[email protected]>

Add docs to _decide_retry_policy

99a8223

Signed-off-by: Adrian Edwards <[email protected]>

Merge pull request #3221 from MoralCode/reset-logs-var

cca041d

determine whether to reset logs via new AUGUR_RESET_LOGS environment variable There are some linting issues revealed by this PR, but they are not a problem for this PR. They should be dealt with separately.

github-actions bot reviewed Jul 11, 2025

View reviewed changes

MoralCode reviewed Jul 11, 2025

View reviewed changes

MoralCode and others added 8 commits July 10, 2025 21:12

Address warnings from linter

537df45

Signed-off-by: Adrian Edwards <[email protected]>

Merge pull request #3223 from chaoss/root-cleanup

bb8b08c

Removed Files Not Needed

Update entrypoint.sh construction of AUGUR_DB from env vars

d3d1ef4

This changes the test for the DB connection components to not exit with failure if AUGUR_DB can't be constructed. Signed-off-by: John Strunk <[email protected]>

fix missing docker -> podman in podman CI job

e4f10e1

Merge pull request #3224 from chaoss/lint-fix

d8f32ac

Address warnings from linter

Merge pull request #3226 from JohnStrunk/fix-db-var

06d0c7a

Update entrypoint.sh construction of AUGUR_DB from env vars

Merge pull request #3227 from MoralCode/podman-CI-typo

efae939

fix missing docker -> podman in podman CI job

Merge pull request #3208 from MoralCode/disabled-issues-410

dc17a8f

Handle 410 gone when querying github

github-actions bot reviewed Jul 14, 2025

View reviewed changes

MoralCode and others added 5 commits July 17, 2025 18:09

use check_output for git log

08aeca4

Signed-off-by: Adrian Edwards <[email protected]>

correct docs links in error messages relating to keys

df4aeb9

Signed-off-by: Adrian Edwards <[email protected]>

another main -> dev from RTD

78cacb1

Signed-off-by: Adrian Edwards <[email protected]>

Merge pull request #3232 from MoralCode/hang

1190187

use check_output for git log fetching so that it can fail more easily

Merge pull request #3233 from MoralCode/fix-docs-links

9447562

correct docs links in error messages relating to keys

MoralCode mentioned this pull request Jul 18, 2025

**[pylint]** <sub>reported by [reviewdog](https://github.com/reviewdog/reviewdog) 🐶</sub><br>E0602: Undefined variable 'num' (undefined-variable) #3234

Closed

Fix variable name in _decide_retry_policy

c45d53b

Signed-off-by: Adrian Edwards <[email protected]>

sgoggins commented Jul 23, 2025

View reviewed changes

Merge pull request #3235 from chaoss/github-api-fix

07b20f7

Fix variable name in _decide_retry_policy

sgoggins requested a review from MoralCode July 23, 2025 20:26

Ulincsys and others added 2 commits July 24, 2025 18:24

cache redis connection on first retrieval

69c1738

Signed-off-by: Ulincsys <[email protected]>

sgoggins requested review from Ulincsys, IsaacMilarky and ABrain7710 July 26, 2025 16:28

sgoggins merged commit 5e49264 into release Jul 26, 2025
23 checks passed

		@@ -32,7 +32,9 @@

		from keyman.KeyClient import KeyClient, KeyPublisher


		@retry(stop=stop_after_attempt(10), wait=wait_fixed(5), retry=retry_if_exception(lambda exc: not isinstance(exc, UrlNotFoundException)))
		@retry(stop=stop_after_attempt(10), wait=wait_fixed(5), retry=retry_if_exception(_decide_retry_policy))

		@@ -1,4 +1,4 @@
		from augur.tasks.init.redis_connection import redis_connection as conn
		from augur.tasks.init.redis_connection import get_redis_connection

UPdate release branch? #3206

UPdate release branch? #3206

Uh oh!

Conversation

sgoggins commented Jul 2, 2025

Uh oh!

Check warning

Copilot Autofix

github-actions bot Jul 11, 2025

Choose a reason for hiding this comment

Uh oh!

MoralCode left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot Jul 14, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Jul 14, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Jul 14, 2025

Choose a reason for hiding this comment

Uh oh!

sgoggins left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!