chore(ci): configure CircleCI for postgres builds #1840

chenba · 2025-10-15T15:58:06Z

Description

This commit enables Postgres builds and tests in CircleCI. Only
Tokenserver supports Postgres at the moment, and the Python based
integration tests have an implicit dependency on MySQL, so the CI jobs only
use Postgres where possible.

The jobs have been split up and named more explicitly. The hope is to
simply delete the mysql jobs in the future.

Testing

The Tokenserver unit tests running on Postgres should pass.

Issue(s)

Closes STOR-353 / #1791

chenba · 2025-10-21T13:24:33Z

.circleci/config.yml

          name: Install MySQL client
          command: sudo apt-get update && sudo apt-get install -y default-mysql-client
-  create-tokenserver-database:
+  create-tokenserver-database-mysql:


Naming the jobs more explicitly since we are introducing another db into the mix.

chenba · 2025-10-21T13:30:34Z

.circleci/config.yml

+        type: string
+    steps:
+      - run:
+          name: Compress and copy coverage data


This doesn't save time when saving data but does save a little bit of time when attaching workspace later.

chenba · 2025-10-21T13:45:25Z

.circleci/config.yml

+          RUST_BACKTRACE: 1
+          # XXX: begin_test_transaction doesn't play nice over threaded tests
+          RUST_TEST_THREADS: 1
+      - image: cimg/postgres:18.0


I don't remember any discussion on postgres versions.

We can do the same thing with pinning a Postgres version like we will with Spanner, but explicitly defining 18 like you have here is perfect

chenba · 2025-10-21T13:54:41Z

Dockerfile

+    if [ "$TOKENSERVER_DATABASE_BACKEND" = "postgres" ]; then \
+        TOKENSERVER_FEATURES="--features=tokenserver-db/postgres"; \
+    fi && \
+    cargo chef cook --release --no-default-features --features=syncstorage-db/$SYNCSTORAGE_DATABASE_BACKEND $TOKENSERVER_FEATURES --features=py_verifier --recipe-path recipe.json


Temporarily support the permutation of Spanner Syncserver and Postgres Tokenserver while we await for Syncserver Postgres support.

chenba · 2025-10-21T13:54:58Z

Dockerfile

+    if [ "$TOKENSERVER_DATABASE_BACKEND" = "postgres" ]; then \
+        TOKENSERVER_FEATURES="--features=tokenserver-db/postgres"; \
+    fi && \
+    cargo chef cook --release --no-default-features --features=syncstorage-db/$SYNCSTORAGE_DATABASE_BACKEND $TOKENSERVER_FEATURES --features=py_verifier --recipe-path recipe.json


Temporarily support the permutation of Spanner Syncserver and Postgres Tokenserver while we await for Syncserver Postgres support.

chenba · 2025-10-21T14:01:19Z

Dockerfile

    # update again now that we trust repo.mysql.com
    apt-get -q update && \
-    apt-get -q install -y build-essential $MYSQLCLIENT_PKG libssl-dev libffi-dev libcurl4 python3-dev python3-pip python3-setuptools python3-wheel python3-venv cargo curl jq pkg-config && \
+    apt-get -q install -y build-essential $MYSQLCLIENT_PKG $POSTGRES_PKG libssl-dev libffi-dev libcurl4 python3-dev python3-pip python3-setuptools python3-wheel python3-venv cargo curl jq pkg-config && \


Both MySQL and Postgres packages are installed because the Python based integration tests have an implicit dependency on MySQL.

Yeah, same issue with Spanner needing SQL, as we unfortunately learned with the public key fiasco...

chenba · 2025-10-21T14:39:47Z

tokenserver-db-postgres/src/models.rs

               AND downed = 0
               AND backoff = 0
-             ORDER BY LOG(current_load) / LOG(capacity)
+             ORDER BY CASE WHEN current_load = 0 THEN -1 ELSE LN(current_load) / LN(capacity) END


(The changes in this file are not directly CI related but they are needed for the Tokenserver unit tests on Postgres to pass.)

When LOG(0) is called in MySQL it returns a NULL along with a warning, and that behavior along with how MySQL's ORDER BY treats NULL made that query work. It does not work in Postgres. Postgres throws an error when the query tries to take the LOG of 0.

chenba · 2025-10-21T14:40:08Z

tokenserver-db-postgres/src/models.rs

               AND email = $4
               AND  generation <= $5
-               AND COALESCE(keys_changed_at, 0) <= COALESCE(<keys_changed_at i64>, keys_changed_at, 0)
+               AND COALESCE(keys_changed_at, 0) <= COALESCE($6, keys_changed_at, 0)


I think it was just a good ol' bug.

chenba · 2025-10-21T14:40:49Z

tokenserver-db-postgres/src/models.rs

-        params: params::GetOrCreateUser,
-    ) -> Result<results::GetOrCreateUser, DbError> {
-        TokenserverPgDb::get_or_create_user(self, params).await
-    }


Infinite call loop from trait impl delegation.

Good call. This was missed when we moved the get_or_create_user call into tokenserver-db-common since it was identical for MySQL and Postgres.

chenba · 2025-10-21T15:30:33Z

I'm considering this ready since the workflow is there. We'll update the jobs and commands as we progress with the Postgres work.

chenba · 2025-10-23T20:03:45Z

tools/integration_tests/pyproject.toml

 hawkauthlib = "^2.0.0"
 konfig = "^1.1"
 mysqlclient = "^2.2.7"
+psycopg2-binary = "^2.9.11"


Added this after #1855 was merged into master and this branch was rebased on master. Might not be necessary if it gets added in #1857 and that PR's merged first.

Either works for me, I'll add them regardless in #1857 along with the poetry lockfile, so both should end up looking the same.

taddes

👍 lgtm, I'll review again after the rebase for Dockerfile we discussed.

taddes · 2025-10-23T21:38:15Z

tokenserver-db-postgres/src/models.rs

-        params: params::GetOrCreateUser,
-    ) -> Result<results::GetOrCreateUser, DbError> {
-        TokenserverPgDb::get_or_create_user(self, params).await
-    }


Good call. This was missed when we moved the get_or_create_user call into tokenserver-db-common since it was identical for MySQL and Postgres.

taddes · 2025-10-23T21:38:30Z

tools/integration_tests/pyproject.toml

 hawkauthlib = "^2.0.0"
 konfig = "^1.1"
 mysqlclient = "^2.2.7"
+psycopg2-binary = "^2.9.11"


Either works for me, I'll add them regardless in #1857 along with the poetry lockfile, so both should end up looking the same.

chenba · 2025-10-24T18:04:29Z

tools/integration_tests/tokenserver/test_support.py

+        # SQLAlchemy 1.4+ wants postgresql
+        if db_url.startswith("postgres://"):
+            db_url = db_url.replace("postgres://", "postgresql://", 1)
+        engine = create_engine(db_url)


Thanks to @taddes for catching this.

Here’s hoping it works 😅🤞🍀

This commit enables Postgres builds and tests in CircleCI. Only Tokenserver supports Postgres at the moment, and the Python based integration tests have an implicit dependency on MySQL, so the CI jobs only use Postgres where possible. The jobs have been split up and named more explicitly. The hope is to simply delete the mysql jobs in the future.

taddes

👍 Thx for this @chenba !

chenba force-pushed the chore/tokenserver-postgres-tests-STOR-353 branch 4 times, most recently from a630225 to e87e670 Compare October 20, 2025 17:58

chenba closed this Oct 21, 2025

chenba force-pushed the chore/tokenserver-postgres-tests-STOR-353 branch from 79f9a21 to 0513c7b Compare October 21, 2025 02:43

chenba reopened this Oct 21, 2025

chenba commented Oct 21, 2025

View reviewed changes

chenba force-pushed the chore/tokenserver-postgres-tests-STOR-353 branch from eb5a174 to 14b0f6b Compare October 21, 2025 14:36

chenba commented Oct 21, 2025

View reviewed changes

chenba changed the title ~~[WIP][DRAFT] run tests on postgres~~ chore(ci): configure CircleCI for postgres builds Oct 21, 2025

chenba marked this pull request as ready for review October 21, 2025 15:29

chenba requested review from pjenvey and taddes October 21, 2025 15:29

taddes mentioned this pull request Oct 21, 2025

feat: tokenserver script postgres support #1857

Merged

chenba force-pushed the chore/tokenserver-postgres-tests-STOR-353 branch 3 times, most recently from fb59533 to ad11ea4 Compare October 22, 2025 19:13

chenba commented Oct 23, 2025

View reviewed changes

taddes previously approved these changes Oct 23, 2025

View reviewed changes

chenba dismissed taddes’s stale review via 6869436 October 24, 2025 02:43

chenba force-pushed the chore/tokenserver-postgres-tests-STOR-353 branch from ad11ea4 to 6869436 Compare October 24, 2025 02:43

chenba force-pushed the chore/tokenserver-postgres-tests-STOR-353 branch 4 times, most recently from a1bd24b to 45f3027 Compare October 24, 2025 18:03

chenba commented Oct 24, 2025

View reviewed changes

taddes previously approved these changes Oct 24, 2025

View reviewed changes

chenba dismissed taddes’s stale review via 3196137 October 24, 2025 21:48

chenba force-pushed the chore/tokenserver-postgres-tests-STOR-353 branch 3 times, most recently from 50e111d to a6fe88e Compare October 27, 2025 17:39

chenba force-pushed the chore/tokenserver-postgres-tests-STOR-353 branch 3 times, most recently from b895ede to 3047868 Compare October 27, 2025 20:35

taddes approved these changes Oct 27, 2025

View reviewed changes

Merge branch 'master' into chore/tokenserver-postgres-tests-STOR-353

1f5bceb

chenba merged commit 0461bce into master Oct 28, 2025
14 checks passed

chenba deleted the chore/tokenserver-postgres-tests-STOR-353 branch October 28, 2025 15:09

chore(ci): configure CircleCI for postgres builds #1840

chore(ci): configure CircleCI for postgres builds #1840

Uh oh!

Conversation

chenba commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing

Issue(s)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chenba commented Oct 21, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

taddes left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

taddes left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

chenba commented Oct 15, 2025 •

edited

Loading