Releasing Packages to PyPI

Marin publishes fourteen distributions to PyPI. marin-release-libs-wheels.yaml is their single build and publish workflow. Its dynamic matrix handles four release families:

• The nine general pure-Python libraries share one version and exact sibling dependency pins.
• Dupekit and Finelog each publish a pure-Python distribution with a native companion.
• Iris publishes its native companion independently.

The distribution name (what you pip install) carries a marin- prefix so the names don't collide on PyPI, which has no namespaces. The import name (what you import) is unchanged.

Distribution	Import	Source
marin-core	marin	lib/marin
marin-iris	iris	lib/iris
marin-fray	fray	lib/fray
marin-haliax	haliax	lib/haliax
marin-levanter	levanter	lib/levanter
marin-rigging	rigging	lib/rigging
marin-zephyr	zephyr	lib/zephyr
marin-finestore	finestore	lib/finestore
marin-shellbox	shellbox	lib/shellbox
marin-finelog	finelog	lib/finelog
marin-finelog-server	finelog_server	lib/finelog/rust
marin-dupekit	dupekit	lib/dupekit
marin-dupekit-native	dupekit_native	lib/dupekit/rust
marin-iris-native	iris_native	lib/iris/rust

All publishing uses OIDC trusted publishing. There is no API token stored in the repository, in GitHub secrets, or anywhere else. At workflow runtime GitHub mints a short-lived OIDC token, PyPI validates it against the publisher binding configured on each project, and PyPI issues a one-shot upload token that expires when the run ends.

How releases happen

Trigger	Result
Native source change merged to main	Package-specific dev release, followed by an automatic compatibility-floor and uv.lock pull request.
Daily pure-library schedule (06:00 UTC)	Coupled pure-library development release, published.
Push a package release tag	Stable release at the tag version, published.
workflow_dispatch (manual mode)	Build-only smoke; nothing is published.
Pull request touching package build inputs	Build the selected family; native wheels are installed and imported; nothing is published.

Stable tags are marin-libs-v<X.Y.Z>, dupekit-v<X.Y.Z>, finelog-v<X.Y.Z>, and iris-native-v<X.Y.Z>.

The nine general libs always share one version per build, and each published wheel pins its sibling marin-* dependencies to that exact version.

Native implementation pull requests compile their changed Rust sources in unified-unit and in the package release workflow. The follow-up dependency pull request changes only the consumer compatibility floor and uv.lock, so CI exercises the newly published wheels before they become the repository default. The shared update branch serializes releases from different native packages and uses a GitHub App token so its pull request triggers normal CI.

Package-family differences are declared in PACKAGES in scripts/ci/package_release.py: source paths, version files, distributions, build legs, import probes, and optional dependency-floor targets. Adding a family extends that registry and adds its PyPI projects' Trusted Publisher bindings to marin-release-libs-wheels.yaml; it does not require another workflow. Add build-driver code only when the family introduces a genuinely new build system.

Versioning

Declared package versions are floors. Development releases use one patch above the greatest supported declared or published version in the family. The GitHub run ID is the unique, retry-stable development serial. Looking across every distribution in a coupled family keeps a legacy development release or one diverged project from sorting above the new release.

After PyPI accepts a complete native family release, automation raises its consumer dependency floor and locks that exact registry version. A targeted lock validation rejects unrelated package churn. The general Python family does not need a follow-up pull request because its source packages remain the workspace defaults.

To cut a stable release, pick the next SemVer version and push the tag — no pyproject.toml edit required:

git tag marin-libs-v0.2.0
git push origin marin-libs-v0.2.0

PyPI rejects re-uploading an existing (name, version) pair, so every stable release must use a fresh version.

One-time PyPI setup

This is performed once by a PyPI admin who owns the marin-community organization. It cannot be automated from the release workflow.

1. Organization

Ensure a marin-community PyPI organization exists with at least two human admins. Every marin-* project is owned by it.

For Shellbox, create the marin-shellbox project under the organization in PyPI's organization Projects page. Configure its trusted publisher before merging the release workflow change: the next general-library release includes Shellbox.

2. Clear the placeholder releases

The older marin-* libraries already exist on PyPI with placeholder releases: 0.99 for seven of them (marin-core, marin-iris, marin-fray, marin-haliax, marin-levanter, marin-rigging, marin-zephyr) and 0.1.0 for marin-finelog. Delete that release from each project (project page → Manage project → Releases → per-release Options → Delete). The project itself stays — its name, ownership, and any configured trusted publisher are preserved; the first real publish simply adds a new release.

PyPI permanently retires a deleted (name, version) pair — it can never be re-uploaded. The libs therefore ship starting at 0.2.0, which avoids both retired placeholder versions (0.99 and 0.1.0), so no project has to be deleted wholesale.

(marin-core is the distribution name for the top-level package; the unprefixed marin name is not used.)

3. Configure a trusted publisher for each distribution

Open each project's publishing page and add the publisher there:

https://pypi.org/manage/project/<name>/settings/publishing/

(A pending publisher can also create a project on first upload, but it does not reserve the project name or assign it to the organization. Create marin-shellbox under the organization first.)

Add a publisher with these values, choosing the workflow that publishes the project:

Field	Value
PyPI project name	the distribution name from the table above
Repository owner	marin-community
Repository name	marin
Workflow filename	marin-release-libs-wheels.yaml
Environment name	pypi-publish

Every distribution produced by the workflow needs its own binding. The publish job remains in this top-level workflow because PyPI Trusted Publishing does not currently support naming a reusable workflow as the publisher.

The existing general-library bindings already name marin-release-libs-wheels.yaml. Before enabling consolidated native publishing, change the five native project bindings (marin-dupekit, marin-dupekit-native, marin-finelog, marin-finelog-server, and marin-iris-native) to that same filename and the pypi-publish environment. No Google Cloud WIF or long-lived credential change is required.

For marin-shellbox, add a trusted publisher to that project's Publishing settings with the field values above. The general-library schedule and marin-libs-v* tags publish Shellbox at the same version as its siblings. No PyPI API token or GitHub secret is needed.

4. The pypi-publish GitHub Actions environment

The release workflow publishes through the pypi-publish deployment environment. It already exists for marin-dupekit. Recommended settings:

• Deployment branches and tags: restrict to main and the release tags (marin-libs-v*, dupekit-v*, finelog-v*, iris-native-v*).
• No required reviewer. Automated releases run unattended; a reviewer gate would block them on a manual approval click. Trust is anchored by branch protection on main plus the publisher binding pinning a specific workflow filename and environment.

Secrets posture

No long-lived PyPI credential exists in the repository, in GitHub secrets, or in GitHub variables.

• The trusted-publisher binding is public information, shown on each PyPI project's publishing page.
• The GitHub OIDC trust config lives in the committed workflow files.
• The per-run PyPI upload token lives only in RAM and expires (~15 min) when the workflow ends.
• The only secret that must be shared is the 2FA recovery codes for the PyPI account(s) that administer the marin-community organization. Store these in the team password vault.

Storing a classic PYPI_API_TOKEN in GitHub Actions secrets is explicitly rejected: it rotates poorly, any workflow on the repo can read it, and it needs manual revocation if exposed.

Troubleshooting

• gh-action-pypi-publish fails with 403 for one project. Its trusted-publisher binding is missing or a field does not match. Re-check the four fields in step 3 — they must match the workflow exactly.
• A family release stopped after some files uploaded. Rerun the failed jobs in the same workflow run. The version is stable across reruns, and the preflight accepts only remote files whose hashes match the complete family manifest. The publisher skips those matching files and uploads the rest.
• A development release is not picked up by pip install --pre. The dev build must sort above the latest stable. Confirm the stable on PyPI is not ahead of what scripts/ci/package_release.py plan computes for the family.