← Về thư mục
📄 / / profiles / r-and-d / raw_evidence / OpenAI4S / scripts / README.md

Maintainer and release scripts

中文说明

Maintainer-facing scripts: environment setup, release validation, secret scanning, directory-documentation coverage, the captured response gates, an upstream-schema canary, the contributor wall, and one opt-in scientific operation. None of them is a native Agent tool, and no module under openai4s/ imports any of them.

Files

File Purpose
relocate_bundle_scripts.py Rewrites bundled Python console-script interpreters to adjacent runtime paths before signing/packaging.
probe_wsl_parity.py Opt-in WSL diagnostic using temporary synthetic data: secure-store availability, real host.bash/kernel behavior, Windows interoperability, source packaging, CLI PATH and corrupt-install recovery. Requires an explicit Windows scratch directory; exit zero means collection finished, not parity passed. See the bilingual parity audit in docs/.
build_macos_dmg.sh Packages a macOS .app and .dmg. The kernel spawns its worker through sys.executable, so freezing the app would break it; instead the bundle embeds a relocatable standalone CPython, ships the source tree as loose .py files, and pre-bakes the CORE science stack into the runtime so the first launch needs no network. Signing follows the release environment: a configured OPENAI4S_MACOS_SIGNING_IDENTITY is actually used (and codesign --verifyed), otherwise it falls back to an ad-hoc signature; the builder never notarizes — notarize_macos_dmg.sh is the only place that talks to Apple, and the release gate checks that evidence separately.
notarize_macos_dmg.sh Sign the DMG, notarytool submit --wait, staple, stapler validate, then spctl. Fail-fast precheck of the smallest Developer ID + notary secret set before any Apple service is contacted. Default unit tests never call the notary; they cover credentials, ticket/digest binding, and omission. The release workflow input macos_asset is notarized or omit (default omit: do not upload a preview DMG).
build_linux_bundle.sh Packages the Linux .tar.gz: the same embedded relocatable CPython, the same pre-baked science stack, the same loose source tree, plus a self-locating launcher, a .desktop template and a per-user install.sh. Deliberately not an AppImage — a FUSE-mounted squashfs is the wrong host for a program whose job is to spawn subprocesses under bubblewrap, and it would fail at cell-execution time rather than at launch. Cross-builds from macOS (nothing is compiled), skipping only the checks that need to run the target and saying so.
build_windows_zip.sh Packages the Windows .zip. Not a native Windows build: platform_support.py refuses to start a kernel on win32, so this ships a Windows launcher wrapped around the Linux bundle, which it installs into WSL2 on first run before opening the Windows browser at the forwarded port. It consumes the exact Linux artifact the same release publishes rather than building a second one that merely ought to match. Sources in windows/.
bundle_contract.py What every shipped desktop bundle must contain, in one place: the pre-baked package manifest, the runtime-only resources (Web UI, R worker, compute templates), separate mandatory floors for curated Skills and the pinned bioSkills collection, the hash-based bytecode rule, and the credential scan. Shared by all three verifiers for the same reason the two sandbox smokes share one implementation — two copies drift until one platform quietly stops checking what the others still do, and that platform is the one that ships the broken image.
capture_response_schemas.py Regenerates (or with --check, verifies) docs/response-schemas.json. Runs the offline suite with the capture installed and records what every route actually returned; a schema derived from real responses cannot describe a response the code does not produce, and coverage becomes a measured number rather than an assertion. --check fails on a change that would break a client — a dropped field, a lost guarantee, a widened type. Shapes that moved additively and routes that gained or lost coverage are printed but do not fail, because the capture also varies with which optional extras are installed and which tests a platform skips, and a gate that cries wolf gets regenerated until it means nothing. Either mode also lists by name the routes no offline test reaches, because a coverage count on its own is not something anyone can act on — and now that the list has reached empty, --check fails on it too: a route can no longer sit in the contract with no frozen shape, the way /frames/<id>/admissions/<id> did for forty-three commits while the list was merely printed as a metric. One provenance rule holds the artifact up: a test that stubs a service must carry the stubbed_backend marker, which pauses the recorder while it runs. Without it a stub's fabricated {"ok": true} is published as the route's real contract — provenance that is wrong rather than absent, which is the worse failure because a reader believes it.
check_directory_readmes.py The CI check that this file has to pass. Every maintained directory needs a README.md and a README_zh.md with the same heading sequence and the same table-row count, a backticked mention of each direct file and child directory, and relative links that actually resolve on disk. Runtime/cache directories, byte-exact fixtures, and vendored Web assets are ignored. The pinned skills/bioskills/ root and its boundary files stay checked, while only its mechanically generated recipe descendants are excluded.
check_css_tokens.py F-21: var(--x) references in openai4s/server/webui/style.css minus --x: declarations must be empty. Stdlib, no network. Wired as a lint job step and a local pre-commit hook. Reports each missing occurrence; an unfixed tree also prints the F-21 audit subset of 40 hard --text-100/--text-300 holes.
extract_webui_contract.mjs F-01 machine scan of the eight tests/browser_*.mjs harnesses. Writes the three E2E compatibility inventories (bare window globals, S field read/write, DOM selectors) into tests/webui-contract.md and refreshes the CONTRACT_GLOBALS typeof preamble in tests/browser_smoke.mjs. Repeatable and sort-stable; --check fails if either file drifted, naming a deleted global; --self-test exercises the scanner without a daemon.
connector_canary.py Asks UniProt, RCSB PDB and OpenAlex whether they still return what the connectors parse. Scheduled/manual only — a public API's outage is not a reason to fail a PR — and it exits non-zero only on real schema drift (a 200 whose required field is gone), never on an upstream being unreachable (timeout, 5xx, an HTML page). The outage-vs-drift distinction is the whole point and is tested offline with an injected fetch.
container_smoke.sh Builds the container image and proves the daemon works inside it, which is a different claim from "the image built". It checks the ways containerization has broken this program specifically: the daemon answers /health without a credential, refuses an unauthenticated API call, accepts the token it minted, runs as uid 1000, imports the science stack, and exits 0 on SIGTERM. The load-bearing case is the restart one — it forces a colliding pidfile onto the volume after a SIGKILL, because the singleton's old bare-liveness check made a container that had been killed refuse to start again, permanently. Needs a Docker daemon; runs the same on a laptop as in CI.
bundled_packages.txt The science stack pre-baked into every shipped desktop bundle, as <pip-name> <import-name> lines — the pip-installable superset of the default python.yml kernel env (rdkit, scanpy, numba, umap, single-cell, cheminformatics …). Single source of truth across platforms: both build scripts install the pip names and every verifier asserts each import resolves from inside the bundle, so neither the two sides nor the two platforms can drift. Torch/fair-esm and the conda-only R and bioconda tools are deliberately excluded.
make_app_icon.py Rebuilds assets/app-icon-1024.png from the brand mark's measured geometry — the five bonded atoms, the terminal block, the red prompt chevron and the cursor bar — as flat vector primitives, supersampled down onto the Big Sur icon grid. The mark ships in the repository only as a 150px glyph and a 64px favicon, and neither survives being resampled up to the 1024px an .icns needs. Dev-only: it needs Pillow, and its committed output is what the DMG build actually consumes.
fold_remote.sh Protenix single-sequence folding on a pre-provisioned trusted GPU host, offline and without MSA. Writes model.pdb, model.cif, confidence.json and plddt.csv, then prints a one-line JSON manifest and the deliverables base64-encoded on stdout, so the caller can harvest everything from the log. Opt-in.
import_bioskills.py Deterministically converts the audited, pinned GPTomics/bioSkills checkout into skills/bioskills/: 561 collision-safe directories, Codex-style scripts/ and references/, OpenAI4S read-only/provenance frontmatter, and a SHA-256 manifest. It never fetches the network; it refuses a dirty checkout, the wrong commit, a changed skill count, duplicate declared names, or a non-empty destination, and reads imported bytes from the pinned Git objects rather than from a platform-filtered working tree. --check re-derives every recorded hash against the committed tree, which is what makes the manifest a gate rather than a claim about a commit.
render_skill_install_sections.py Renders (or with --check, verifies) the ## Install / ## 安装 section on every Skill page under skills/, with that page's own directory name, the Node floor from cli.mjs, the repository from source.mjs and the collection's member count substituted. One template per language, stamped per page: the ninety pages are a checked artifact rather than ninety hand edits, and tests/test_skills_installer_contract.py runs --check, so a Skill directory whose pages lack the section — or carry a hand-edited one — fails the suite until the script has been run.
release_import_smoke.py Imports the installed dependency-free wheel with the isolated environment's interpreter from outside the checkout, and refuses to pass if the import resolved back to the source tree. It then checks what a plain import test misses: the packaged R worker, the compute templates and the Web UI, the four environment specs, both the curated Skill floor and the required 561-recipe bioSkills collection with its marker/manifest/license, working python -m openai4s --help and checkpoint-registry module entry points, and a core that still declares no non-extra dependencies.
setup_envs.sh A thin sh wrapper that execs python -m openai4s setup for the four conda environments. It forwards its arguments, so --only python and --dry-run work through it.
source_secret_scan.py Scans the release source tree for credential-shaped material and fails closed. It prints the detector name, path and line number, never the matched value. Dependency-free: git selects the candidate files, and a deterministic filesystem walk takes over where git is unavailable, such as an unpacked source archive.
protocol_fuzzer.py Feeds arbitrary bytes to the two untrusted wire decoders — the WebSocket frame reader and the share-tunnel control/data decoders — under Atheris coverage guidance. Bounded, fully offline, and reads no session data or credentials; .github/workflows/fuzz.yml runs it for 60 seconds on every pull request and 10 minutes weekly. Documented protocol rejections are expected; any other exception surfaces as a crash. Linux-only via the fuzz extra, so it does not burden ordinary setup.
update_contributors.py Rebuilds the Community Contributors wall. Fetches commit contributors from the GitHub API with the repository's own token, appends a maintained list of publicly recognized non-commit contributors, crops each avatar into a circular PNG under .github/contributors/, and rewrites the block between the CONTRIBUTORS markers in both root READMEs. Requires Pillow.
verify_macos_bundle.py Stdlib-only inspection of a built .app or .dmg, the contract the wheel checks cannot see. Attaches the image read-only, then fails closed on an embedded interpreter that did not relocate into the bundle, any CORE_PACKAGES import that is missing from the pre-baked runtime, an Info.plist that disagrees with openai4s.__version__, a missing Web UI, R worker, compute template or Skill catalog, a python -m openai4s --help that does not run offline, a code signature that does not verify, or any dotenv/credential-shaped material swept into the image.
verify_linux_bundle.py Stdlib-only inspection of a built .tar.gz or unpacked bundle. Fails closed on an archive with the wrong shape or a launcher that lost its executable bit, a version the three declarations disagree on, a missing science package, timestamp bytecode, an icon ladder that is not at the sizes it claims, a desktop entry nothing substitutes, or credential material in the archive. On a matching Linux host it does strictly more — it runs the embedded interpreter — and the report says which of the two depths it reached rather than letting a skipped check read as a pass.
verify_windows_zip.py Stdlib-only inspection of a built .zip or staged package, checking the failures a Windows package specifically invites: CRLF in the WSL bootstrap (which dies with bad interpreter on the user's machine, not here), a payload that no longer matches its checksum sidecar, macOS resource-fork litter, and — the one that matters most — a launcher that has grown a native Windows execution path that platform_support.py would refuse.
describe_macos_image.py Mounts a built .dmg and writes two sidecars beside it: the signature's authority chain read from codesign, and the package inventory of the runtime actually embedded in the image. The job that stages a release runs on Linux and has neither, so this evidence has to be produced where the image is built and travel with it — without it the release gate fell back to "the signing-identity variable is set", which an ad-hoc image satisfies.
verify_release_artifacts.py Stdlib-only inspection of a built wheel and sdist. Checks that the required packaged files are present and that nothing unsafe rode along (symlinks, bytecode, caches, .env files); both archive formats must carry the bioSkills collection marker and every manifest-listed file with its exact size and SHA-256, with no unmanifested payload. It then reads the wheel metadata for the MIT license, the four Project-URL entries, the openai4s console entry point, the platform-independent py3-none-any tag, the absence of the test suite, and the absence of core dependencies.
verify_release_tag.py Fails closed unless a vMAJOR.MINOR.PATCH tag matches both literal version declarations: [project] version in pyproject.toml and openai4s.__version__.
release_pipeline.py The release itself, as a script rather than workflow YAML. A step embedded in an event trigger can only be exercised by cutting a real release — tested by the very thing it is supposed to protect — so this runs on a laptop, in --dry-run, and under pytest. Everything irreversible is last: the GitHub flip happens after PyPI has the version, and refuses until the draft carries exactly the distributions PyPI carries.
run_quality_gates.py Runs every offline gate and writes a receipt bound to the checkout's SHA. The release pipeline used to answer "not run: the suite gated the build that produced these artifacts" while the build job ran no suite at all; staging now verifies this receipt and re-derives the commit itself rather than trusting the one written down.
revalidate_release_tag.sh Proves a release tag still names the commit the run froze, immediately before a boundary mutates GitHub, PyPI or ghcr. A tag is mutable and a platform build takes tens of minutes, so a job's checkout proves which source it built and nothing about a ref rewrite since. It force-fetches the remote tag into one fixed local name and asserts both the annotated object type and its peeled commit. This was three byte-identical inline blocks in release.yml; one copy per boundary means a fourth boundary silently gets none, and the gate that checked them asserted substrings of each copy rather than that they agree.
release_gates.py The canonical release quality-gate manifest, imported by both run_quality_gates.py (which runs it) and release_pipeline.py (which requires a receipt to describe exactly it). The producer used to own the list privately while the consumer compared nothing but exit codes, so a two-row receipt naming pytest with the argv ["pytest"] staged a release. Verification is exact-match: a missing, duplicated, unknown or command-substituted gate, a different schema version and a different manifest digest are each a hard failure. Also holds the check-suite attestation — the browser and Python support matrices plus the real Linux private-PID Python/R interrupt smoke are proven by GitHub's own check runs pinned to the release head_sha, with their run ids recorded, rather than re-executed. The interrupt smoke deliberately allows raw networking on the hosted runner, so it is evidence for PID targeting and persistence, not the broader Linux egress boundary.
release_receipts.py Build receipts and the stage attestation, plus the CLI the build jobs call. A build receipt binds one job's artifacts to the frozen source SHA and records the builder's OS/arch/interpreter, so staging can check that the wheel and the DMG are the same commit — every job used to check out the mutable tag independently with nothing comparing them. The stage attestation records the exact staged asset set and digests and travels through the workflow artifact store, because step_publish used to re-hash the draft against the draft's own SHA256SUMS: a document vouching for itself, which anything able to replace an asset could replace in the same motion.
reaudit_crosswalk.py Records, per closed row of docs/plan-crosswalk.json, a digest of the evidence files that row names — and with --check, refuses when any of them has changed since. 25 of the 48 closed rows rested on a test file modified after the audit the document declares, one across twelve commits, and every existing assertion passed on all 25 because they check that the file exists. Content digests rather than a commit SHA: a SHA cannot be checked from inside the commit that re-audits.
capture_response_contract.py The companion gate, asking a different question: capture_response_schemas.py freezes the shape of the JSON bodies the suite happened to provoke, while this freezes what kind of answer each route gives at all — json, stream, redirect, binary or empty — and with which statuses. It does not rerun the suite. It drives every route in the inventory straight at a real handler over a real Store with no parameters, so most answer 4xx, and that is the point: an error response is a promise too, and a route nothing can drive shows up as a missing entry rather than as an entry full of nothing. An inventory entry that its own concretisation cannot route is named and never counted as covered. docs/response-contract.json is captured rather than written by hand, for the same reason its companion is: a hand-maintained list of "these routes stream" is correct on the day it is written and silently wrong afterwards. Capturing pins OPENAI4S_SECRET_STORE=plaintext inside a temp data dir, because /search/config reads a credential and auto resolves to whatever the capturing machine has — a laptop with a keychain froze the route, a runner with neither skipped it, and the artifact then looked out of date everywhere except where it was written.

Subdirectories

Directory Purpose
windows/ The Windows launcher, as real reviewable files rather than heredocs inside a build script: OpenAI4S.cmd, openai4s.ps1, and the bootstrap.sh that runs inside WSL2. build_windows_zip.sh stages them with the line endings each side needs.

Where this fits

The release and security scripts check the control plane from outside it; none of them is part of it. fold_remote.sh is deliberately not a general deployment guarantee either. The registered remote-science services still run their own capability checks, and they must return an explicit error when the required remote installation is unavailable.