← Về thư mục
📄 / / profiles / r-and-d / raw_evidence / OpenAI4S / openai4s / README.md

openai4s package

中文说明

This is the top-level Python package. The core is implemented; where a surface is still a Prototype, or where an operation can only ever end Partial by design, it is marked as such where it is described. The outer agent loop, the native JSON control tools, the persistent scientific kernels, the Host RPC services, storage, security, and the Web/CLI adapters all hang off this directory, and the control plane that composes them is standard-library only.

Where this fits

OpenAI4S has two nested loops. The outer loop in agent/ accepts at most one routed action per model step: an ordered native-tool batch, an Engine-owned finalize_response, or one complete Python/R cell. The inner loop in kernel/ keeps the language workers alive and answers synchronous host.* calls while a Python cell is still running. host_dispatch.py is the compatibility and composition boundary between those two planes; the behavior behind it lives in the focused services under host/.

Control-only work can finish through the Engine-owned finalizer. From inside a Python Cell, host.submit_output(...) is the only thing that completes a task, and a later sole valid finalize_response can still close the Engine after earlier Cells have run. Ordinary prose, native-tool results, R cells, cancellation, and turn exhaustion are not themselves completion signals.

Files

File Responsibility
__init__.py Names the package and holds its version. Importing it starts nothing.
__main__.py Makes python -m openai4s work by handing off to the CLI entry point.
artifact_restore.py The one Artifact restore path, shared by the native and Web routes. It verifies the historical snapshot before it copies those bytes back inside the workspace. What it records is a fresh version. History is never rewritten. The trusted snapshot roots — every directory the daemon itself writes immutable snapshots into, session imports included — are derived in one place, because two hand-maintained lists is how a directory comes to be written to and then refused on read. That boundary is containment, not integrity: the bytes are still checked against the version row's recorded sha256 and size on every read. Its refusals are typed, so a message this module wrote ("checksum verification failed") reaches the caller while the text of an escaped OS error, which routinely names an absolute path, does not.
bash_capability.py Holds the language-neutral version marker and command digest that bind a short-lived, one-shot shell capability.
capabilities.py Resolves whether a scoped capability or specialist profile is enabled, going through repository ports.
config.py Loads .env with no dependencies and defines the LLMConfig, SecurityConfig, ShareConfig, and global Config dataclasses. Only the LLM credential, base URL, and model id resolve through a layered chain: the per-provider variable, then the generic OPENAI4S_LLM_* one, then the provider's built-in default — and for the key, the provider's own conventional variable (ANTHROPIC_API_KEY, OPENAI_API_KEY, …) as a last resort. Every other field follows its own default: ports and turn budgets read one variable and fall back to a literal, data_dir and skills_dir fall back to computed paths (~/.openai4s and the repo's skills/), and egress_allowlist reads no variable at all because it is copied from egress.EGRESS_GROUPS. ensure_dirs() also hardens the data dir and every subdirectory it creates to owner-only: it holds the credential database, the artifacts, and the logs, and at the process umask every local account could read all of it.
datapro.py The single managed Volcengine DataPro connector: fixed public transport metadata, SecretBroker-only Agent Plan Key resolution with an Ark-only fallback, strict structuredContent.code interpretation, credential redaction, and result-Artifact payloads shared by Web and Agent calls.
doubao_search.py The fixed, pure-stdlib Doubao Search client. It resolves the shared Agent Plan Key from the calling Store immediately before each bounded POST, refuses redirects, scrubs reflected credentials, and normalizes verified Result.WebResults for the existing web_search tool.
endpoint_identity.py One spelling for an LLM endpoint, and the one place that strips its secrets. base_url was stored, published by GET /model-profiles, and frozen into an immutable revision exactly as typed — so a URL carrying userinfo or a query token put a credential in all three, which is 7.2's "secrets do not enter the snapshot" violated through the endpoint rather than the key. Normalising at the point of storage means nothing has to remember to redact it later.
execution_principal.py Who a unit of work is running as, carried with the execution in a ContextVar the way the correlation id is. Team mode answered "who is this request?" at the HTTP boundary and then lost it, so host.frames read every tenant's rows. Three properties make it a boundary rather than a convenience: absent is refused (resolve() raises in team mode), None is never an operator (the single-user daemon and the loopback CLI carry explicit principals), and it is stored on nothing reused — a HostDispatcher serves every turn in a session, so an identity field there would be one turn answering another turn's authorization question.
egress.py The Host-owned allowlist of outbound domains. The Web and shell policy boundaries consult it, but it is opt-in: unless OPENAI4S_EGRESS is set to an enforcing value (allowlist, on, 1, enforce, …), the mode is off and outbound calls fail open with no allowlist check. When it is on, it complements the OS sandbox rather than replacing it.
host_dispatch.py The compatibility and composition facade for the kernel's host_call RPC. A call passes permission, approval, audit, replay, screening, and step-event policy here before it reaches a focused Host service.
http_deadline.py A shared pure-stdlib absolute deadline for authenticated HTTP exchanges. Custom HTTP(S) connections register only their live socket with a short-lived watchdog, so TCP connect, proxy CONNECT, TLS, response status/headers, and body reads consume one wall-clock budget; the timer is cancelled and joined on every exit. System DNS resolution remains the documented non-cancellable stdlib boundary.
jobs.py Runs background compute jobs process-locally, caps how many may be active at once, and bounds captured output in bytes on a binary read — a character cap on a text pipe was trimmed only after readline had already allocated whatever the job printed before its next newline. A terminal status is decided in one place and never overwritten, and stopping a job goes through the same process-group logic as the kernel's shell executor, because the case that matters is the one where the shell exits and the work it started does not. The registry lives in memory, but each job also leaves an atomically written receipt under the data dir: a restart used to make a four-hour run simply vanish from the Jobs panel, so unfinished jobs are now readopted in the terminal state abandoned — not revived, and deliberately neither failed (which would blame the command) nor cancelled (which would claim somebody meant to stop it).
mcp_client.py The pure-stdlib MCP client facade and process-wide manager. Stdio JSON-RPC remains the default; an explicit managed connector can dispatch to the Streamable HTTP sibling. Connections cover tools, resources, and prompts, with absolute deadlines and per-connector single-flight construction. Server-initiated sampling remains out of scope.
mcp_http.py The pure-stdlib MCP Streamable HTTP transport: independent JSON-RPC POSTs, JSON/SSE responses, negotiated sessions and protocol versions, bounded bodies, policy/SSRF checks, and redirect refusal so authenticated headers cannot move to another origin.
mcp_protocol.py Shared MCP response-size bounds and typed transport failures used by both the stdio and Streamable HTTP clients.
onboarding.py The first-run model and provider configuration used by the headless CLI and by GET/POST /api/v1/onboarding. The API key is written and read through the secret broker, never as a plain settings row: after migration that row holds a reference, and a reference is truthy whether or not the keychain still has the value behind it — read raw, a key revoked by hand would still report as configured. The Web projection is redacted and zero-outbound.
permissions.py The process-wide permission broker. It resolves allow/deny/ask rules; when a user has to answer, it persists a durable approval request and blocks the turn, and it handles cancellation and timeouts. Unattended execution fails closed by default, and only by default: an operator who sets OPENAI4S_UNATTENDED_APPROVAL=allow opts into fail-open, and every unanswerable prompt is then allowed.
pkgscan.py Scans Python, conda, and R environments for normalized package availability, without importing any of those packages into the core.
platform_support.py Which platforms a kernel may start on, declared once. Windows is refused at the spawn path rather than warned about at onboarding — a program that warns and proceeds has made a different promise from one that refuses, and a half-working kernel is the worse outcome for a product whose claim is trustworthy results.
prompts.py The small single-purpose prompts the core sends on its own: compaction, review gates, provenance, Skill retrieval, extraction, editing, and security.
replay.py Records successful host.* results into an offline replay tape — internal plumbing calls such as provenance edges and credential reads are deliberately excluded — and detects call-order drift when an exported notebook replays that tape.
review.py Runs one bounded, tool-free review over the evidence of a completed turn and normalizes the JSON verdict. The reviewer cannot mutate the workspace.
scientific_reviewer.py Stage 3 Scientific Reviewer V2: independent snapshot-only context, strict pass/issues/incomplete schema, and a frozen provider/base_url/model fingerprint. An omitted artifact cannot pass.
specialists.py The built-in specialist profiles — one source of truth for both the catalog roster the gateway serves and the runtime persona/capability policy the delegation resolver applies. Host-layer importable; never imports the server. A stored agents-table row with the same name overrides its builtin.
store.py The compatibility facade over persistence. One SQLite connection lives here, and so do the schema and the guarded read queries. The focused storage repositories are handed that same connection and the same lock. Migrations are no longer part of the facade: they are versioned, transactional, and checksum-recorded in storage/migrations.py, because re-probing every table on every open and swallowing each failed ALTER made "is this database current" a question the code could not answer. close() is idempotent and evicts only that exact cached instance, so get_store(path) can open a fresh generation for the same path afterwards.
webtools.py Host-side web search, fetch, probe, and download. Transports are stdlib-first. Content conversion happens here, and this is also where the network switches, the SSRF checks, and egress enforcement bite — on every hop, which is why redirects are followed by hand instead of by urllib: a chain the opener walked internally was checked once at the first URL and not again. A capability that reaches the network without coming through here (a bundled Skill pulling an archive with raw urllib, an existence probe built on a different function) has skipped both the allowlist and the guard.

Subdirectories

Directory Responsibility
adapters/ Optional ecosystem adapters kept outside the stdlib runtime core.
agent/ The provider-neutral outer loop. It routes actions and finalizes them, compacts context past a token threshold, fans work out to sub-agents, and composes the local runtime.
cli/ Command-line lifecycle and one-shot task entry points, plus the environment, support, benchmark, and sharing commands.
compute/ Host-side registry and job orchestration for BYOC/remote compute. General remote compute remains a Prototype surface.
execution/ What a scientific cell goes through outside the kernel: admission, cancellation, dependency projection, result values, and timeout recovery.
host/ Focused services behind the HostDispatcher composition facade.
kernel/ Home of the persistent Python and R workers. The language-neutral manager protocol lives here too, along with environment selection, sandbox integration, and the in-cell Host RPC.
llm/ The provider-neutral LLM client. Capabilities, normalized messages and tools, and a stdlib transport sit above one wire adapter per provider.
mcp_servers/ Bundled pure-stdlib stdio MCP servers: the demonstration/test fixture example_server.py, and the deployable protein_design/ backend adapter whose heavy model dependencies stay outside the core.
orchestration/ The cluster control plane: what work was asked for, what resource one attempt at it was granted, and the backend Protocol any resource plane presents. Its defining property is what it does not contain — a checked rule (INV-2) keeps every scheduler word out of the core's source and import graph, so policy code cannot quietly grow scheduler-shaped assumptions.
sdk/ Compatible host facade injected into Python cells and the remote-compute namespace.
security/ Sandboxing and child-environment isolation. It also screens code and content, checks for injection, and carries the policy helpers those layers lean on. Each layer is independent, and several can fail open.
server/ The stdlib HTTP/WebSocket workbench: session services, projections, recovery, and the static UI. Recovery execution, the branch fork/activate/revert/undo controls, and all four Notebook export forms are wired end to end — each behind a capability gate that disables the control with a stated reason rather than letting it fail at the route, which is why Fork is offered only on a record carrying a proven checkpoint mapping. Partial here names a recovery outcome, not an unfinished feature: an arbitrary historical namespace is deliberately never serialized, so a recovery that cannot rebuild and verify one ends Partial by design.
share/ Web sharing transport: the tunnel wire protocol, a stdlib WSS client, the daemon's outbound TunnelClient, the stateless public relay, and the SSRF-hardened bundle fetch. The snapshot itself is built server-side in server/share_projection.py.
skills_loader/ Finds Skills and discloses them progressively: name and summary first, the body only on load. It also validates sidecars, installs versions, and rolls them back.
telemetry/ Opt-in anonymous telemetry, off by default. Counts and enumerations only, zero free text — and the enforcement is over values: {"error_type": "ValueError"} and {"error_type": "FileNotFoundError: /home/y/unpublished/cohort.csv"} pass the same key check. There is deliberately no domain that can hold free text, so adding such a field requires adding a domain class.
storage/ Focused SQLite repositories used through Store, and the versioned migration runner behind them.
benchmark/ The runner for the versioned science-workflow benchmark whose manifests live in workflows/. Every step drives production code — the real Store, kernel manager, host dispatcher and compute manager — and only what cannot run offline is injected: the model, the network, and a package manager. A declared outcome is part of the contract, so a case expecting failure fails on a clean run.
tools/ Class-based provider-native control tools. Each one carries its own schema. Around them sit the registry, the dynamic-tool lifecycle, and compatibility support for fenced calls.

Change rules

Trust Foundation modules