← Về thư mục
📄 / / usr / local / lib / hermes-agent / website / docs / developer-guide / adding-providers.md

sidebar_position: 5 title: "Adding Providers" description: "How to add a new inference provider to Hermes Agent — auth, runtime resolution, CLI flows, adapters, tests, and docs"

Adding Providers

Hermes can already talk to any OpenAI-compatible endpoint through the custom provider path. Do not add a built-in provider unless you want first-class UX for that service:

If the provider is just "another OpenAI-compatible base URL and API key", a named custom provider may be enough.

The mental model

A built-in provider has to line up across a few layers:

  1. hermes_cli/auth.py decides how credentials are found.
  2. hermes_cli/runtime_provider.py turns that into runtime data:
  3. provider
  4. api_mode
  5. base_url
  6. api_key
  7. source
  8. run_agent.py uses api_mode to decide how requests are built and sent.
  9. hermes_cli/models.py and hermes_cli/main.py make the provider show up in the CLI. (hermes_cli/setup.py delegates to main.py automatically — no changes needed there.)
  10. agent/auxiliary_client.py and agent/model_metadata.py keep side tasks and token budgeting working.

The important abstraction is api_mode.

Choose the implementation path first

Path A — OpenAI-compatible provider

Use this when the provider accepts standard chat-completions style requests.

Typical work:

You usually do not need a new adapter or a new api_mode.

Path B — Native provider

Use this when the provider does not behave like OpenAI chat completions.

Examples in-tree today:

This path includes everything from Path A plus:

File checklist

Required for every built-in provider

  1. hermes_cli/auth.py
  2. hermes_cli/models.py
  3. hermes_cli/runtime_provider.py
  4. hermes_cli/main.py
  5. agent/auxiliary_client.py
  6. agent/model_metadata.py
  7. tests
  8. user-facing docs under website/docs/

:::tip hermes_cli/setup.py does not need changes. The setup wizard delegates provider/model selection to select_provider_and_model() in main.py — any provider added there is automatically available in hermes setup. :::

Additional for native / non-OpenAI providers

  1. agent/<provider>_adapter.py
  2. run_agent.py
  3. pyproject.toml if a provider SDK is required

Fast path: Simple API-key providers

If your provider is just an OpenAI-compatible endpoint that authenticates with a single API key, you do not need to touch auth.py, runtime_provider.py, main.py, or any of the other files in the full checklist below.

All you need is:

  1. A plugin directory under plugins/model-providers/<your-provider>/ containing:
  2. __init__.py — calls register_provider(profile) at module-level
  3. plugin.yaml — manifest (name, kind: model-provider, version, description)
  4. That's it. Provider plugins auto-load the first time anything calls get_provider_profile() or list_providers() — bundled plugins (this repo) and user plugins at $HERMES_HOME/plugins/model-providers/ both get picked up.

When you add a plugin and it calls register_provider(), the following wire up automatically:

  1. PROVIDER_REGISTRY entry in auth.py (credential resolution, env-var lookup)
  2. api_mode set to chat_completions
  3. base_url sourced from the config or the declared env var
  4. env_vars checked in priority order for the API key
  5. fallback_models list registered for the provider
  6. --provider CLI flag accepts the provider id
  7. hermes model menu includes the provider
  8. hermes setup wizard delegates to main.py automatically
  9. provider:model alias syntax works
  10. Runtime resolver returns the correct base_url and api_key
  11. --provider <name> CLI flag accepts the provider id
  12. Fallback model activation can switch into the provider cleanly

User plugins at $HERMES_HOME/plugins/model-providers/<name>/ override bundled plugins of the same name (last-writer-wins in register_provider()) — so third parties can monkey-patch or replace any built-in profile without editing the repo.

See plugins/model-providers/nvidia/ or plugins/model-providers/gmi/ as a template, and the full Model Provider Plugin guide for field reference, hook idioms, and end-to-end examples.

Full path: OAuth and complex providers

Use the full checklist below when your provider needs any of the following:

Step 1: Pick one canonical provider id

Choose a single provider id and use it everywhere.

Examples from the repo:

That same id should appear in:

If the id differs between those files, the provider will feel half-wired: auth may work while /model, setup, or runtime resolution silently misses it.

Step 2: Add auth metadata in hermes_cli/auth.py

For API-key providers, add a ProviderConfig entry to PROVIDER_REGISTRY with:

Also add aliases to _PROVIDER_ALIASES.

Use the existing providers as templates:

Questions to answer here:

If the provider needs something more than "look up an API key", add a dedicated credential resolver instead of shoving logic into unrelated branches.

Step 3: Add model catalog and aliases in hermes_cli/models.py

Update the provider catalog so the provider works in menus and in provider:model syntax.

Typical edits:

If the provider exposes a live model list, prefer that first and keep _PROVIDER_MODELS as the static fallback.

This file is also what makes inputs like these work:

anthropic:claude-sonnet-4-6
kimi:model-name

If aliases are missing here, the provider may authenticate correctly but still fail in /model parsing.

Step 4: Resolve runtime data in hermes_cli/runtime_provider.py

resolve_runtime_provider() is the shared path used by CLI, gateway, cron, ACP, and helper clients.

Add a branch that returns a dict with at least:

{
    "provider": "your-provider",
    "api_mode": "chat_completions",  # or your native mode
    "base_url": "https://...",
    "api_key": "...",
    "source": "env|portal|auth-store|explicit",
    "requested_provider": requested_provider,
}

If the provider is OpenAI-compatible, api_mode should usually stay chat_completions.

Be careful with API-key precedence. Hermes already contains logic to avoid leaking an OpenRouter key to unrelated endpoints. A new provider should be equally explicit about which key goes to which base URL.

Step 5: Wire the CLI in hermes_cli/main.py

A provider is not discoverable until it shows up in the interactive hermes model flow.

Update these in hermes_cli/main.py:

:::tip hermes_cli/setup.py does not need changes — it calls select_provider_and_model() from main.py, so your new provider appears in both hermes model and hermes setup automatically. :::

Step 6: Keep auxiliary calls working

Two files matter here:

agent/auxiliary_client.py

Add a cheap / fast default aux model to _API_KEY_PROVIDER_AUX_MODELS if this is a direct API-key provider.

Auxiliary tasks include things like:

If the provider has no sensible aux default, side tasks may fall back badly or use an expensive main model unexpectedly.

agent/model_metadata.py

Add context lengths for the provider's models so token budgeting, compression thresholds, and limits stay sane.

Step 7: If the provider is native, add an adapter and run_agent.py support

If the provider is not plain chat completions, isolate the provider-specific logic in agent/<provider>_adapter.py.

Keep run_agent.py focused on orchestration. It should call adapter helpers, not hand-build provider payloads inline all over the file.

A native provider usually needs work in these places:

New adapter file

Typical responsibilities:

run_agent.py

Search for api_mode and audit every switch point. At minimum, verify:

Also search run_agent.py for self.client.. Any code path that assumes the standard OpenAI client exists can break when a native provider uses a different client object or self.client = None.

Prompt caching and provider-specific request fields

Prompt caching and provider-specific knobs are easy to regress.

Examples already in-tree:

When you add a native provider, double-check that Hermes is only sending fields that provider actually understands.

Step 8: Tests

At minimum, touch the tests that guard provider wiring.

Common places:

For docs-only examples, the exact file set may differ. The point is to cover:

Run the targeted tests (or use scripts/run_tests.sh, which runs each file in its own subprocess):

source venv/bin/activate
python -m pytest tests/hermes_cli/test_runtime_provider_resolution.py tests/cli/test_cli_provider_resolution.py tests/hermes_cli/test_setup_model_provider.py tests/run_agent/test_provider_parity.py -q

For deeper changes, run the full suite before pushing:

source venv/bin/activate
python -m pytest tests/ -n0 -q

Step 9: Live verification

After tests, run a real smoke test.

source venv/bin/activate
python -m hermes_cli.main chat -q "Say hello" --provider your-provider --model your-model

Also test the interactive flows if you changed menus:

source venv/bin/activate
python -m hermes_cli.main model
python -m hermes_cli.main setup

For native providers, verify at least one tool call too, not just a plain text response.

Step 10: Update user-facing docs

If the provider is meant to ship as a first-class option, update the user docs too:

A developer can wire the provider perfectly and still leave users unable to discover the required env vars or setup flow.

OpenAI-compatible provider checklist

Use this if the provider is standard chat completions.

Native provider checklist

Use this when the provider needs a new protocol path.

Common pitfalls

1. Adding the provider to auth but not to model parsing

That makes credentials resolve correctly while /model and provider:model inputs fail.

2. Forgetting that config["model"] can be a string or a dict

A lot of provider-selection code has to normalize both forms.

3. Assuming a built-in provider is required

If the service is just OpenAI-compatible, a custom provider may already solve the user problem with less maintenance.

4. Forgetting auxiliary paths

The main chat path can work while summarization, memory flushes, or vision helpers fail because aux routing was never updated.

5. Native-provider branches hiding in run_agent.py

Search for api_mode and self.client.. Do not assume the obvious request path is the only one.

6. Sending OpenRouter-only knobs to other providers

Fields like provider routing belong only on the providers that support them.

7. Updating hermes model but not hermes setup

Both flows need to know about the provider.

Good search targets while implementing

If you are hunting for all the places a provider touches, search these symbols: