← Về thư mục
📄 / / root / .hermes / skills / software-development / skill-creator / SKILL.md

name: skill-creator description: "Create new skills, modify and improve existing skills, and measure skill performance. Use when the user wants to create a skill from scratch, edit or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. Trigger on: 'create a skill', 'make a skill', 'turn this into a skill', 'improve this skill', 'update skill', 'skill for X', 'save this workflow as a skill'." version: 1.0.0 source: "https://github.com/anthropics/skills/tree/main/skills/skill-creator"

Skill Creator

A skill for creating new skills and iteratively improving them.

Core loop

Capture intent → Draft SKILL.md → Run test cases (with-skill + baseline)
→ Human review (eval viewer) → Improve → Repeat until satisfied
→ Optimize description → Package

Figure out where the user is in this loop and jump in. If they say "I want to make a skill for X", start from the top. If they hand you a draft, go straight to testing.

Step 1: Capture Intent

Extract from conversation history first (tools used, steps taken, corrections made). Then ask:

  1. What should this skill enable the agent to do?
  2. When should it trigger? (user phrases, contexts)
  3. What is the expected output format?
  4. Do we need test cases? (yes for objective outputs; optional for subjective ones)

Step 2: Write SKILL.md

Required frontmatter fields: - name: skill identifier (lowercase, hyphens) - description: This is the primary trigger mechanism. Include both what it does AND when to use it. Be slightly pushy — err toward over-triggering. Example: instead of "Builds dashboards", write "Builds dashboards. Use whenever user mentions dashboards, data visualization, internal metrics, or wants to display any company data — even if they don't say 'dashboard'."

Anatomy

skill-name/
├── SKILL.md             ← required, keep under 500 lines
└── (optional)
    ├── scripts/         ← deterministic/repetitive code
    ├── references/      ← docs loaded on demand
    └── assets/          ← templates, icons, fonts

Three loading levels

  1. Metadata (name + description) — always in context
  2. SKILL.md body — loaded when skill triggers
  3. Bundled resources — loaded on demand only

Keep SKILL.md under 500 lines. If approaching limit, split into references/ files with clear pointers.

Writing principles

Step 3: Run Test Cases

For each test case, spawn two subagents in the same turn: one with the skill, one without (baseline). Don't do with-skill first then come back for baseline.

Workspace layout

<skill-name>-workspace/
└── iteration-1/
    ├── eval-<name>/
    │   ├── with_skill/outputs/
    │   ├── without_skill/outputs/   (or old_skill/ when improving)
    │   ├── eval_metadata.json
    │   └── timing.json
    └── benchmark.json

Create directories as you go, not upfront.

eval_metadata.json

{
  "eval_id": 0,
  "eval_name": "descriptive-name-not-eval-0",
  "prompt": "The exact test prompt",
  "assertions": []
}

timing.json — capture immediately on subagent completion

{
  "total_tokens": 84852,
  "duration_ms": 23332,
  "total_duration_seconds": 23.3
}

This data comes only in the task notification. Capture it immediately.

Step 4: Draft Assertions (while runs are in progress)

Good assertions are objectively verifiable and have descriptive names. Don't force assertions on subjective outputs — use human review instead.

Save to evals/evals.json:

{
  "skill_name": "example-skill",
  "evals": [
    {
      "id": 1,
      "prompt": "User's task prompt",
      "expected_output": "Description of expected result",
      "assertions": []
    }
  ]
}

Explain assertions to the user while waiting for runs.

Step 5: Grade + Aggregate + Launch Viewer

  1. Grade — run grader (see agents/grader.md). grading.json must use fields: text, passed, evidence (exact names — viewer depends on these).

  2. Aggregate:

python -m scripts.aggregate_benchmark <workspace>/iteration-N --skill-name <name>
  1. Launch viewer (Cowork/headless: use --static <output_path> instead of server):
nohup python <skill-creator-path>/eval-viewer/generate_review.py \
  <workspace>/iteration-N \
  --skill-name "my-skill" \
  --benchmark <workspace>/iteration-N/benchmark.json \
  > /dev/null 2>&1 &

For iteration 2+: add --previous-workspace <workspace>/iteration-N-1.

IMPORTANT: Generate the eval viewer BEFORE evaluating outputs yourself. Get them in front of the human first.

Tell the user: "Two tabs — 'Outputs' to review each test case, 'Benchmark' for quantitative comparison. Come back when done."

Step 6: Improve the Skill

After reading feedback:

  1. Generalize — don't over-fit to test examples. The skill will run millions of times on varied inputs.
  2. Stay lean — remove instructions that aren't earning their weight.
  3. Explain why — reframe rigid rules as reasoning.
  4. Bundle repeated scripts — if all test runs independently wrote the same helper code, put it in scripts/.

Rerun into iteration-<N+1>/, launch viewer with --previous-workspace, repeat until: - User says they're happy - All feedback is empty - No meaningful progress being made

Description Optimization (after skill is stable)

Generate 20 trigger eval queries (8–10 should-trigger, 8–10 should-not-trigger). Near-misses make the best negatives — adjacent domains, keyword overlap but different need.

Run optimization loop:

python -m scripts.run_loop \
  --eval-set <path-to-trigger-eval.json> \
  --skill-path <path-to-skill> \
  --model <model-id-from-system-prompt> \
  --max-iterations 5 \
  --verbose

Apply best_description to frontmatter. Show user before/after + scores.

Hermes-specific notes

Reference files (load when needed)