How to validate skills, agents and AGENTS.md

A skill, agent or instructions file can be valid and still not work: it loads, but the AI never picks it, or picks it for the wrong request. This guide shows the two ways to find out: checks, which read the files against the published rules, and tests, which have an AI tool actually use them.

Checks and tests at a glance

Checks run in your browser, on every edit. Nothing is uploaded. Tests need a model, so your own AI tool runs them through Connect AI.

Check a project in Skill Builder

Open a project and choose Checks in the right-hand panel. It updates as you type and covers every skill, agent, rule and the project instructions:

• Errors break the format: a tool may not load the file (a name with capitals, a missing description, a link to a file that is gone).
• Warnings are official guidance a tool depends on (a description in the first person, a reference file only reachable through another one).
• Info is advice worth a look (a description that never says when to use it, text tied to a date).

Every finding has Why?, with the reason and the document it comes from: the Agent Skills specification, Anthropic’s authoring guide, or the Claude Code, VS Code and GitHub docs. Findings without a source are marked as Skill Builder’s own recommendation.

Many findings have a button that fixes them, such as Rename to…, Add link or Move to resource file. Select the file name under a finding to open the file it is about.

Before you download, the Export dialog says Draft while the checks still find errors and Checked once they do not. Every export is also security scanned, and a file with a secret is never exported.

Check a file without a project

The SKILL.md validator checks files you have not added to Skill Builder:

• Paste a SKILL.md, an agent file, an AGENTS.md or a CLAUDE.md, choose its type, and select Validate.
• Switch to Files, folder or zip to check a whole skill with its references/, scripts/ and assets/, or several skills and agents at once. Each gets its own report, file by file, with syntax checks for JSON, YAML, TOML, JavaScript, TypeScript, XML and Python.

Nothing is stored. Open in Skill Builder turns a file that passes the security scan into a project, so you can fix it with blocks and the one-click fixes.

Check from your AI tool, with an AI review

With Connect AI on and your AI tool paired (Claude Code, GitHub Copilot, Codex, Cursor or Claude Desktop), your AI tool runs the same checks, and adds a review only a model can do.

• Ask it to “check the skill in .claude/skills/pdf with Skill Builder”. It sends the folder to check_files. Nothing is added to Skill Builder.
• For the project open in Skill Builder, it calls validate, which returns exactly what the Checks tab shows.
• Run /review-skill for the whole routine: check, review, fix, check again and summarise. It works on files in your repository and on what is open in Skill Builder.

Both checks also return an AI review rubric, and your AI tool applies it to the files. It covers what coded rules can not judge:

• Skills: whether the description would be picked for the right requests and not near-misses (it suggests some of each), whether the body does what the description promises, clear steps, the right level of detail for fragile work, contradictions, missing inputs, outputs or checks, examples, padding, when to read each file, overlap with other skills in the project, and actions that could cause harm without asking.
• Agents: when work would be handed to it, whether its prompt stands on its own, tools that match its job, specific boundaries, what it hands back, and its skills.
• Instructions and rules: whether every command and folder is true of the repository, specific rules, what belongs in a skill or a scoped rule instead, file patterns that match the right files, and conflicts between files.

Each problem it finds comes with the criterion, a short quote as evidence and a fix, kept apart from the rule findings. It is a model’s judgment, so read it as advice; the tests show whether a change really helped.

In VS Code Copilot Chat the slash command is /mcp.skill-builder.review-skill; in Claude Code it is /mcp__skill-builder__review-skill.

What the checks look at

Skills

• Name: 1 to 64 lowercase letters, numbers and single hyphens, the same as the folder name. VS Code does not load a skill whose name differs from its folder.
• Description: present, at most 1024 characters, in the third person, saying what it does and when to use it, with no XML tags.
• Size: SKILL.md under 500 lines; long sections moved to reference files.
• Files: every reference file linked straight from SKILL.md, reference files over 100 lines starting with a table of contents, every script and data file mentioned, no broken or moved paths.
• Text: no dated conditions (“before August 2025”), forward slashes in paths.
• Fields: ones outside the six in the specification are noted, because only Claude Code and Copilot read them and claude.ai uploads refuse them.

Agents

• Name and description: the same rules as skills. The description should name the situation it is for, not just a role.
• Fields: spelled the way Claude Code reads them (disallowedTools, not DisallowedTools).
• Tools: ones that exist, and that fit the role. An agent described as read-only that can still use Write, Edit or Bash is flagged, because only its tools list enforces that.
• Output: what it hands back, since only its final message returns to the caller.
• Skills: how much it preloads. Claude Code puts the full text of linked skills in its context every time it starts.

Project instructions and rules

• AGENTS.md: under 200 lines, with headings and the exact build, test and lint commands. No vague advice (“write clean code”), unfinished text, paths on one computer or secrets.
• Hints: “never” rules about risky actions (push, delete, deploy), which also need a hook or a permission rule to be enforced; rules a formatter already enforces; and rules repeated in Copilot’s instructions.
• Rule files: Copilot path instructions need Applies to, a Cursor rule has to apply somehow, and a Claude rule without paths loads in every session.

Test that it works

Checks read the files; tests watch an AI use them. Open a skill’s or agent’s details and expand Tests:

1. Under Should it be used?, add 8 to 10 requests that should use it, written the way people ask, and 8 to 10 near-misses: requests that share its words but need something else. “Convert this Word file to PDF” is a good near-miss for a PDF form-filling skill; “write a poem” is too easy to tell you anything.
2. Under Does it do the job?, add a few realistic tasks, what a good result looks like, and checks anyone could verify (“every form field has a value”).
3. In your AI tool, run /evaluate-skill (in Claude Code, /mcp__skill-builder__evaluate-skill pdf-forms). It runs each request in a fresh context, runs each task with and without the skill, and sends the results back.

Results show as counts, such as “8 of 10 trigger tests passed”, with the AI tool and model that ran them. Read them with three things in mind:

• Simulated means the AI tool could not start a fresh session with the skill installed, so it asked a fresh one which skill it would use. That is close, but not the same as real selection.
• Out of date means the skill changed after the run. Run the tests again before you trust the numbers.
• A failed near-miss usually means the description is too broad; a missed request means it is too narrow or uses different words than people do.

A skill’s tests are exported with it as evals/evals.json and evals/triggers.json, the format of Anthropic’s skill-creator, which AI tools do not load.

A routine before you share a skill

1. Open Checks and clear every error. Read Why? for each warning and fix the ones that apply.
2. Write the tests, run /evaluate-skill, and adjust the description until the right requests pass and the near-misses do not.
3. Save a version, so you can compare and go back.
4. Export and check the dialog says Checked.

Related guides

• How to write a SKILL.md
• Agent skills vs agents
• How to write an AGENTS.md
• Export skills for Claude Code, Copilot and Codex