How to Write an Effective AI Agent Skill: The Four-Layer Architecture

Most teams building AI agents spend 90% of their time on the code and 10% on the methodology that drives it. That ratio is backwards. A great AI agent skill is not defined by how many scripts it ships with. It is defined by the quality of the instructions baked into the SKILL.md file that an agent reads before it executes a single command. Get that file wrong and the agent drifts, skips steps, retests what it already covered, and generates incomplete output.

At Strobes, we have built skills that run complete security assessments autonomously, covering web penetration testing, API security, cloud configuration audits, and source code review. Here is the architecture we use and the lessons we learned building it.

What Is an AI Agent Skill?

A skill is a self-contained package that gives an AI agent a domain-specific methodology. It is not just a folder of scripts. It is a structured combination of instructions, tools, shared utilities, and persistent state that allows an agent to move through a complex, multi-phase workflow without hand-holding.

The analogy is straightforward: scripts are the hands, the SKILL.md is the brain. A well-built skill teaches the agent a proven methodology, not just a set of commands to run.

Every effective skill has four layers, each with a distinct job:

The Methodology Layer lives in SKILL.md. It contains the phases, decision trees, playbooks, and non-negotiable rules. This is what separates a great skill from a toolbox with no instructions. The Scripts Layer provides CLI tools the agent executes to take action. The Shared Library Layer contains reusable modules for database access and output formatting so every script behaves consistently. The Data Layer is a SQLite database that acts as the agent's persistent memory across phases.

Most skill authors spend almost all their time on the scripts layer and almost none on the methodology layer. That is the wrong approach. Flip the ratio.

Why the SKILL.md Is the Most Critical File

The SKILL.md is not a README for humans. It is a set of instructions loaded directly into the agent's context when the skill activates. Every section serves a functional purpose.

Phase methodology is where most authors under-invest. Each phase needs a one-sentence goal, numbered steps with exact commands, decision points with explicit if-this-then-that structure, and a clear statement of what feeds into the next phase. Vague instructions like "analyze the results" are useless to an agent. Exact instructions like "run analyze.py scan --target X and flag any results with severity above high" give the agent something to act on.

Decision trees eliminate the guessing that causes agents to produce inconsistent output. When an agent hits a fork in the workflow, it should have an explicit map:

What type of input are we working with?
├── Structured data (JSON, CSV) → Use parser.py with --format flag
├── Unstructured text → Use analyzer.py with --mode text
│   ├── Short (<1000 chars) → Use --batch single
│   └── Long (>1000 chars) → Use --batch chunked
└── Binary files → Use extractor.py first, then analyzer.py

Playbooks are end-to-end recipes for specific scenarios. They walk through identifying targets, executing with realistic inputs, verifying results, and recording output. The agent does not improvise well under ambiguity. Playbooks remove ambiguity before the agent encounters it.

Rules are the non-negotiable guardrails: always verify scope before executing, never overwrite existing results without confirmation, always record evidence alongside findings. Write them down explicitly. The agent has no conscience unless you give it one.

Why This Matters for Agentic Security Testing

The web penetration testing use case illustrates exactly why architecture matters. A web pentest is not one task. It is hundreds of coordinated decisions: tracking which endpoints have been discovered versus tested, managing a sitemap that grows as the agent crawls, running OWASP Top 10 testcases against each endpoint, fuzzing parameters, recording findings with evidence, and knowing the authentication token expires in 20 minutes.

Without a structured skill, the agent drowns. It retests endpoints it already covered. It misses entire vulnerability categories. The four-layer architecture solves this problem: the methodology layer defines phases and order of operations, the scripts layer provides tools for each action, the shared library keeps database access and output formatting consistent, and the SQLite database is the single source of truth that holds state across the entire assessment.

This is precisely how agentic pentesting works at Strobes. The orchestrator agent uses the skill to plan, track, and manage the overall assessment while spinning up specialized subagents to go deep on specific tasks. The orchestrator owns the plan. The subagents own execution. Neither role works well without the shared state layer that the database provides.

If you want to understand how we handle the hardest technical problem in AI-powered testing, read about why crawling is the hardest part of AI-powered pen testing and how we solved it. For a broader view of the agent stack, see how Strobes built an agent stack specialized for offensive security.

The Orchestrator-Subagent Split

One architectural decision that pays off repeatedly: the skill does not do everything itself. The orchestrator agent uses the skill to manage scope, track testcases, measure coverage, and generate reports. When it is time to actually execute a complex task, the orchestrator delegates to a subagent.

That subagent operates with full autonomy. It is not constrained by the skill's workflow. It can craft custom payloads, chain multiple requests, explore unexpected behavior, and follow a thread wherever it leads. When it finishes, it writes results to the shared database and returns control to the orchestrator.

This clean separation of responsibility is what allows Strobes AI to run security assessments with the rigor of an experienced practitioner. The orchestrator provides structure. The subagents provide depth. The shared state layer connects them without requiring constant context-passing through conversation.

For a deeper look at how we build the AI harness that makes this architecture reliable, see what it takes to turn LLMs into reliable pentest operators.

How to Build a Skill: The Right Order

Build in this sequence. Start with the methodology: define phases, key actions per phase, data that persists between phases, decision points, and non-negotiable rules. Then design the database schema around the entities and state your skill needs. Build the shared library layer next. Build scripts in phase order. Write the SKILL.md once you know what the tools actually do. Verify that every script reference, flag name, and example in the SKILL.md matches the actual code. Then test with the agent.

That last step is where the real work happens. Run end-to-end, identify every point where the agent gets confused, add clarity to the SKILL.md, and repeat. The agent is your QA team for the methodology. Confusion means the instructions are not precise enough.

Common Mistakes That Break Skills

Writing the SKILL.md as documentation is the most common error. It is not a README. It is imperative instructions optimized for an AI agent to execute. Every phase description should tell the agent exactly what to run and when to move on.

Vague phase descriptions break execution. "Analyze the data" tells the agent nothing. "Run analyze.py scan --target X and check if any results have severity above high" tells it everything it needs.

Missing decision trees leave the agent guessing at forks. Skipping a rules section means the agent will take shortcuts that produce incomplete or inconsistent output. Script flags that do not match the SKILL.md cause silent failures. Skipping the end-to-end test with the actual agent will surprise you every time.

Key Takeaways

• Spend more time on the SKILL.md than on the scripts. The methodology is the hard part.
• Every phase needs exact commands, decision trees, and explicit rules. Vague instructions produce unpredictable agent behavior.
• Use SQLite as shared state so orchestrators and subagents share context without passing it through conversation.
• The orchestrator-subagent split gives you both structure and depth: the orchestrator manages the plan, subagents execute autonomously.
• Test end-to-end with the actual agent. Every point of confusion is a gap in the methodology.
• The description field in your skill registration is not a docstring. It is the trigger condition. Write it like an if-statement.

Build the methodology first. The code follows.