Improvable agent skills
I use AI agents every day, and in the past few months, I’ve written a bunch of agent skills to make my team and me more productive. My first ones were pretty bad: they made the same mistakes over and over, didn’t always trigger, and had no way to evolve. I treat my tools like I treat anything in life: put feedback loops in place so I can improve over time. This post describes what I consider a good skill: one you can evaluate and improve.
Structure
In this post we’ll focus on a git-commit skill, committing changes with a sensible message and pushing them:
git-commit/
├── SKILL.md # instructions
├── eval.md # pass/fail checks
├── example.md # one worked run
└── memory.md # log of corrections
SKILL.md: the instructions
This file describes the skill itself: what it does, when it triggers, and the steps to follow. The steps don’t need to be verbose: the model can infer a lot from what is asked. The first and last steps are what make the skill improvable: it reads its lessons before starting and writes new ones before finishing.
---
name: git-commit
description: Commit the current changes with a conventional commit message and push them. Use when asked to "commit this", "commit and push" or "ship it". Not for opening pull requests or rewriting history.
---
# git-commit — Commit the current changes
## Step 1 — Load memory
Read `memory.md` and apply every lesson in it.
## Step 2 — Commit
Run `git diff`, write a conventional commit message like the one in `example.md`, then commit.
## Step 3 — Push
Push to the current branch. Never force push.
## Step 4 — Grade
Grade every check in `eval.md` and show the scorecard: one line per check, then `Result: PASS (n/N)`.
## Step 5 — Log corrections
If the user corrected anything, add one line to `memory.md`: `YYYY-MM-DD: <≤3 sentences>`.
Otherwise, say "memory: no corrections".
## Done when
Every check in `eval.md` PASSes.
eval.md: the success criteria
A list of binary checks that either pass or fail. No scores, no “mostly”: the run succeeds only if every check passes. This allows the agent to know when it’s done working.
# git-commit — Evals
Every check is binary: PASS or FAIL. A run succeeds **iff every check PASSes**.
| ID | Check | How |
|----|------------------------------------------------------|----------------------------------|
| E1 | Subject follows `type(scope): subject` in ≤ 72 chars | `git log -1 --format=%s` |
| E2 | Subject is imperative ("add", not "added") | read the first word |
| E3 | The commit is on the remote branch | `git status` is up to date |
| E4 | Nothing was force pushed | no `--force` in the commands run |
example.md: a reproducible example
One complete run: the input, the output, and the scorecard. Agents are good at copying a shape, so a single good example says more than a page of rules.
# git-commit — Example
## Input
> commit and push
## Output
feat(blog): add improvable agent skills post
Explain why skills should carry their own evals and memory.
## Scorecard
E1 PASS feat(blog): add improvable agent skills post
E2 PASS "add"
E3 PASS pushed to origin/master
E4 PASS regular push
Result: PASS (4/4)
memory.md: learning from corrections
Every time I correct the agent, the correction goes here. One line per day, three sentences at most. Memory is read on every skill invocation, so corrections take effect immediately. This information is also useful when reworking skills.
# git-commit — Memory
2026-10-02: Take the scope from the top-level directory, not the file name.
2026-10-05: The body was too long; keep it to 3 lines or fewer.
Wrapping up
A good skill is more than a prompt. With four small files, it says what to do, how to check it was done right, what good output looks like, and what went wrong before. Start with a simple skill, use it, correct it, and let it get better over time. I have created a /skill-editor skill following these principles. I highly recommend doing something similar!