Skip to main content

How do I capture my agent skills' runs in Coolhand?

Last updated 7 min read

Install a Stop hook that runs `coolhand capture-turn` and Coolhand records each turn of your skill sessions in Claude Code or VS Code GitHub Copilot, with the skill's text and a workload per skill. It needs coolhand-cli 0.13.0 or later and your public API key, never the private one.

coolhand capture-turn is a command in the Coolhand CLI that an agent hook runs after every turn. When the session is running one of your skills, it sends that conversation to Coolhand, so you can see how the skill performs in real use. The person running the skill does not log in or install anything by hand.

It works with two agents today: Claude Code and VS Code GitHub Copilot (agent mode). The setup differs, and each has its own guide:

Capture is separate from Coolhand's usual ingestion of Claude Code and Copilot transcripts after a session ends (see Claude Code best practices and GitHub Copilot best practices). A hook sends each turn as it finishes, only for the skills you choose, and attaches the skill's text and a workload.

How it works

  1. A hook runs after every turn of an agent session. Claude Code and Copilot both pass it the path of the session transcript.
  2. coolhand capture-turn reads the transcript and checks whether a skill you asked it to capture was invoked in this session. If not, it sends nothing.
  3. If one was, it sends the conversation since the skill was invoked, using your Coolhand public API key.

The hook always exits quietly, so it cannot block or confuse the agent.

Choose which skills to capture

The hook command takes a --skill value:

Value Captures
'acme-*' Every skill whose name starts with acme-. Use this for a library with a shared name prefix.
'qa-*\|skill-for-qas' Several alternatives joined with \|, each a prefix pattern ending in * or an exact name: every skill starting with qa-, plus the skill skill-for-qas. Use this when a library's skills do not share one prefix.
'qa-login' Just that one skill.
'*' Every skill the person runs, including their own skills that are not part of your library. Each one gets a workload on your Coolhand account, so only use it if the people running the skills have agreed.

Always quote the value: the shell would expand an unquoted *, and an unquoted | starts a pipeline. A skill is recognised by its folder name or its name: in SKILL.md; if the two differ, the skill is never matched and nothing is sent.

Before you start

  • Use your Coolhand public API key in the hook. It can only write logs. Never use a private key: the CLI cannot tell them apart, and the key sits in a file that anyone who can read your skill can read.
  • You need coolhand-cli 0.13.0 or later. Patterns such as 'acme-*' need 0.13.1 or later for Copilot and 0.13.2 or later for Claude Code, and joining alternatives with | needs 0.13.3 or later for either agent. An older install treats the pattern as a literal skill name and sends nothing.
  • Install the CLI globally on every machine that runs the skills: npm i -g coolhand-cli@latest. Without it, the hook downloads and starts the package with npx on every turn. Node.js 20 or later is required either way.

What gets sent

Each turn is recorded as one request log. A whole session is one chat log in Coolhand: the newest turn replaces the stored one, and a late-arriving older turn never rolls it back.

What Details
Conversation Everything since the skill was invoked: the user's prompts, the model's thinking, its tool calls (name and input), the tool results, and its replies
The skill's text The body of the skill as the agent loaded it, so you can see which instructions produced a result
Workload Declared as claude-skill:NAME or copilot-skill:NAME, so every session of one skill lands in one workload
Token usage The running total for the session, where the agent reports it
Who ran it The tester's identity (see below) and the session's working directory
Skill version Only if the hook passes --skill-version

Not sent: subagent traffic, local command output (/compact, /context, !cmd), and text the agent harness injected rather than the user typing it.

Every string is passed through a secret scrubber first, which replaces recognised API keys, tokens and passwords with [REDACTED]. Tool results can contain file contents and command output, and redaction is pattern-based: it catches likely secrets, not everything. Tell people not to paste real secrets into a chat that uses a captured skill, and weigh this before capturing skills that read sensitive files.

Long conversations are trimmed to stay within size limits: the oldest exchanges are dropped first, and the log is flagged with history_truncated when that happens.

Who ran it

Turns from one shared public key are attributed to a tester, chosen in this order:

  1. COOLHAND_CAPTURE_USER, if the person set it in their environment.
  2. The person's git email (git config user.email).
  3. An anonymous machine id, a random UUID stored in ~/.coolhand/machine-id.

The git email is personal data. The skill author can add --anonymous to the hook command to keep it, and the working directory, off the wire. Someone running the skill can also set COOLHAND_CAPTURE_USER to any other identifier.

Turn capture off

Set COOLHAND_CAPTURE_DISABLED=1 (or true or yes) in the environment. The hook then reads nothing and sends nothing. For Copilot you can also delete ~/.copilot/hooks/coolhand-capture.json.

If a turn does not arrive

  • A turn that cannot be sent (offline, timeout) is saved in ~/.coolhand/pending/ and retried after the next successful send. Each turn is retried up to 3 times and dropped after 7 days.
  • Errors are appended to ~/.coolhand/capture-turns/hook.log. Look there first.
  • Progress per session is kept in ~/.coolhand/capture-turns/. After you run a skill, a claude-<session>-*.json file (Claude Code) appearing there is the quickest sign the hook is working.
  • A claimed skill that never appears is almost always a name mismatch between the folder, the name: and the pattern, or a CLI older than the pattern syntax needs.

Known limits

  • One skill is recorded per session: the first matching skill invoked. If a chat runs a second skill, it stays under the first.
  • A resumed Claude Code session (claude --resume) captures nothing until a matching skill is invoked again in it.
  • With a pattern, Claude Code captures a session only when the user invoked the skill with a slash command. A skill the model loads on its own does not count.
  • Pattern matching for Copilot has been checked against recorded transcripts but not yet in a live VS Code chat, and Windows has not been tested.
  • Copilot sessions are currently labelled as Claude Code in the Coolhand dashboard.

Related articles