Download OpenAPI specification:
LLM Request Logging API Documentation.
This API allows clients to log and store request/response data from Large Language Model (LLM) API calls
for monitoring, analysis, and debugging purposes. The API accepts any JSON structure in the raw_request
field, making it flexible for different LLM providers and use cases.
API uses API Key authentication. Clients are issued unique API keys that must be included in requests using one of these methods:
X-API-Key: your_api_key_hereAuthorization: Bearer your_api_key_hereContact your administrator to obtain an API key.
Every client is issued two independent keys:
ch_pub_...) — safe to embed in SDK setup snippets, agent-facing code, or
anywhere else it might be exposed to an end user. It only permits ingest-style writes (e.g.
logging a new request, submitting feedback) — it cannot read back any data.ch_priv_...) — required for read endpoints and other sensitive operations
(e.g. optimization evidence links). Never embed a private key in
client-side or agent-facing code; treat it like a server-side secret.Each endpoint below documents which key type(s) it accepts. Endpoints that require a private key are marked accordingly.
Log requests and responses from OpenAI's chat completions, embeddings, and other endpoints.
The flexible JSON structure supports any LLM provider's request/response format.
Aggregate usage statistics, token consumption, and performance metrics across LLM calls.
When interacting with the LLM Request Logging API, you'll encounter standard error response formats for different types of failures.
object Contains error details. |
{- "errors": {
- "__field_name__": [
- "Password should be at least 6 chars long",
- "Password should contain at least one number"
]
}
}Understanding the HTTP status codes for this API:
| HTTP Status Code | Description |
|---|---|
201 Created |
LLM request log successfully created |
400 Bad Request |
Malformed request or invalid JSON structure |
401 Unauthorized |
Missing, invalid, or expired API key |
422 Unprocessable Entity |
Validation error - required fields missing or invalid format |
5xx Server Errors |
Server-side issues - please report these for investigation |
API endpoints for logging and storing LLM (Large Language Model) request/response data
Creates a new LLM request log entry for monitoring and analysis purposes. See the Understanding an LLM Request Log guide for a full description of the fields Coolhand captures, which fields each provider exposes, and common gotchas. id is the log's hashid (a string), not the internal integer primary key — every log identifier returned by this API is a hashid, matching the GET endpoints below; consumers that previously parsed id as an integer need to treat it as an opaque string instead. A request made with the public API key is capped at 10MB for the whole request body (413 above that) and 255 characters for collector (422 above that); private keys are not capped.
| X-API-Key | string API key for authentication |
| Authorization | string Bearer token with API key |
required | object |
{- "llm_request_log": {
- "raw_request": { },
- "collector": "string",
- "metadata": { }
}
}{- "id": "string",
- "collector": "string",
- "source_api": "string",
- "source_application": "string",
- "metadata": { },
- "source_api_result": "string",
- "model": "string",
- "template_id": "string",
- "template_name": "string",
- "input_tokens": 0,
- "output_tokens": 0,
- "latency_ms": 0,
- "created_at": "2019-08-24T14:15:22Z",
- "updated_at": "2019-08-24T14:15:22Z",
- "ingest_evidence": { },
- "warnings": [
- "string"
]
}Returns this client's LLM request logs as a bare JSON array (unlike GET /api/v2/llm_request_log_feedbacks's { feedback:, pagination: } envelope - coolhand-cli's fetchLastSync relies on this response being Array.isArray, so the body shape is intentionally not wrapped). Pagination totals are exposed via X-Page/X-Per-Page response headers, plus X-Total-Count/X-Total-Pages when include_total=true is passed (omitted by default to skip the COUNT(*) query - analyze-claude-sessions's frequent per=1 dedup-cutoff poll never needs a total). Supports Ransack search via q[...] params (e.g. q[source_api_in][]=claude_code), sorting via q[s] (e.g. created_at desc for newest first; when no q[s] is supplied, the endpoint applies a stable default sort of id desc for newest first so pagination is deterministic - pass q[s] explicitly to override), and pagination via page/per (per_page also accepted; default 25, max 100). Requires the private API key - the public key is write-only and cannot read. id is always the log's hashid (not the internal integer PK) - the sort itself still orders by the real primary key internally, but a cutoff/cursor derived from results should key off created_at, not id, since hashids don't sort lexically in creation order. analyze-claude-sessions uses this to derive its dedup cutoff by requesting the newest claude_code/claude_cowork record (per=1), instead of a bespoke endpoint. Also supports named filters (applied on top of any q[...] filtering, not in place of it): template_id/workload_id (hashid), system_prompt_contains/user_prompt_contains (case-insensitive substring), model, source_api, source_api_result, project_path (exact match against metadata.project_path), unmatched_only, days_back (unset by default - unlike search_logs's MCP tool, index has always returned unrestricted results and does not implicitly apply a 30-day window), and include_prompts (adds system_prompt/user_prompt, truncated to 500 chars, to each result). Every result carries cost (USD, priced by the same SQL as the dashboard's cost figures, so tiered pricing, cached-token discounts and reasoning tokens are applied; null when the log has no tokens or its model has no pricing). since/until (ISO8601; since inclusive, until exclusive; a missing offset means UTC and a + offset must be URL-encoded) bound created_at and replace days_back when either is given. min_cost (USD) keeps only logs whose cost is at least that, and order=cost_desc sorts by cost, highest first (replacing any q[s] sort); both are limited to logs that can be priced (those with a non-null cost), failed requests included, unlike the metrics' cost totals. On a client with a lot of history, combine min_cost/order=cost_desc with since or days_back: the cost is computed per log, so an unbounded query can time out (504). A malformed value for any of them returns 422.
| q[source_api_in][] | Array of strings Filter to these source_api values (Ransack in-predicate) |
| q[s] | string Sort expression, e.g. 'created_at desc' |
| page | integer Page number |
| per | integer Records per page (default 25, max 100; per_page also accepted) |
| per_page | integer Alias for per, matching search_logs's MCP tool param name (same 25/100 bounds) |
| template_id | string Filter by template hashid |
| workload_id | string Filter by workload hashid (matches all templates in that workload) |
| system_prompt_contains | string Case-insensitive substring to match in the system prompt |
| user_prompt_contains | string Case-insensitive substring to match in the user prompt |
| model | string Filter by model name |
| source_api | string Filter by source API (e.g. 'openai', 'anthropic', 'vertex') |
| source_api_result | string Filter by result status: success, failed, operational, unmatched |
| source_application | string Filter by exact source_application |
| project_path | string Filter by exact match against metadata.project_path |
| unmatched_only | boolean Only return logs with no assigned template |
| days_back | integer Limit to logs created in the last N days (max 36500; no default - unrestricted unless given; ignored when since is given; with only until, the window is days_back long ending at until) |
| since | string <date-time> ISO8601 lower bound (inclusive) on created_at. A date alone or a timestamp without an offset is UTC; encode a |
| until | string <date-time> ISO8601 upper bound (exclusive) on created_at; must be after since. With only until, days_back (if given) sets how far back the window starts |
| min_cost | number Only logs whose per-log |
| order | string Value: "cost_desc"
|
| include_prompts | boolean Include truncated system_prompt/user_prompt in each result |
| include_total | boolean Include X-Total-Count/X-Total-Pages response headers (default: false, to skip the COUNT(*) query on this frequently-polled endpoint) |
| X-API-Key | string Private API key for authentication |
| Authorization | string Bearer token with the private API key |
[- {
- "id": "string",
- "collector": "string",
- "source_api": "string",
- "source_application": "string",
- "metadata": { },
- "source_api_result": "string",
- "model": "string",
- "template_id": "string",
- "template_name": "string",
- "input_tokens": 0,
- "output_tokens": 0,
- "latency_ms": 0,
- "created_at": "2019-08-24T14:15:22Z",
- "updated_at": "2019-08-24T14:15:22Z",
- "ingest_evidence": { },
- "cost": 0,
- "system_prompt": "string",
- "user_prompt": "string"
}
]Fetches full input/output content for a single log by its ID (hashid). Accepts any of this client's private credentials: the client's own private API key, a team member's UserClient private key, an AdminClientCredential, or the scoped, rotating key of the client's agent user (see Clients::AgentUserProvisioner) used by the change-suggestion coding agent to ground a fix in the log it was told to look at. Supports section/max_chars for large logs and search_query for snippet search. Only directly-collected logs are fetchable here (matching index's .client_logs restriction) - internally generated records (evals, synthetic logs) 404 even when the hashid is known.
| id required | string Log hashid |
| section | string full, beginning, or end (default: full) |
| max_chars | integer Maximum characters to return per content field |
| search_query | string Text to search for within the log content |
| include_thinking | boolean Include thinking/reasoning response content (default: false) |
| X-API-Key | string Any of this client's private credentials (see description) |
{- "id": "string",
- "url": "string",
- "collector": "string",
- "metadata": { },
- "model": "string",
- "source_api": "string",
- "source_application": "string",
- "source_api_result": "string",
- "template_id": "string",
- "template_name": "string",
- "input_tokens": 0,
- "output_tokens": 0,
- "latency_ms": 0,
- "cost": 0,
- "cost_breakdown": {
- "total_cost": 0,
- "input_cost": 0,
- "output_cost": 0,
- "cached_input_cost": 0,
- "cache_creation_input_cost": 0,
- "reasoning_output_cost": 0
}, - "created_at": "2019-08-24T14:15:22Z",
- "updated_at": "2019-08-24T14:15:22Z",
- "ingest_evidence": { },
- "system_prompt": "string",
- "user_prompt": "string",
- "output": "string",
- "truncated": true,
- "total_chars": {
- "system_prompt": 0,
- "user_prompt": 0,
- "output": 0
}, - "search_query": "string",
- "matches": {
- "system_prompt": [
- "string"
], - "user_prompt": [
- "string"
], - "output": [
- "string"
]
}, - "thinking_response": [
- "string"
]
}API endpoints for collecting user feedback on LLM request/response quality and accuracy
Creates a new feedback entry for LLM request monitoring and improvement purposes.
Field Guide - All fields are optional, but here's how to get the best results:
Reusing an existing placeholder: if original_output + coolhand_fingerprint_id match
an existing unrated feedback record, this request updates that record instead of creating a
new one. For a public-key caller, only like/sentiment/explanation/revised_output/
llm_request_log_id are applied to it — the rest of the request is ignored (and reported
in warnings), the same restriction PATCH /api/v2/llm_request_log_feedbacks/{id} applies
to a public key. llm_request_log_id stays writable here even though PATCH treats it as
immutable for every caller: it's already scoped to this client, so linking it in doesn't
cross the same boundary the other restricted fields do. Private key / OAuth Bearer token
callers can update every field on the existing record. This restriction only applies when
reusing an existing placeholder — a genuinely new record still accepts the full field set
from a public key.
Size limit: a request made with the public key is capped at 10MB (413 above that); private keys are not capped.
Response shape: a public-key caller receives only the record's id back (plus
warnings, and created_partial_id when a partial was created) — reading a record back is
a private-credential operation. Private key / OAuth Bearer callers get the full payload
documented below.
| api_key | string API key as query parameter |
| collector | string 🏷️ Optional collector identifier (can also be provided in request body) |
| X-API-Key | string API key for authentication |
| Authorization | string Bearer token with API key |
required | object |
{- "llm_request_log_feedback": {
- "llm_request_log_id": "string",
- "like": true,
- "explanation": "Great response! Very helpful and accurate.",
- "revised_output": "This is a dummy revised output",
- "llm_provider_unique_id": "req_1234567890abcdef",
- "original_output": "Dear John Doe,\n\nThank you for reaching out to us regarding the issue with your recent order #12345...",
- "client_unique_id": "email-service-1760025972258",
- "creator_unique_id": "user-789",
- "collector": "coolhand-node-0.1.0",
- "workload_hashid": "abc123def456",
- "sentiment": "like",
- "coolhand_fingerprint_id": "fp-abc123xyz",
- "focus_section": "This specific sentence needs improvement.",
- "focus_range": {
- "start": 100,
- "end": 150
}
}
}{- "id": "string",
- "client_id": "string",
- "llm_request_log_id": "string",
- "workload_id": "string",
- "like": true,
- "sentiment": "string",
- "creator_type": "string",
- "explanation": "string",
- "revised_output": "string",
- "llm_provider_unique_id": "string",
- "original_output": "string",
- "client_unique_id": "string",
- "creator_unique_id": "string",
- "collector": "string",
- "coolhand_fingerprint_id": "string",
- "created_at": "2019-08-24T14:15:22Z",
- "updated_at": "2019-08-24T14:15:22Z",
- "warnings": [
- {
- "message": "string",
- "timestamp": "2019-08-24T14:15:22Z"
}
], - "created_partial_id": "string",
- "feedback_partials": [
- {
- "id": "string",
- "llm_request_log_feedback_id": "string",
- "client_id": "string",
- "focus_section": "string",
- "focus_range": {
- "start": 0,
- "end": 0
}, - "sentiment": "like",
- "like": true,
- "explanation": "string",
- "creator_unique_id": "string",
- "coolhand_fingerprint_id": "string",
- "created_at": "2019-08-24T14:15:22Z",
- "updated_at": "2019-08-24T14:15:22Z"
}
]
}Returns this client's feedback entries. Supports Ransack search via q[...] params (e.g. q[sentiment_eq]=2 - sentiment is stored as the integer codes 0=dislike/1=neutral/2=like, which is what responses render as the strings 'dislike'/'neutral'/'like' - or q[created_at_gteq]=...), sorting via q[s] (e.g. created_at desc; when no q[s] is supplied, the endpoint applies a stable default sort of id desc so pagination is deterministic), and pagination via page/per (default 25, max 100). Filtering and sorting on original_output/revised_output are not supported. Accepts any of this client's private credentials - the client's own private API key, a team member's UserClient private key, an AdminClientCredential, or the scoped, rotating key of the client's agent user (see Clients::AgentUserProvisioner) used by the change-suggestion coding agent to ground a fix in the feedback it was told to look at - but not the public key, which is write-only and cannot read. List items omit original_output/revised_output (which can each hold up to 1GB) - fetch a specific item via GET /api/v2/llm_request_log_feedbacks/{id} to get the full text and any feedback_partials. Also sets X-Total-Count/X-Page/X-Per-Page/X-Total-Pages response headers - the same pagination signal every paginated v2 endpoint exposes (see GET /api/v2/llm_request_logs); the body's pagination object is kept for backward compatibility, not the pattern to rely on for new integrations.
| q[sentiment_eq] | integer Filter by sentiment integer code: 0 = dislike, 1 = neutral, 2 = like |
| q[workload_id_eq] | string Filter by workload hashid (the same value rendered as each item's workload_id) |
| q[s] | string Sort expression, e.g. 'created_at desc' |
| page | integer Page number |
| per | integer Records per page (default 25, max 100) |
| X-API-Key | string Any of this client's private credentials (see description) |
| Authorization | string Bearer token with the private API key |
{- "feedback": [
- {
- "id": "string",
- "client_id": "string",
- "llm_request_log_id": "string",
- "workload_id": "string",
- "like": true,
- "sentiment": "string",
- "creator_type": "string",
- "explanation": "string",
- "llm_provider_unique_id": "string",
- "client_unique_id": "string",
- "creator_unique_id": "string",
- "collector": "string",
- "coolhand_fingerprint_id": "string",
- "created_at": "2019-08-24T14:15:22Z",
- "updated_at": "2019-08-24T14:15:22Z"
}
], - "pagination": {
- "current_page": 0,
- "per_page": 0,
- "total_count": 0,
- "total_pages": 0,
- "has_next_page": true,
- "has_prev_page": true
}
}Updates an existing feedback entry. Only specific fields can be updated - identity fields are immutable.
Partial Feedback Updates:
When partial_id is provided in the request body, the update is applied to that specific FeedbackPartial
instead of the parent feedback. This is useful when adding explanations to partial feedback (highlighted text sections).
The response will include updated_partial_id to confirm which partial was updated.
Updatable Fields (Parent Feedback) - public key:
Updatable Fields (Parent Feedback) - private key / OAuth Bearer token: Everything above, plus:
A public-key request that includes a private-key-only field does not error - that field is silently ignored, and only the public-key-allowed fields in the request are applied.
Response shape: a public-key caller receives only the record's id (reading a record
back is a private-credential operation - see GET /api/v2/llm_request_log_feedbacks/{id}),
plus warnings and, for a partial update, updated_partial_id so it can be confirmed. Private key
and OAuth Bearer callers get the full payload documented below. Any field the request sent
that the credential may not write is reported back in warnings.
Updatable Fields (FeedbackPartial - when partial_id is provided) - all credential types:
Partials have no creator_type - it is a parent-feedback field only, and is ignored here
for every credential type.
Immutable Fields (cannot be updated):
Note: After a successful update, if the feedback is not already matched to a log and has original_output, the fuzzy matching job will be re-triggered to attempt matching.
| id required | string Feedback hashid |
| X-API-Key | string API key for authentication |
| Authorization | string Bearer token with API key |
object |
{- "llm_request_log_feedback": {
- "like": true,
- "explanation": "string",
- "revised_output": "string",
- "original_output": "string",
- "llm_provider_unique_id": "string",
- "collector": "string",
- "client_unique_id": "string",
- "workload_hashid": "string",
- "partial_id": "string",
- "sentiment": "like",
- "creator_type": "human",
- "coolhand_fingerprint_id": "string"
}
}{- "id": "string",
- "client_id": "string",
- "llm_request_log_id": "string",
- "workload_id": "string",
- "like": true,
- "sentiment": "string",
- "creator_type": "string",
- "explanation": "string",
- "revised_output": "string",
- "llm_provider_unique_id": "string",
- "original_output": "string",
- "client_unique_id": "string",
- "creator_unique_id": "string",
- "collector": "string",
- "coolhand_fingerprint_id": "string",
- "created_at": "2019-08-24T14:15:22Z",
- "updated_at": "2019-08-24T14:15:22Z",
- "updated_partial_id": "string",
- "feedback_partials": [
- {
- "id": "string",
- "llm_request_log_feedback_id": "string",
- "client_id": "string",
- "focus_section": "string",
- "focus_range": {
- "start": 0,
- "end": 0
}, - "sentiment": "like",
- "like": true,
- "explanation": "string",
- "creator_unique_id": "string",
- "coolhand_fingerprint_id": "string",
- "created_at": "2019-08-24T14:15:22Z",
- "updated_at": "2019-08-24T14:15:22Z"
}
]
}Fetches a single feedback entry by hashid, including original_output, revised_output, and any feedback_partials - the full record that GET /api/v2/llm_request_log_feedbacks omits from list results. Accepts any of this client's private credentials - the client's own private API key, a team member's UserClient private key, an AdminClientCredential, or the scoped, rotating key of the client's agent user (see Clients::AgentUserProvisioner) used by the change-suggestion coding agent to ground a fix in the feedback it was told to look at - but not the public key, which is write-only and cannot read.
| id required | string Feedback hashid |
| X-API-Key | string Any of this client's private credentials (see description) |
| Authorization | string Bearer token with the private API key |
{- "id": "string",
- "client_id": "string",
- "llm_request_log_id": "string",
- "workload_id": "string",
- "like": true,
- "sentiment": "string",
- "creator_type": "string",
- "explanation": "string",
- "original_output": "string",
- "revised_output": "string",
- "llm_provider_unique_id": "string",
- "client_unique_id": "string",
- "creator_unique_id": "string",
- "collector": "string",
- "coolhand_fingerprint_id": "string",
- "created_at": "2019-08-24T14:15:22Z",
- "updated_at": "2019-08-24T14:15:22Z",
- "feedback_partials": [
- {
- "id": "string",
- "llm_request_log_feedback_id": "string",
- "client_id": "string",
- "focus_section": "string",
- "focus_range": {
- "start": 0,
- "end": 0
}, - "sentiment": "like",
- "like": true,
- "explanation": "string",
- "creator_unique_id": "string",
- "coolhand_fingerprint_id": "string",
- "created_at": "2019-08-24T14:15:22Z",
- "updated_at": "2019-08-24T14:15:22Z"
}
]
}Read-only API endpoints for listing, searching and inspecting the prompt templates an LLM request log is matched against. Template mutation stays on the MCP surface.
Returns this client's LLM request templates as a bare JSON array (the same body shape as
GET /api/v2/llm_request_logs, not the { feedback:, pagination: } envelope used by
GET /api/v2/llm_request_log_feedbacks). Pagination metadata is always exposed via the
X-Page, X-Per-Page, X-Total-Count and X-Total-Pages response headers — unlike
/llm_request_logs, this endpoint has no include_total opt-out, because a client's
template count is small enough that counting the templates themselves is cheap.
log_count is the expensive part of this response, not the pagination total. It
aggregates over llm_request_logs, so its cost scales with how many logs the matched
templates hold rather than with how many templates come back — the Unmatched bucket
reached via include_system=true can hold every log that never matched a template.
The 10-second guard is per statement, not per response — read this before sizing a
client timeout. #index issues three statements inside it (the page load, the
pagination COUNT, and the grouped log count), and Postgres applies statement_timeout
to each one independently. Database time for a single response can therefore reach three
times that figure before anything is cancelled, and a 504 here means one query
exceeded 10 seconds — never that the response as a whole did. Total response time is
not bounded by this endpoint. Do not treat the absence of a 504 as evidence a
response was fast, and do not derive a client timeout from the 10-second number: size it
from observed end-to-end latency instead. Narrow with workload_id, search, or a
smaller per if you do hit a 504.
log_count counts only directly-collected client logs — the same records
GET /api/v2/llm_request_logs?template_id=… returns — so the two numbers agree.
Internally generated records (evals, bakeoff comparisons, synthetic logs) are excluded,
which is why this can be lower than the count the search_templates MCP tool reports.
The client is always derived from the authenticating API key. There is no client_id
parameter and one cannot be supplied.
Requires the private API key — the public key is write-only and cannot read data back.
Results are ordered newest first (created_at DESC, with the internal primary key as a
tiebreaker so paging is deterministic when two templates share a timestamp).
Prompt patterns are not included by default. Pass include_patterns=true to add
user_prompt_pattern / system_prompt_pattern to every row, or fetch a single template
via GET /api/v2/llm_request_templates/{id}.
System templates. Every client has two system buckets, Unmatched and
Ignored API Calls (LlmRequestTemplate::SYSTEM_TEMPLATE_NAMES), created automatically
with the client. They are excluded from this list by default and returned when
include_system=true is passed — the Unmatched bucket is what you inspect when logs
are misrouting. Each row carries a system_template boolean so callers do not have to
match on names.
Deprecated templates are excluded by default and returned when
include_deprecated=true is passed. A deprecated row is identifiable by a non-null
deprecated_at. (The originating issue called this filter deprecated; it is named
include_deprecated here to match both include_system above and the existing
search_templates MCP parameter, and because deprecated=true would be ambiguous
between "only deprecated" and "also deprecated".)
Archived workloads are not filtered out by default, so the list agrees with
GET /api/v2/llm_request_templates/{id}, which can always fetch such a template by id.
Pass include_archived=false to hide templates whose workload has been archived (what the
search_templates MCP tool does by default).
Metrics. Pass include_metrics=true to add a metrics object to every row: the same
cost and performance numbers the dashboard's template table shows, computed by the same
SQL (LlmCostSql, so tiered pricing, cached-token discounts and reasoning tokens are
applied) in a fixed number of grouped queries for the whole page. Averages cover
non-failed directly-collected client logs inside the days_back window (default 28, the
dashboard's window, max 365); request_count and error_rate cover the same window;
first_request_at/last_request_at are lifetime; sentiment_score and
revision_score are all-time, as on the dashboard. avg_cost_per_request is null when no
log in the window could be priced. Metrics add one more set of aggregate statements to the
response, so expect it to be slower on a large page.
Window. days_back is a rolling lookback ending now. Pass since and/or until
(ISO8601; a missing offset means UTC, and a + offset must be URL-encoded as %2B)
for an explicit window instead: since is inclusive, until exclusive and defaulting to
now, and when since is given days_back is ignored and returned as null. With only
until, the window is days_back long ending there. The prior window behind
error_rate_change is the same length, immediately before the window. The resolved
bounds come back as metrics.since/metrics.until. A malformed timestamp, since not
before until, or a window over 365 days returns 422.
Additional counters over the same window, priced with the same cost expression as the dashboard:
failure_count (failed logs; error_rate is null when it is 0), total_input_tokens/
total_output_tokens (sums of the input_tokens/output_tokens columns over non-failed
logs), priced_request_count (non-failed logs with tokens whose model has pricing, i.e.
the logs total_cost and avg_cost_per_request cover; this excludes failed logs and logs with a zero token count, which the per-log cost on GET /api/v2/llm_request_logs does not, so summing that cost can exceed total_cost) and long_context_request_count
(priced logs that crossed their model's input-token pricing tier).
Not included in this endpoint: template version history. LlmRequestTemplate
versions user_prompt_pattern, system_prompt_pattern, name, status, workload_id
and the deprecation fields via PaperTrail, and a versions sub-resource was considered
alongside these two endpoints, but it is deliberately deferred to its own change
rather than shipped here — it is a distinct resource with its own response shape and
its own disclosure surface (historical prompt patterns). Template mutation likewise
stays on the MCP surface; this REST surface is read-only.
| search | string Case-insensitive substring match against the template name |
| workload_id | string Filter to a single workload, by workload hashid. A hashid that does not decode, or that belongs to another client, returns 422 rather than an empty list. |
| status | string Enum: "draft" "published" "failure" Filter by status: draft, published, failure. Any other non-empty value returns 422; an empty value is treated as no filter. |
| include_deprecated | boolean Include deprecated templates (default: false) |
| include_system | boolean Include the Unmatched / Ignored API Calls system templates (default: false) |
| include_archived | boolean Include templates of archived workloads (default: true) |
| include_patterns | boolean Include user_prompt_pattern and system_prompt_pattern on every row (default: false) |
| include_metrics | boolean Include a |
| days_back | integer Lookback window in days for |
| since | string <date-time> ISO8601 start (inclusive) of the |
| until | string <date-time> ISO8601 end (exclusive) of the |
| page | integer Page number |
| per | integer Records per page (default 25, max 100; per_page also accepted) |
| per_page | integer Alias for per, matching search_templates's MCP param name (same 25/100 bounds) |
| X-API-Key | string Private API key for authentication |
| Authorization | string Bearer token with the private API key |
[- {
- "id": "string",
- "name": "string",
- "status": "string",
- "version": "string",
- "group": "string",
- "workload_id": "string",
- "workload_name": "string",
- "system_template": true,
- "deprecated_at": "2019-08-24T14:15:22Z",
- "log_count": 0,
- "source_repository": "string",
- "source_file_path": "string",
- "created_at": "2019-08-24T14:15:22Z",
- "updated_at": "2019-08-24T14:15:22Z",
- "metrics": {
- "days_back": 0,
- "since": "2019-08-24T14:15:22Z",
- "until": "2019-08-24T14:15:22Z",
- "request_count": 0,
- "failure_count": 0,
- "error_rate": 0,
- "error_rate_change": 0,
- "avg_cost_per_request": 0,
- "total_cost": 0,
- "priced_request_count": 0,
- "long_context_request_count": 0,
- "total_input_tokens": 0,
- "total_output_tokens": 0,
- "avg_input_tokens": 0,
- "avg_output_tokens": 0,
- "avg_latency_ms": 0,
- "correctness_score": 0,
- "sentiment_score": 0,
- "revision_score": 0,
- "first_request_at": "2019-08-24T14:15:22Z",
- "last_request_at": "2019-08-24T14:15:22Z"
}, - "user_prompt_pattern": "string",
- "system_prompt_pattern": "string"
}
]Returns a single template belonging to the authenticating client, including
user_prompt_pattern and system_prompt_pattern — the full untruncated regexes, which
the list endpoint omits.
Unlike the list endpoint, this action applies no default filtering beyond client
ownership: a deprecated template or a system template (Unmatched,
Ignored API Calls) is reachable by id without any opt-in flag, because inspecting one
of those is the usual reason to fetch a template directly.
A template belonging to another client returns 404, not 403 — its existence is not disclosed.
As on the list endpoint, log_count is the expensive part: it aggregates over
llm_request_logs, and fetching the Unmatched bucket by id counts every log that never
matched a template. That count is the one statement bounded by the 10-second
statement_timeout, so on this action the guard does correspond to a single query — but
it still bounds only that query, not the whole response (the record lookup and
authentication run outside it). A 504 means the count exceeded 10 seconds. See the list
endpoint for why total response time is not bounded either. log_count counts only
directly-collected client logs, matching
GET /api/v2/llm_request_logs?template_id=….
Always includes a metrics object — the same cost and performance numbers the dashboard
shows for this template (see the list endpoint for the exact definitions). days_back
sets its lookback window (default 28, the dashboard's window; max 365); since/until
set an explicit window instead, exactly as on the list endpoint.
Requires the private API key.
| id required | string Template hashid (the |
| include_metrics | boolean Include the |
| days_back | integer Lookback window in days for |
| since | string <date-time> ISO8601 start (inclusive) of the |
| until | string <date-time> ISO8601 end (exclusive) of the |
| X-API-Key | string Private API key for authentication |
| Authorization | string Bearer token with the private API key |
{- "id": "string",
- "name": "string",
- "status": "string",
- "version": "string",
- "group": "string",
- "workload_id": "string",
- "workload_name": "string",
- "system_template": true,
- "deprecated_at": "2019-08-24T14:15:22Z",
- "log_count": 0,
- "source_repository": "string",
- "source_file_path": "string",
- "created_at": "2019-08-24T14:15:22Z",
- "updated_at": "2019-08-24T14:15:22Z",
- "metrics": {
- "days_back": 0,
- "since": "2019-08-24T14:15:22Z",
- "until": "2019-08-24T14:15:22Z",
- "request_count": 0,
- "failure_count": 0,
- "error_rate": 0,
- "error_rate_change": 0,
- "avg_cost_per_request": 0,
- "total_cost": 0,
- "priced_request_count": 0,
- "long_context_request_count": 0,
- "total_input_tokens": 0,
- "total_output_tokens": 0,
- "avg_input_tokens": 0,
- "avg_output_tokens": 0,
- "avg_latency_ms": 0,
- "correctness_score": 0,
- "sentiment_score": 0,
- "revision_score": 0,
- "first_request_at": "2019-08-24T14:15:22Z",
- "last_request_at": "2019-08-24T14:15:22Z"
}, - "user_prompt_pattern": "string",
- "system_prompt_pattern": "string"
}Read-only API endpoints for listing the workloads (groups of prompt templates) a client has, with per-workload counts and the same cost and performance rollup the dashboard shows.
Returns this client's workloads — the groups of prompt templates that make up one task or
agent — as a bare JSON array, ordered by name. Pagination metadata is exposed via the
X-Page, X-Per-Page, X-Total-Count and X-Total-Pages response headers.
Each row carries the workload's active-template count (template_count), its draft
count, log_count and last_activity across every log attached to any of its templates
(all generators, so log_count can exceed the sum of the templates' own log_count, which
counts only directly-collected client logs).
Archived workloads and the system workloads (Unmatched, Embedding Requests, …) are
excluded unless include_archived / include_system is passed.
Metrics. Pass include_metrics=true to add a metrics rollup per workload across all
of its templates: the same cost and performance numbers the dashboard shows, computed by
the same SQL (LlmCostSql, so tiered pricing, cached-token discounts and reasoning
tokens are applied) in a fixed number of grouped queries for the whole page. Averages
cover non-failed directly-collected client logs inside the days_back window (default 28,
the dashboard's window; max 365). See GET /api/v2/llm_request_templates for the field
definitions.
Window. days_back is a rolling lookback ending now. Pass since and/or until
(ISO8601; a missing offset means UTC, and a + offset must be URL-encoded as %2B)
for an explicit window instead: since is inclusive, until exclusive and defaulting to
now, and when since is given days_back is ignored and returned as null. With only
until, the window is days_back long ending there. The prior window behind
error_rate_change is the same length, immediately before the window. The resolved
bounds come back as metrics.since/metrics.until. A malformed timestamp, since not
before until, or a window over 365 days returns 422.
Additional counters over the same window, priced with the same cost expression as the dashboard:
failure_count (failed logs; error_rate is null when it is 0), total_input_tokens/
total_output_tokens (sums of the input_tokens/output_tokens columns over non-failed
logs), priced_request_count (non-failed logs with tokens whose model has pricing, i.e.
the logs total_cost and avg_cost_per_request cover; this excludes failed logs and logs with a zero token count, which the per-log cost on GET /api/v2/llm_request_logs does not, so summing that cost can exceed total_cost) and long_context_request_count
(priced logs that crossed their model's input-token pricing tier).
Pass include_templates=true to also list each workload's active templates with their
routing regex patterns.
The client is always derived from the authenticating API key. Requires the private API key.
| search | string Case-insensitive substring match against the workload name |
| include_archived | boolean Include archived workloads (default: false) |
| include_system | boolean Include system workloads like Unmatched and Embedding Requests (default: false) |
| include_templates | boolean Include each workload's active templates and their patterns (default: false) |
| include_metrics | boolean Include a |
| days_back | integer Lookback window in days for |
| since | string <date-time> ISO8601 start (inclusive) of the |
| until | string <date-time> ISO8601 end (exclusive) of the |
| page | integer Page number |
| per | integer Records per page (default 25, max 100; per_page also accepted) |
| per_page | integer Alias for per (same 25/100 bounds) |
| X-API-Key | string Private API key for authentication |
| Authorization | string Bearer token with the private API key |
[- {
- "id": "string",
- "name": "string",
- "description": "string",
- "archived": true,
- "system": true,
- "merged": true,
- "template_count": 0,
- "draft_template_count": 0,
- "log_count": 0,
- "last_activity": "2019-08-24T14:15:22Z",
- "templates": [
- {
- "id": "string",
- "name": "string",
- "status": "string",
- "user_prompt_pattern": "string",
- "system_prompt_pattern": "string"
}
], - "metrics": {
- "days_back": 0,
- "since": "2019-08-24T14:15:22Z",
- "until": "2019-08-24T14:15:22Z",
- "request_count": 0,
- "failure_count": 0,
- "error_rate": 0,
- "error_rate_change": 0,
- "avg_cost_per_request": 0,
- "total_cost": 0,
- "priced_request_count": 0,
- "long_context_request_count": 0,
- "total_input_tokens": 0,
- "total_output_tokens": 0,
- "avg_input_tokens": 0,
- "avg_output_tokens": 0,
- "avg_latency_ms": 0,
- "correctness_score": 0,
- "sentiment_score": 0,
- "revision_score": 0,
- "first_request_at": "2019-08-24T14:15:22Z",
- "last_request_at": "2019-08-24T14:15:22Z"
}
}
]Returns a list of all inference models tracked by Coolhand, including per-token pricing metadata.
Authentication:
Authentication is optional. Unauthenticated callers are rate-limited (see below).
Authenticated callers bypass the rate limit entirely. Pass a public API key via the
X-API-Key header or as a Bearer token in the Authorization header. Sign up at
coolhand.ai to get a free API key. A private key is not accepted here — this endpoint
only recognizes public keys (and rejects a private key the same way it would an invalid
one).
How pricing data is gathered:
Pricing data is collected by the Godfrey agent, an automated system that
periodically fetches and parses pricing pages published by model providers.
All records undergo human review before being considered authoritative.
The sources field on each record links to the original provider pricing
page used as the data source.
Cost units:
Every _per_token field is a raw USD cost-per-token value. Multiply by 1,000,000 for
the "per 1M tokens" pricing commonly published by providers.
per_cache_storage_cost_per_hour_per_token is the one exception: it's priced per hour
of storage, not per request — an ongoing hourly charge for holding tokens in a cache
(as Gemini's explicit context caching bills), not a one-time per-token cost.
Capabilities:
Each record carries a capabilities array of tags for features the model is known to
support (e.g. function_calling, vision). A tag is only ever a positive claim; null
means the capabilities are not recorded (unknown), not that the model lacks them. Filter with
capability (see below).
Rate limiting:
Requests are rate limited to 1 request per minute per IP address. Exceeding the limit
returns a 429 Too Many Requests response. Requests authenticated with a private API
key are never rate limited. A public API key is accepted but does not lift the limit — it
is designed to be published in web pages, so it can't prove who is calling.
| q[source_api_eq] | string Filter by API identifier (e.g. 'openai', 'anthropic'). For some values (e.g. 'opencode', 'openrouter', 'azure', 'bedrock', 'vertex') this is a hosting gateway that re-lists another vendor's models at its own pricing, distinct from the model's actual vendor — see q[provider_eq]. |
| q[model_eq] | string Filter by the exact model identifier passed in API calls to the source API itself — e.g. if your code calls OpenAI with model: 'gpt-4o', pass q[model_eq]=gpt-4o here, not the display name (see the model field on the response schema below). Combine with q[source_api_eq] to fetch a single model, since (source_api, model) is unique. |
| q[model_type_eq] | string Filter by model type: 'llm' for text-generating models, 'system_one' for decision models (e.g. TypeSafe's Jev) that return typed answers instead of generated text. |
| q[size_tier_eq] | string Filter LLMs by size tier: 'micro' (on-device), 'small', 'medium', 'large' or 'extra_large' (frontier flagship). Unclassified and System One models are excluded. |
| q[provider_eq] | string Filter by the model's underlying vendor, regardless of which source_api/gateway serves it. Values are lowercase (e.g. 'openai', 'anthropic', 'google') and matched case-sensitively. |
| q[context_window_gteq] | integer Only models whose context window is at least this many tokens. Models whose context window is unknown (null) are excluded. Other integer predicates (_lteq, _gt, _lt, _eq) work the same way; text predicates such as _cont return 400. |
| q[max_output_tokens_gteq] | integer Only models that can generate at least this many tokens in one response. Models whose limit is unknown (null) are excluded. Other integer predicates (_lteq, _gt, _lt, _eq) work the same way; text predicates such as _cont return 400. |
| q[weights_access_eq] | string Filter by whether the model's weights can be downloaded: 'closed', 'open_permissive' (OSI-style license such as Apache-2.0 or MIT) or 'open_restricted' (custom or community license with use limits, e.g. Llama). Models whose status is unknown (null) are excluded. To list every open-weights model pass q[weights_access_in][]=open_permissive&q[weights_access_in][]=open_restricted. |
| q[knowledge_cutoff_gteq] | string Only models whose knowledge cutoff is on or after this date (YYYY-MM-DD). Models with an unknown (null) cutoff are excluded. The _lteq, _lt, _gt, _eq and _null predicates work the same way. |
| q[s] | string Sort expression, e.g. 'model desc'. Defaults to source_api then model. Other ransackable columns (pricing, max_output_tokens, deprecation_notes, sources, not_human_reviewed, slug) can be filtered the same q[column_predicate] way; not enumerated individually here since #index only ever shows the summary fields documented in the response schema below — see GET /api/v2/inference_apis/{id} for the full field set those filter. |
| available_for_bakeoff | boolean If true, only return bakeoff-eligible models. Not a Ransack predicate (not under q[...]) — like include_deprecated below, it changes which set is returned rather than narrowing a search. |
| capability | Array of arrays Only return models tagged with ALL of the given capabilities. Repeat the param (capability[]=function_calling&capability[]=vision) or comma-separate it (capability=function_calling,vision). Models whose capabilities are not recorded (null) never match. An unrecognized tag returns 400. Not a Ransack predicate: q[capabilities_cont] and similar are rejected. Tags: |
| input_modality | string Only return models whose input_modalities contain every value given. Comma-separated (input_modality=image,pdf) or repeated (input_modality[]=image). Allowed values: text, image, audio, video, pdf, file. Models whose modalities are unknown (null) are excluded. An unrecognized value returns 400. Not a Ransack predicate. |
| output_modality | string Only return models whose output_modalities contain every value given. Same syntax as input_modality. Allowed values: text, image, audio, video, embeddings, rerank, decision. |
| include_deprecated | boolean Include deprecated models. Defaults to true. Pass false to hide them. Not a Ransack predicate — it changes which set is returned rather than narrowing it. |
| q[id_eq] | string NOT a supported filter — id was intentionally removed from ransackable_attributes. Documented here only to show the resulting contract: an unrecognized predicate returns 400 rather than silently ignoring the filter and returning the whole catalog. |
| X-API-Key | string API key for authentication |
| Authorization | string Bearer token with API key |
[- {
- "id": 0,
- "source_api": "string",
- "model": "string",
- "display_name": "string",
- "provider": "string",
- "model_type": "llm",
- "size_tier": "micro",
- "context_window": 0,
- "per_input_cost_per_token": 0.1,
- "per_output_cost_per_token": 0.1,
- "per_cached_input_cost_per_token": 0.1,
- "per_cache_creation_input_cost_per_token": 0.1,
- "per_cache_creation_5m_input_cost_per_token": 0.1,
- "per_cache_creation_1h_input_cost_per_token": 0.1,
- "per_cache_storage_cost_per_hour_per_token": 0.1,
- "capabilities": [
- "function_calling"
], - "max_output_tokens": 0,
- "weights_access": "closed",
- "weights_license": "string",
- "available_for_bakeoff": true,
- "input_modalities": [
- "text"
], - "output_modalities": [
- "text"
], - "deprecated_at": "2019-08-24T14:15:22Z",
- "deprecation_notes": "string",
- "knowledge_cutoff": "2019-08-24",
- "sources": "string",
- "created_at": "2019-08-24T14:15:22Z",
- "updated_at": "2019-08-24T14:15:22Z"
}
]Returns every field #index's summary response has, plus the fields #index omits: the
internal not_human_reviewed review-status flag, the slug used in this model's public
page URL, the batch and reasoning-output per-token pricing columns, edge_model /
downstep_model, pricing_tiers, and history (index has never exposed any of these).
context_window (maximum context window in tokens; null when unknown) and max_output_tokens
(the most tokens the model can generate in one response; reasoning tokens count toward it; null
when unknown) are on both, as are weights_access (closed, open_permissive or
open_restricted; null when unknown) and weights_license.
Same authentication and rate limiting as #index.
history: past pricing/status changes, most recent first, reconstructed from this
record's audit trail. Each entry's whodunnit is normalized to "automated" (an
automated pricing scan or cross-check) or "manual" (a human edit) — never an internal
user/admin id. Each changed attribute in changes includes raw before/after values
plus a human-readable description.
last_change_detected_at: the timestamp of the most recently detected pricing/status
change, not a literal "last time we checked" timestamp — a scan that finds no change
leaves no trace, so this can under-report true scan frequency for a model whose data
hasn't moved recently.
edge_model / downstep_model: curated comparison references — a newer "same tier"
model, and a cheaper alternative, respectively. Each is a minimal {slug, display_name}
object, or null if none is set or the related model has no slug.
pricing_tiers: pricing overrides that replace the base rates for a request once a
condition is met: input_token_threshold (a flat higher rate for the whole request past a
large input-token threshold, condition_trigger) or utc_time_window (a different rate
during recurring UTC hours, e.g. peak-hour pricing, given by active_utc_weekdays and
active_utc_hours). An empty array if none are configured for this model.
| id required | string The model's slug (e.g. 'openai-gpt-4o'), matching its public page URL at /inference-apis/:slug — not the internal numeric id, which isn't a stable public identifier and isn't filterable via q[...] on #index either. |
| X-API-Key | string API key for authentication |
| Authorization | string Bearer token with API key |
{- "id": 0,
- "slug": "string",
- "source_api": "string",
- "model": "string",
- "display_name": "string",
- "provider": "string",
- "model_type": "llm",
- "size_tier": "micro",
- "context_window": 0,
- "per_input_cost_per_token": 0.1,
- "per_output_cost_per_token": 0.1,
- "per_cached_input_cost_per_token": 0.1,
- "per_cache_creation_input_cost_per_token": 0.1,
- "per_cache_creation_5m_input_cost_per_token": 0.1,
- "per_cache_creation_1h_input_cost_per_token": 0.1,
- "per_cache_storage_cost_per_hour_per_token": 0.1,
- "capabilities": [
- "function_calling"
], - "max_output_tokens": 0,
- "weights_access": "closed",
- "weights_license": "string",
- "available_for_bakeoff": true,
- "input_modalities": [
- "string"
], - "output_modalities": [
- "string"
], - "deprecated_at": "2019-08-24T14:15:22Z",
- "deprecation_notes": "string",
- "knowledge_cutoff": "2019-08-24",
- "sources": "string",
- "updated_at": "2019-08-24T14:15:22Z",
- "created_at": "2019-08-24T14:15:22Z",
- "not_human_reviewed": true,
- "per_input_cost_batch_per_token": 0.1,
- "per_output_cost_batch_per_token": 0.1,
- "per_reasoning_output_cost_per_token": 0.1,
- "edge_model": {
- "slug": "string",
- "display_name": "string"
}, - "downstep_model": {
- "slug": "string",
- "display_name": "string"
}, - "pricing_tiers": [
- {
- "condition_type": "input_token_threshold",
- "condition_trigger": 0,
- "active_utc_weekdays": [
- 0
], - "active_utc_hours": [
- 0
], - "condition_description": "string",
- "per_input_cost_per_token": 0.1,
- "per_output_cost_per_token": 0.1,
- "per_cached_input_cost_per_token": 0.1,
- "per_cache_creation_input_cost_per_token": 0.1,
- "per_cache_creation_5m_input_cost_per_token": 0.1,
- "per_cache_creation_1h_input_cost_per_token": 0.1,
- "per_reasoning_output_cost_per_token": 0.1,
- "notes": "string"
}
], - "last_change_detected_at": "2019-08-24T14:15:22Z",
- "history": [
- {
- "changed_at": "2019-08-24T14:15:22Z",
- "whodunnit": "automated",
- "initial": true,
- "changes": {
- "property1": {
- "before": null,
- "after": null,
- "description": "string"
}, - "property2": {
- "before": null,
- "after": null,
- "description": "string"
}
}
}
]
}Attaches an existing LlmRequestLogFeedback to an optimization as supporting evidence. Requires a private credential (the client's own private API key, a team member's UserClient private key, or an AdminClientCredential) or a Doorkeeper Bearer token - the public api_key is rejected on this endpoint. Breaking change: the response id field is now the evidence link's hashid (a string), not the internal integer primary key - pass it back as-is to the corresponding DELETE .../{id} endpoint; consumers that previously parsed or stored it as an integer need to treat it as an opaque string.
| optimization_id required | string Optimization hashid |
| X-API-Key | string Any of this client's private credentials (see description) - not the public api_key |
| Authorization | string Bearer token with the private API key, or a Doorkeeper access token |
| feedback_id | string Feedback hashid to link |
| feedback_ids | Array of strings Bulk mode, used instead of |
| note | string or null Optional note explaining the link |
{- "feedback_id": "string",
- "feedback_ids": [
- "string"
], - "note": "string"
}{- "linked": 0,
- "already_linked": 0,
- "errored": 0,
- "not_found": [
- "string"
]
}Removes a feedback-evidence link from an optimization. Requires a private credential (see the create operation's description) - the public api_key is rejected.
| optimization_id required | string Optimization hashid |
| id required | string The link's hashid, as returned in the create response's |
| X-API-Key | string Any of this client's private credentials (see description) - not the public api_key |
| Authorization | string Bearer token with the private API key, or a Doorkeeper access token |
{- "error": "string"
}Attaches an existing LlmRequestLog to an optimization as supporting evidence. Requires a private credential (see the feedback-links create operation's description) - the public api_key is rejected on this endpoint.
| optimization_id required | string Optimization hashid |
| X-API-Key | string Any of this client's private credentials (see description) - not the public api_key |
| Authorization | string Bearer token with the private API key, or a Doorkeeper access token |
| log_id required | string LLM request log hashid to link |
| note | string or null Optional note explaining the link |
{- "log_id": "string",
- "note": "string"
}{- "id": "string",
- "optimization_id": "string",
- "note": "string",
- "created_at": "2019-08-24T14:15:22Z",
- "log_id": "string"
}Removes a log-evidence link from an optimization. Requires a private credential (see the create operation's description) - the public api_key is rejected.
| optimization_id required | string Optimization hashid |
| id required | string The link's hashid, as returned in the create response's |
| X-API-Key | string Any of this client's private credentials (see description) - not the public api_key |
| Authorization | string Bearer token with the private API key, or a Doorkeeper access token |
{- "error": "string"
}Attaches an existing LlmRequestEval to an optimization as supporting evidence. Requires a private credential (see the feedback-links create operation's description) - the public api_key is rejected on this endpoint.
| optimization_id required | string Optimization hashid |
| X-API-Key | string Any of this client's private credentials (see description) - not the public api_key |
| Authorization | string Bearer token with the private API key, or a Doorkeeper access token |
| eval_id required | string LLM request eval hashid to link |
| note | string or null Optional note explaining the link |
{- "eval_id": "string",
- "note": "string"
}{- "id": "string",
- "optimization_id": "string",
- "note": "string",
- "created_at": "2019-08-24T14:15:22Z",
- "eval_id": "string"
}Removes an eval-evidence link from an optimization. Requires a private credential (see the create operation's description) - the public api_key is rejected.
| optimization_id required | string Optimization hashid |
| id required | string The link's hashid, as returned in the create response's |
| X-API-Key | string Any of this client's private credentials (see description) - not the public api_key |
| Authorization | string Bearer token with the private API key, or a Doorkeeper access token |
{- "error": "string"
}Uploads a document as a ClientFile, e.g. so a tool like coolhand-cli can attach project files to a client. id is the file's hashid. metadata is an optional free-form object for client-supplied context - see the project_path convention in the Understanding an LLM Request Log guide's Metadata section, which applies here too. Uploads always land as draft: status is not settable through this endpoint, since nothing reviews an uploaded file's content - promote a file to published from the admin UI once it's been reviewed. Requires the private API key - the public key cannot upload files. Files are currently proxied through the API and capped at 20MB; larger uploads are not yet supported. Accepted formats: pdf, md/markdown, txt, docx/doc, xlsx/xls, pptx/ppt, odt/ods/odp and html/htm. The type is detected from the file's contents and must match its extension; anything else is rejected with a 422 on file.
| X-API-Key | string Private API key for authentication |
| Authorization | string Bearer token with the private API key |
Display name for the file
{- "id": "string",
- "name": "string",
- "file_type": "string",
- "status": "string",
- "description": "string",
- "metadata": { },
- "created_at": "2019-08-24T14:15:22Z"
}Displays the browser-based token authorization page for the Coolhand CLI.
Flow overview:
GET /cli/auth?redirect_uri=http://localhost:<port>/cb&state=<random>&scope=<scope>./users/sign_in,
which stores the CLI auth URL as the post-login return URL.redirect_uri with token(s) in the query string
(see POST /cli/auth).scope parameter:
public only (backward compatible with
callers written before scope was configurable).public — returns only the public API key (token). Use for log
ingest and REST API calls.private — returns only the private MCP key (private_token). The UI
displays an owner-consent confirmation checkbox; the CLI Connect button
is disabled until it's checked, and the form is rejected with 422 if it
is submitted without it. Only request private scope when the CLI needs
to call the /mcp endpoint.Regardless of which scope is requested, the page always renders both the
public and private key checkboxes and pre-checks only the requested
one(s), so the user can grant more than was asked for without re-running
the CLI with a different --scope.
public,private (or repeated/space-separated) — returns both keys.
Requesting an explicit scope with zero valid values (e.g. scope=)
is rejected with 400 Bad Request — a CLI must ask for at least one
of public or private.Security: redirect_uri must be an http:// URL with host
localhost, 127.0.0.1, or ::1. HTTPS and non-localhost hosts are
rejected with 400 Bad Request.
| redirect_uri required | string Localhost HTTP callback URL for the CLI (e.g. |
| state required | string Opaque CSRF state string echoed back in the callback. Max 256 characters. |
| scope | string Key scope(s) to request: |
Processes the CLI auth form submission and redirects the browser to the CLI's local callback URL with API key(s) in the query string.
Callback query parameters:
| Parameter | Always present | Description |
|---|---|---|
token |
Only when scope[] includes public |
Public API key (ch_pub_*). Use for log ingest and REST API calls. |
private_token |
Only when scope[] includes private |
Private MCP key (ch_priv_*). Required for POST /mcp. |
state |
✓ | Echoed back from the request — verify this matches your initial state to prevent CSRF. |
client_name |
✓ | Human-readable name of the selected Coolhand account. |
client_id |
✓ | Hashid of the selected Coolhand account. |
At least one of token / private_token is always present — submitting an
empty scope[] is rejected with 422 Unprocessable Entity.
Using the private key:
Pass private_token as the X-API-Key header when calling POST /mcp.
The public token is used everywhere else (SDK configuration, ingest endpoint).
Never embed the private key in client-side code or commit it to source control.
New client creation:
Pass new_client=1 and new_client_name=<name> instead of client_id to
create a new Coolhand account on the fly. The callback will carry the new
account's freshly generated keys.
Localhost HTTP callback URL (must match the GET request value)
Handles the "I didn't request this" escape hatch on the CLI auth screen.
Never redirects to the CLI's redirect_uri — instead sends the user to
their own dashboard and posts a Slack alert so the team can investigate a
possible spoofed or malicious CLI auth link. No token is issued.
The request URI included in that alert is rebuilt server-side from the
validated redirect_uri / state / scope params, so there is no
caller-supplied field for it — a tampered form cannot inject text into
the alert.
Localhost HTTP callback URL (must match the GET request value)
| id | integer LLM Request Log ID |
| collector | string or null Optional name/stamp identifying the collection method (e.g., 'nodejs-sdk-v1.2.0', 'ruby-gem-v2.1.3') |
| metadata | object Optional free-form client-supplied context (e.g. { "project_path": "..." }) |
| created_at | string <date-time> (shared_created_at) Date and time when object was created formatted according to RFC 3339. Timezone is UTC |
| updated_at | string <date-time> (shared_updated_at) Date and time when object was updated formatted according to RFC 3339. Timezone is UTC |
| ingest_evidence | object Facts Coolhand extracted while processing the log, grouped by topic (max_output_tokens, context_window, modalities, capabilities). A group appears only when the payload carried that signal. Always {} in the create response, since processing happens afterwards. |
{- "id": 78933,
- "collector": "nodejs-sdk-v1.2.0",
- "metadata": {
- "project_path": "/Users/me/my-project"
}, - "created_at": "2026-10-07T18:06:07+00:00",
- "updated_at": "2026-10-07T18:06:07+00:00",
- "ingest_evidence": {
- "max_output_tokens": {
- "truncated": true
}
}
}| id | string Unique hashid identifier for the feedback entry. Use this to reference the feedback in update requests. |
| client_id | string Hashid of the client that owns this feedback entry. |
| llm_request_log_id | string or null 🎯 Exact Match - Hashid of the LLM request log this feedback is linked to. Provides exact matching to connect feedback to a specific logged request. |
| workload_id | string or null 📦 Workload Association - Hashid of the workload this feedback is associated with. Set automatically when a valid workload_hashid is provided in the request. |
| like | boolean or null 👍 Low Signal - Boolean like/dislike rating (deprecated, use sentiment instead). Computed from sentiment for backwards compatibility: like=true, dislike=false, neutral/nil=nil. |
| sentiment | string or null Enum: "like" "dislike" "neutral" 🎭 Sentiment Rating - String sentiment value: 'like', 'dislike', or 'neutral'. Preferred over boolean 'like' field. Takes precedence if both are provided. |
| creator_type | string or null Enum: "human" "agent" "unknown" 🧑🤝🤖 Creator Type - What kind of creator submitted this feedback: 'human' (a person), 'agent' (an AI agent or automated tool), or 'unknown'. Defaults to 'unknown' when omitted. |
| explanation | string or null 💬 Medium Signal - End user explanation of why the response was good or bad. Valuable qualitative data for understanding user preferences and improving model performance. |
| revised_output | string or null ⭐ Best Signal - End user revision of the LLM response. The highest value data for improving quality scores. This is the user's improved version of what the AI should have said. |
| llm_provider_unique_id | string or null 🎯 Exact Match - The x-request-id from the LLM API response (e.g., 'req_xxxxxxx'). Provides exact matching to connect feedback to the specific LLM provider request. |
| original_output | string or null 🔍 Fuzzy Match - The original LLM response text. Provides fuzzy matching but isn't 100% reliable. Use when you don't have llm_provider_unique_id or llm_request_log_id. |
| client_unique_id | string or null 🔗 Your Internal Matcher - Connect to an identifier from your system for internal matching. This helps you correlate feedback with your own request tracking system. |
| creator_unique_id | string or null 👤 User Tracking - Unique ID to match feedback to the end user who created it. Useful for analyzing feedback patterns by user and preventing duplicate feedback. |
| collector | string or null 🏷️ Metadata - Optional name/stamp identifying the collection method or SDK version. Helps track which system or version collected the feedback. |
| coolhand_fingerprint_id | string or null 🔒 Optional - Unique fingerprint ID set by the CoolhandJS SDK (https://github.com/Coolhand-Labs/coolhand-js). DO NOT set this field when calling the API directly - it is reserved for CoolhandJS. |
| created_at | string <date-time> (shared_created_at) Date and time when object was created formatted according to RFC 3339. Timezone is UTC |
| updated_at | string <date-time> (shared_updated_at) Date and time when object was updated formatted according to RFC 3339. Timezone is UTC |
{- "id": "abc123xyz789",
- "client_id": "xyz789abc123",
- "llm_request_log_id": "abc123xyz789",
- "workload_id": "xyz789abc123",
- "like": true,
- "sentiment": "like",
- "creator_type": "human",
- "explanation": "Great response! Very helpful and accurate.",
- "revised_output": "This is a dummy revised output",
- "llm_provider_unique_id": "req_1234567890abcdef",
- "original_output": "Dear John Doe,\n\nThank you for reaching out to us regarding the issue with your recent order #12345...",
- "client_unique_id": "email-service-1760025972258",
- "creator_unique_id": "user-789",
- "collector": "coolhand-node-0.1.0",
- "coolhand_fingerprint_id": "fp_abc123xyz789",
- "created_at": "2026-10-07T18:06:07+00:00",
- "updated_at": "2026-10-07T18:06:07+00:00"
}| id | string Client file hashid |
| name | string Display name for the file |
| file_type | string One of slide_deck, report, document |
| status | string One of draft, published, archived |
| description | string or null Optional free-text description |
object Optional free-form client-supplied context (e.g. { "project_path": "..." }) | |
| created_at | string <date-time> (shared_created_at) Date and time when object was created formatted according to RFC 3339. Timezone is UTC |
{- "id": "abc123",
- "name": "Q1 Report",
- "file_type": "document",
- "status": "draft",
- "description": "string",
- "metadata": {
- "project_path": "/Users/me/my-project"
}, - "created_at": "2026-10-07T18:06:07+00:00"
}