Coolhand LLM Request Logging API (v2)

Download OpenAPI specification:

LLM Request Logging API Documentation.

Introduction

This API allows clients to log and store request/response data from Large Language Model (LLM) API calls for monitoring, analysis, and debugging purposes. The API accepts any JSON structure in the raw_request field, making it flexible for different LLM providers and use cases.

Authentication

API uses API Key authentication. Clients are issued unique API keys that must be included in requests using one of these methods:

  • X-API-Key header: X-API-Key: your_api_key_here
  • Authorization Bearer header: Authorization: Bearer your_api_key_here

Contact your administrator to obtain an API key.

Public vs. private keys

Every client is issued two independent keys:

  • Public key (ch_pub_...) — safe to embed in SDK setup snippets, agent-facing code, or anywhere else it might be exposed to an end user. It only permits ingest-style writes (e.g. logging a new request, submitting feedback) — it cannot read back any data.
  • Private key (ch_priv_...) — required for read endpoints and other sensitive operations (e.g. optimization evidence links). Never embed a private key in client-side or agent-facing code; treat it like a server-side secret.

Each endpoint below documents which key type(s) it accepts. Endpoints that require a private key are marked accordingly.

publicApiKey

A client's public key (ch_pub_...). Safe to embed in SDK/agent-facing code. Only permits ingest-style writes — cannot read data back.

Security Scheme Type: API Key
Header parameter name: X-API-Key

privateApiKey

A client's private key (ch_priv_...). Required for read endpoints and other sensitive operations. Never embed in client-side or agent-facing code.

Security Scheme Type: API Key
Header parameter name: X-API-Key

bearerAuth

Either key type (public or private, per the operation's requirements) or a Doorkeeper OAuth access token, sent as Authorization: Bearer <token>.

Security Scheme Type: HTTP
HTTP Authorization Scheme: bearer

Common Use Cases

OpenAI API Logging

Log requests and responses from OpenAI's chat completions, embeddings, and other endpoints.

Anthropic Claude Logging

Store conversation data and model responses from Claude API calls.

Custom LLM Provider Logging

The flexible JSON structure supports any LLM provider's request/response format.

Analytics and Monitoring

Aggregate usage statistics, token consumption, and performance metrics across LLM calls.

API Error Handling

Error Response Structure

When interacting with the LLM Request Logging API, you'll encounter standard error response formats for different types of failures.

object

Contains error details.

{
  • "errors": {
    }
}

HTTP Status Codes

Understanding the HTTP status codes for this API:

HTTP Status Code Description
201 Created LLM request log successfully created
400 Bad Request Malformed request or invalid JSON structure
401 Unauthorized Missing, invalid, or expired API key
422 Unprocessable Entity Validation error - required fields missing or invalid format
5xx Server Errors Server-side issues - please report these for investigation

LLM Request Logs

API endpoints for logging and storing LLM (Large Language Model) request/response data

Create LLM Request Log

Creates a new LLM request log entry for monitoring and analysis purposes. See the Understanding an LLM Request Log guide for a full description of the fields Coolhand captures, which fields each provider exposes, and common gotchas. id is the log's hashid (a string), not the internal integer primary key — every log identifier returned by this API is a hashid, matching the GET endpoints below; consumers that previously parsed id as an integer need to treat it as an opaque string instead. A request made with the public API key is capped at 10MB for the whole request body (413 above that) and 255 characters for collector (422 above that); private keys are not capped.

Authorizations:
publicApiKeyprivateApiKeybearerAuth
header Parameters
X-API-Key
string

API key for authentication

Authorization
string

Bearer token with API key

Request Body schema: application/json
required
required
object

Responses

Request samples

Content type
application/json
{
  • "llm_request_log": {
    }
}

Response samples

Content type
application/json
{
  • "id": "string",
  • "collector": "string",
  • "source_api": "string",
  • "source_application": "string",
  • "metadata": { },
  • "source_api_result": "string",
  • "model": "string",
  • "template_id": "string",
  • "template_name": "string",
  • "input_tokens": 0,
  • "output_tokens": 0,
  • "latency_ms": 0,
  • "created_at": "2019-08-24T14:15:22Z",
  • "updated_at": "2019-08-24T14:15:22Z",
  • "ingest_evidence": { },
  • "warnings": [
    ]
}

List LLM Request Logs Private key required

Returns this client's LLM request logs as a bare JSON array (unlike GET /api/v2/llm_request_log_feedbacks's { feedback:, pagination: } envelope - coolhand-cli's fetchLastSync relies on this response being Array.isArray, so the body shape is intentionally not wrapped). Pagination totals are exposed via X-Page/X-Per-Page response headers, plus X-Total-Count/X-Total-Pages when include_total=true is passed (omitted by default to skip the COUNT(*) query - analyze-claude-sessions's frequent per=1 dedup-cutoff poll never needs a total). Supports Ransack search via q[...] params (e.g. q[source_api_in][]=claude_code), sorting via q[s] (e.g. created_at desc for newest first; when no q[s] is supplied, the endpoint applies a stable default sort of id desc for newest first so pagination is deterministic - pass q[s] explicitly to override), and pagination via page/per (per_page also accepted; default 25, max 100). Requires the private API key - the public key is write-only and cannot read. id is always the log's hashid (not the internal integer PK) - the sort itself still orders by the real primary key internally, but a cutoff/cursor derived from results should key off created_at, not id, since hashids don't sort lexically in creation order. analyze-claude-sessions uses this to derive its dedup cutoff by requesting the newest claude_code/claude_cowork record (per=1), instead of a bespoke endpoint. Also supports named filters (applied on top of any q[...] filtering, not in place of it): template_id/workload_id (hashid), system_prompt_contains/user_prompt_contains (case-insensitive substring), model, source_api, source_api_result, project_path (exact match against metadata.project_path), unmatched_only, days_back (unset by default - unlike search_logs's MCP tool, index has always returned unrestricted results and does not implicitly apply a 30-day window), and include_prompts (adds system_prompt/user_prompt, truncated to 500 chars, to each result). Every result carries cost (USD, priced by the same SQL as the dashboard's cost figures, so tiered pricing, cached-token discounts and reasoning tokens are applied; null when the log has no tokens or its model has no pricing). since/until (ISO8601; since inclusive, until exclusive; a missing offset means UTC and a + offset must be URL-encoded) bound created_at and replace days_back when either is given. min_cost (USD) keeps only logs whose cost is at least that, and order=cost_desc sorts by cost, highest first (replacing any q[s] sort); both are limited to logs that can be priced (those with a non-null cost), failed requests included, unlike the metrics' cost totals. On a client with a lot of history, combine min_cost/order=cost_desc with since or days_back: the cost is computed per log, so an unbounded query can time out (504). A malformed value for any of them returns 422.

Authorizations:
privateApiKeybearerAuth
query Parameters
q[source_api_in][]
Array of strings

Filter to these source_api values (Ransack in-predicate)

q[s]
string

Sort expression, e.g. 'created_at desc'

page
integer

Page number

per
integer

Records per page (default 25, max 100; per_page also accepted)

per_page
integer

Alias for per, matching search_logs's MCP tool param name (same 25/100 bounds)

template_id
string

Filter by template hashid

workload_id
string

Filter by workload hashid (matches all templates in that workload)

system_prompt_contains
string

Case-insensitive substring to match in the system prompt

user_prompt_contains
string

Case-insensitive substring to match in the user prompt

model
string

Filter by model name

source_api
string

Filter by source API (e.g. 'openai', 'anthropic', 'vertex')

source_api_result
string

Filter by result status: success, failed, operational, unmatched

source_application
string

Filter by exact source_application

project_path
string

Filter by exact match against metadata.project_path

unmatched_only
boolean

Only return logs with no assigned template

days_back
integer

Limit to logs created in the last N days (max 36500; no default - unrestricted unless given; ignored when since is given; with only until, the window is days_back long ending at until)

since
string <date-time>

ISO8601 lower bound (inclusive) on created_at. A date alone or a timestamp without an offset is UTC; encode a + offset as %2B

until
string <date-time>

ISO8601 upper bound (exclusive) on created_at; must be after since. With only until, days_back (if given) sets how far back the window starts

min_cost
number

Only logs whose per-log cost (USD) is at least this (0 to 1000000); unpriceable logs are excluded, priceable failed ones are not

order
string
Value: "cost_desc"

cost_desc sorts by per-log cost, highest first (priceable logs only), replacing any q[s] sort

include_prompts
boolean

Include truncated system_prompt/user_prompt in each result

include_total
boolean

Include X-Total-Count/X-Total-Pages response headers (default: false, to skip the COUNT(*) query on this frequently-polled endpoint)

header Parameters
X-API-Key
string

Private API key for authentication

Authorization
string

Bearer token with the private API key

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Get LLM Request Log Content Private key required

Fetches full input/output content for a single log by its ID (hashid). Accepts any of this client's private credentials: the client's own private API key, a team member's UserClient private key, an AdminClientCredential, or the scoped, rotating key of the client's agent user (see Clients::AgentUserProvisioner) used by the change-suggestion coding agent to ground a fix in the log it was told to look at. Supports section/max_chars for large logs and search_query for snippet search. Only directly-collected logs are fetchable here (matching index's .client_logs restriction) - internally generated records (evals, synthetic logs) 404 even when the hashid is known.

Authorizations:
privateApiKeybearerAuth
path Parameters
id
required
string

Log hashid

query Parameters
section
string

full, beginning, or end (default: full)

max_chars
integer

Maximum characters to return per content field

search_query
string

Text to search for within the log content

include_thinking
boolean

Include thinking/reasoning response content (default: false)

header Parameters
X-API-Key
string

Any of this client's private credentials (see description)

Responses

Response samples

Content type
application/json
{
  • "id": "string",
  • "url": "string",
  • "collector": "string",
  • "metadata": { },
  • "model": "string",
  • "source_api": "string",
  • "source_application": "string",
  • "source_api_result": "string",
  • "template_id": "string",
  • "template_name": "string",
  • "input_tokens": 0,
  • "output_tokens": 0,
  • "latency_ms": 0,
  • "cost": 0,
  • "cost_breakdown": {
    },
  • "created_at": "2019-08-24T14:15:22Z",
  • "updated_at": "2019-08-24T14:15:22Z",
  • "ingest_evidence": { },
  • "system_prompt": "string",
  • "user_prompt": "string",
  • "output": "string",
  • "truncated": true,
  • "total_chars": {
    },
  • "search_query": "string",
  • "matches": {
    },
  • "thinking_response": [
    ]
}

LLM Request Log Feedbacks

API endpoints for collecting user feedback on LLM request/response quality and accuracy

Create LLM Request Log Feedback

Creates a new feedback entry for LLM request monitoring and improvement purposes.

Field Guide - All fields are optional, but here's how to get the best results:

Matching Fields

  • llm_request_log_id 🎯 Exact Match - Hashid from the Coolhand API response when the original LLM request was logged. Provides exact matching.
  • llm_provider_unique_id 🎯 Exact Match - The x-request-id from the LLM API response (e.g., "req_xxxxxxx")
  • original_output 🔍 Fuzzy Match - The original LLM response text. Provides fuzzy matching but isn't 100% reliable.
  • client_unique_id 🔗 Your Internal Matcher - Connect to an identifier from your system for internal matching
  • workload_hashid 📦 Workload Association - Optional. Associate feedback with a specific workload to increase fuzzy match success rate

Quality Data

  • revised_output ⭐ Best Signal - End user revision of the LLM response. The highest value data for improving quality scores.
  • explanation 💬 Medium Signal - End user explanation of why the response was good or bad. Valuable qualitative data.
  • like 👍 Low Signal - Boolean like/dislike. Lower quality signal but easy for users to provide.
  • creator_unique_id 👤 User Tracking - Unique ID to match feedback to the end user who created it
  • collector 🏷️ Metadata - Optional name/stamp identifying the collection method (e.g., "coolhand-node-0.1.0")

Reusing an existing placeholder: if original_output + coolhand_fingerprint_id match an existing unrated feedback record, this request updates that record instead of creating a new one. For a public-key caller, only like/sentiment/explanation/revised_output/ llm_request_log_id are applied to it — the rest of the request is ignored (and reported in warnings), the same restriction PATCH /api/v2/llm_request_log_feedbacks/{id} applies to a public key. llm_request_log_id stays writable here even though PATCH treats it as immutable for every caller: it's already scoped to this client, so linking it in doesn't cross the same boundary the other restricted fields do. Private key / OAuth Bearer token callers can update every field on the existing record. This restriction only applies when reusing an existing placeholder — a genuinely new record still accepts the full field set from a public key.

Size limit: a request made with the public key is capped at 10MB (413 above that); private keys are not capped.

Response shape: a public-key caller receives only the record's id back (plus warnings, and created_partial_id when a partial was created) — reading a record back is a private-credential operation. Private key / OAuth Bearer callers get the full payload documented below.

Authorizations:
publicApiKeyprivateApiKeybearerAuth
query Parameters
api_key
string

API key as query parameter

collector
string

🏷️ Optional collector identifier (can also be provided in request body)

header Parameters
X-API-Key
string

API key for authentication

Authorization
string

Bearer token with API key

Request Body schema: application/json
required
required
object

Responses

Request samples

Content type
application/json
{
  • "llm_request_log_feedback": {
    }
}

Response samples

Content type
application/json
{
  • "id": "string",
  • "client_id": "string",
  • "llm_request_log_id": "string",
  • "workload_id": "string",
  • "like": true,
  • "sentiment": "string",
  • "creator_type": "string",
  • "explanation": "string",
  • "revised_output": "string",
  • "llm_provider_unique_id": "string",
  • "original_output": "string",
  • "client_unique_id": "string",
  • "creator_unique_id": "string",
  • "collector": "string",
  • "coolhand_fingerprint_id": "string",
  • "created_at": "2019-08-24T14:15:22Z",
  • "updated_at": "2019-08-24T14:15:22Z",
  • "warnings": [
    ],
  • "created_partial_id": "string",
  • "feedback_partials": [
    ]
}

Search LLM Request Log Feedbacks Private key required

Returns this client's feedback entries. Supports Ransack search via q[...] params (e.g. q[sentiment_eq]=2 - sentiment is stored as the integer codes 0=dislike/1=neutral/2=like, which is what responses render as the strings 'dislike'/'neutral'/'like' - or q[created_at_gteq]=...), sorting via q[s] (e.g. created_at desc; when no q[s] is supplied, the endpoint applies a stable default sort of id desc so pagination is deterministic), and pagination via page/per (default 25, max 100). Filtering and sorting on original_output/revised_output are not supported. Accepts any of this client's private credentials - the client's own private API key, a team member's UserClient private key, an AdminClientCredential, or the scoped, rotating key of the client's agent user (see Clients::AgentUserProvisioner) used by the change-suggestion coding agent to ground a fix in the feedback it was told to look at - but not the public key, which is write-only and cannot read. List items omit original_output/revised_output (which can each hold up to 1GB) - fetch a specific item via GET /api/v2/llm_request_log_feedbacks/{id} to get the full text and any feedback_partials. Also sets X-Total-Count/X-Page/X-Per-Page/X-Total-Pages response headers - the same pagination signal every paginated v2 endpoint exposes (see GET /api/v2/llm_request_logs); the body's pagination object is kept for backward compatibility, not the pattern to rely on for new integrations.

Authorizations:
privateApiKeybearerAuth
query Parameters
q[sentiment_eq]
integer

Filter by sentiment integer code: 0 = dislike, 1 = neutral, 2 = like

q[workload_id_eq]
string

Filter by workload hashid (the same value rendered as each item's workload_id)

q[s]
string

Sort expression, e.g. 'created_at desc'

page
integer

Page number

per
integer

Records per page (default 25, max 100)

header Parameters
X-API-Key
string

Any of this client's private credentials (see description)

Authorization
string

Bearer token with the private API key

Responses

Response samples

Content type
application/json
{
  • "feedback": [
    ],
  • "pagination": {
    }
}

Update LLM Request Log Feedback

Updates an existing feedback entry. Only specific fields can be updated - identity fields are immutable.

Partial Feedback Updates: When partial_id is provided in the request body, the update is applied to that specific FeedbackPartial instead of the parent feedback. This is useful when adding explanations to partial feedback (highlighted text sections). The response will include updated_partial_id to confirm which partial was updated.

Updatable Fields (Parent Feedback) - public key:

  • like - Boolean like/dislike rating
  • sentiment - Sentiment value (like/dislike/neutral)
  • explanation - End user explanation
  • revised_output - End user revision of the LLM response

Updatable Fields (Parent Feedback) - private key / OAuth Bearer token: Everything above, plus:

  • creator_type - Who supplied the feedback (human/agent/unknown)
  • original_output - The original LLM response text
  • llm_provider_unique_id - The x-request-id from the LLM API response
  • collector - Collection method identifier
  • client_unique_id - Your internal request identifier
  • coolhand_fingerprint_id - Fingerprint id used for dedup/matching
  • workload_hashid - Associate with a workload (or clear by setting to null/empty)

A public-key request that includes a private-key-only field does not error - that field is silently ignored, and only the public-key-allowed fields in the request are applied.

Response shape: a public-key caller receives only the record's id (reading a record back is a private-credential operation - see GET /api/v2/llm_request_log_feedbacks/{id}), plus warnings and, for a partial update, updated_partial_id so it can be confirmed. Private key and OAuth Bearer callers get the full payload documented below. Any field the request sent that the credential may not write is reported back in warnings.

Updatable Fields (FeedbackPartial - when partial_id is provided) - all credential types:

  • explanation - End user explanation for this specific partial
  • sentiment - Sentiment value (like/dislike/neutral)
  • like - Boolean (automatically converted to sentiment)

Partials have no creator_type - it is a parent-feedback field only, and is ignored here for every credential type.

Immutable Fields (cannot be updated):

  • creator_unique_id
  • llm_request_log_id

Note: After a successful update, if the feedback is not already matched to a log and has original_output, the fuzzy matching job will be re-triggered to attempt matching.

Authorizations:
publicApiKeyprivateApiKeybearerAuth
path Parameters
id
required
string

Feedback hashid

header Parameters
X-API-Key
string

API key for authentication

Authorization
string

Bearer token with API key

Request Body schema: application/json
required
object

Responses

Request samples

Content type
application/json
{
  • "llm_request_log_feedback": {
    }
}

Response samples

Content type
application/json
{
  • "id": "string",
  • "client_id": "string",
  • "llm_request_log_id": "string",
  • "workload_id": "string",
  • "like": true,
  • "sentiment": "string",
  • "creator_type": "string",
  • "explanation": "string",
  • "revised_output": "string",
  • "llm_provider_unique_id": "string",
  • "original_output": "string",
  • "client_unique_id": "string",
  • "creator_unique_id": "string",
  • "collector": "string",
  • "coolhand_fingerprint_id": "string",
  • "created_at": "2019-08-24T14:15:22Z",
  • "updated_at": "2019-08-24T14:15:22Z",
  • "updated_partial_id": "string",
  • "feedback_partials": [
    ]
}

Get LLM Request Log Feedback Private key required

Fetches a single feedback entry by hashid, including original_output, revised_output, and any feedback_partials - the full record that GET /api/v2/llm_request_log_feedbacks omits from list results. Accepts any of this client's private credentials - the client's own private API key, a team member's UserClient private key, an AdminClientCredential, or the scoped, rotating key of the client's agent user (see Clients::AgentUserProvisioner) used by the change-suggestion coding agent to ground a fix in the feedback it was told to look at - but not the public key, which is write-only and cannot read.

Authorizations:
privateApiKeybearerAuth
path Parameters
id
required
string

Feedback hashid

header Parameters
X-API-Key
string

Any of this client's private credentials (see description)

Authorization
string

Bearer token with the private API key

Responses

Response samples

Content type
application/json
{
  • "id": "string",
  • "client_id": "string",
  • "llm_request_log_id": "string",
  • "workload_id": "string",
  • "like": true,
  • "sentiment": "string",
  • "creator_type": "string",
  • "explanation": "string",
  • "original_output": "string",
  • "revised_output": "string",
  • "llm_provider_unique_id": "string",
  • "client_unique_id": "string",
  • "creator_unique_id": "string",
  • "collector": "string",
  • "coolhand_fingerprint_id": "string",
  • "created_at": "2019-08-24T14:15:22Z",
  • "updated_at": "2019-08-24T14:15:22Z",
  • "feedback_partials": [
    ]
}

LLM Request Templates

Read-only API endpoints for listing, searching and inspecting the prompt templates an LLM request log is matched against. Template mutation stays on the MCP surface.

List LLM Request Templates Private key required

Returns this client's LLM request templates as a bare JSON array (the same body shape as GET /api/v2/llm_request_logs, not the { feedback:, pagination: } envelope used by GET /api/v2/llm_request_log_feedbacks). Pagination metadata is always exposed via the X-Page, X-Per-Page, X-Total-Count and X-Total-Pages response headers — unlike /llm_request_logs, this endpoint has no include_total opt-out, because a client's template count is small enough that counting the templates themselves is cheap.

log_count is the expensive part of this response, not the pagination total. It aggregates over llm_request_logs, so its cost scales with how many logs the matched templates hold rather than with how many templates come back — the Unmatched bucket reached via include_system=true can hold every log that never matched a template.

The 10-second guard is per statement, not per response — read this before sizing a client timeout. #index issues three statements inside it (the page load, the pagination COUNT, and the grouped log count), and Postgres applies statement_timeout to each one independently. Database time for a single response can therefore reach three times that figure before anything is cancelled, and a 504 here means one query exceeded 10 seconds — never that the response as a whole did. Total response time is not bounded by this endpoint. Do not treat the absence of a 504 as evidence a response was fast, and do not derive a client timeout from the 10-second number: size it from observed end-to-end latency instead. Narrow with workload_id, search, or a smaller per if you do hit a 504.

log_count counts only directly-collected client logs — the same records GET /api/v2/llm_request_logs?template_id=… returns — so the two numbers agree. Internally generated records (evals, bakeoff comparisons, synthetic logs) are excluded, which is why this can be lower than the count the search_templates MCP tool reports.

The client is always derived from the authenticating API key. There is no client_id parameter and one cannot be supplied.

Requires the private API key — the public key is write-only and cannot read data back.

Results are ordered newest first (created_at DESC, with the internal primary key as a tiebreaker so paging is deterministic when two templates share a timestamp).

Prompt patterns are not included by default. Pass include_patterns=true to add user_prompt_pattern / system_prompt_pattern to every row, or fetch a single template via GET /api/v2/llm_request_templates/{id}.

System templates. Every client has two system buckets, Unmatched and Ignored API Calls (LlmRequestTemplate::SYSTEM_TEMPLATE_NAMES), created automatically with the client. They are excluded from this list by default and returned when include_system=true is passed — the Unmatched bucket is what you inspect when logs are misrouting. Each row carries a system_template boolean so callers do not have to match on names.

Deprecated templates are excluded by default and returned when include_deprecated=true is passed. A deprecated row is identifiable by a non-null deprecated_at. (The originating issue called this filter deprecated; it is named include_deprecated here to match both include_system above and the existing search_templates MCP parameter, and because deprecated=true would be ambiguous between "only deprecated" and "also deprecated".)

Archived workloads are not filtered out by default, so the list agrees with GET /api/v2/llm_request_templates/{id}, which can always fetch such a template by id. Pass include_archived=false to hide templates whose workload has been archived (what the search_templates MCP tool does by default).

Metrics. Pass include_metrics=true to add a metrics object to every row: the same cost and performance numbers the dashboard's template table shows, computed by the same SQL (LlmCostSql, so tiered pricing, cached-token discounts and reasoning tokens are applied) in a fixed number of grouped queries for the whole page. Averages cover non-failed directly-collected client logs inside the days_back window (default 28, the dashboard's window, max 365); request_count and error_rate cover the same window; first_request_at/last_request_at are lifetime; sentiment_score and revision_score are all-time, as on the dashboard. avg_cost_per_request is null when no log in the window could be priced. Metrics add one more set of aggregate statements to the response, so expect it to be slower on a large page.

Window. days_back is a rolling lookback ending now. Pass since and/or until (ISO8601; a missing offset means UTC, and a + offset must be URL-encoded as %2B) for an explicit window instead: since is inclusive, until exclusive and defaulting to now, and when since is given days_back is ignored and returned as null. With only until, the window is days_back long ending there. The prior window behind error_rate_change is the same length, immediately before the window. The resolved bounds come back as metrics.since/metrics.until. A malformed timestamp, since not before until, or a window over 365 days returns 422.

Additional counters over the same window, priced with the same cost expression as the dashboard: failure_count (failed logs; error_rate is null when it is 0), total_input_tokens/ total_output_tokens (sums of the input_tokens/output_tokens columns over non-failed logs), priced_request_count (non-failed logs with tokens whose model has pricing, i.e. the logs total_cost and avg_cost_per_request cover; this excludes failed logs and logs with a zero token count, which the per-log cost on GET /api/v2/llm_request_logs does not, so summing that cost can exceed total_cost) and long_context_request_count (priced logs that crossed their model's input-token pricing tier).

Not included in this endpoint: template version history. LlmRequestTemplate versions user_prompt_pattern, system_prompt_pattern, name, status, workload_id and the deprecation fields via PaperTrail, and a versions sub-resource was considered alongside these two endpoints, but it is deliberately deferred to its own change rather than shipped here — it is a distinct resource with its own response shape and its own disclosure surface (historical prompt patterns). Template mutation likewise stays on the MCP surface; this REST surface is read-only.

Authorizations:
privateApiKeybearerAuth
query Parameters
search
string

Case-insensitive substring match against the template name

workload_id
string

Filter to a single workload, by workload hashid. A hashid that does not decode, or that belongs to another client, returns 422 rather than an empty list.

status
string
Enum: "draft" "published" "failure"

Filter by status: draft, published, failure. Any other non-empty value returns 422; an empty value is treated as no filter.

include_deprecated
boolean

Include deprecated templates (default: false)

include_system
boolean

Include the Unmatched / Ignored API Calls system templates (default: false)

include_archived
boolean

Include templates of archived workloads (default: true)

include_patterns
boolean

Include user_prompt_pattern and system_prompt_pattern on every row (default: false)

include_metrics
boolean

Include a metrics object per template — the dashboard's cost and performance numbers (default: false)

days_back
integer

Lookback window in days for metrics (default 28; larger values are clamped to 365, whereas a since window over 365 days is a 422). Must be a positive integer; ignored unless metrics are requested.

since
string <date-time>

ISO8601 start (inclusive) of the metrics window; overrides days_back. A date alone or a timestamp without an offset is UTC; encode a + offset as %2B. Ignored unless metrics are requested.

until
string <date-time>

ISO8601 end (exclusive) of the metrics window (default: now), same formats as since. Ignored unless metrics are requested.

page
integer

Page number

per
integer

Records per page (default 25, max 100; per_page also accepted)

per_page
integer

Alias for per, matching search_templates's MCP param name (same 25/100 bounds)

header Parameters
X-API-Key
string

Private API key for authentication

Authorization
string

Bearer token with the private API key

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Show LLM Request Template Private key required

Returns a single template belonging to the authenticating client, including user_prompt_pattern and system_prompt_pattern — the full untruncated regexes, which the list endpoint omits.

Unlike the list endpoint, this action applies no default filtering beyond client ownership: a deprecated template or a system template (Unmatched, Ignored API Calls) is reachable by id without any opt-in flag, because inspecting one of those is the usual reason to fetch a template directly.

A template belonging to another client returns 404, not 403 — its existence is not disclosed.

As on the list endpoint, log_count is the expensive part: it aggregates over llm_request_logs, and fetching the Unmatched bucket by id counts every log that never matched a template. That count is the one statement bounded by the 10-second statement_timeout, so on this action the guard does correspond to a single query — but it still bounds only that query, not the whole response (the record lookup and authentication run outside it). A 504 means the count exceeded 10 seconds. See the list endpoint for why total response time is not bounded either. log_count counts only directly-collected client logs, matching GET /api/v2/llm_request_logs?template_id=….

Always includes a metrics object — the same cost and performance numbers the dashboard shows for this template (see the list endpoint for the exact definitions). days_back sets its lookback window (default 28, the dashboard's window; max 365); since/until set an explicit window instead, exactly as on the list endpoint.

Requires the private API key.

Authorizations:
privateApiKeybearerAuth
path Parameters
id
required
string

Template hashid (the id returned by the list endpoint)

query Parameters
include_metrics
boolean

Include the metrics object (default: true); false omits it and skips window validation

days_back
integer

Lookback window in days for metrics (default 28, max 365)

since
string <date-time>

ISO8601 start (inclusive) of the metrics window; overrides days_back. A date alone or a timestamp without an offset is UTC; encode a + offset as %2B. Ignored unless metrics are requested.

until
string <date-time>

ISO8601 end (exclusive) of the metrics window (default: now), same formats as since. Ignored unless metrics are requested.

header Parameters
X-API-Key
string

Private API key for authentication

Authorization
string

Bearer token with the private API key

Responses

Response samples

Content type
application/json
{
  • "id": "string",
  • "name": "string",
  • "status": "string",
  • "version": "string",
  • "group": "string",
  • "workload_id": "string",
  • "workload_name": "string",
  • "system_template": true,
  • "deprecated_at": "2019-08-24T14:15:22Z",
  • "log_count": 0,
  • "source_repository": "string",
  • "source_file_path": "string",
  • "created_at": "2019-08-24T14:15:22Z",
  • "updated_at": "2019-08-24T14:15:22Z",
  • "metrics": {
    },
  • "user_prompt_pattern": "string",
  • "system_prompt_pattern": "string"
}

Workloads

Read-only API endpoints for listing the workloads (groups of prompt templates) a client has, with per-workload counts and the same cost and performance rollup the dashboard shows.

List Workloads Private key required

Returns this client's workloads — the groups of prompt templates that make up one task or agent — as a bare JSON array, ordered by name. Pagination metadata is exposed via the X-Page, X-Per-Page, X-Total-Count and X-Total-Pages response headers.

Each row carries the workload's active-template count (template_count), its draft count, log_count and last_activity across every log attached to any of its templates (all generators, so log_count can exceed the sum of the templates' own log_count, which counts only directly-collected client logs).

Archived workloads and the system workloads (Unmatched, Embedding Requests, …) are excluded unless include_archived / include_system is passed.

Metrics. Pass include_metrics=true to add a metrics rollup per workload across all of its templates: the same cost and performance numbers the dashboard shows, computed by the same SQL (LlmCostSql, so tiered pricing, cached-token discounts and reasoning tokens are applied) in a fixed number of grouped queries for the whole page. Averages cover non-failed directly-collected client logs inside the days_back window (default 28, the dashboard's window; max 365). See GET /api/v2/llm_request_templates for the field definitions.

Window. days_back is a rolling lookback ending now. Pass since and/or until (ISO8601; a missing offset means UTC, and a + offset must be URL-encoded as %2B) for an explicit window instead: since is inclusive, until exclusive and defaulting to now, and when since is given days_back is ignored and returned as null. With only until, the window is days_back long ending there. The prior window behind error_rate_change is the same length, immediately before the window. The resolved bounds come back as metrics.since/metrics.until. A malformed timestamp, since not before until, or a window over 365 days returns 422.

Additional counters over the same window, priced with the same cost expression as the dashboard: failure_count (failed logs; error_rate is null when it is 0), total_input_tokens/ total_output_tokens (sums of the input_tokens/output_tokens columns over non-failed logs), priced_request_count (non-failed logs with tokens whose model has pricing, i.e. the logs total_cost and avg_cost_per_request cover; this excludes failed logs and logs with a zero token count, which the per-log cost on GET /api/v2/llm_request_logs does not, so summing that cost can exceed total_cost) and long_context_request_count (priced logs that crossed their model's input-token pricing tier).

Pass include_templates=true to also list each workload's active templates with their routing regex patterns.

The client is always derived from the authenticating API key. Requires the private API key.

Authorizations:
privateApiKeybearerAuth
query Parameters
search
string

Case-insensitive substring match against the workload name

include_archived
boolean

Include archived workloads (default: false)

include_system
boolean

Include system workloads like Unmatched and Embedding Requests (default: false)

include_templates
boolean

Include each workload's active templates and their patterns (default: false)

include_metrics
boolean

Include a metrics rollup per workload (default: false)

days_back
integer

Lookback window in days for metrics (default 28; larger values are clamped to 365, whereas a since window over 365 days is a 422)

since
string <date-time>

ISO8601 start (inclusive) of the metrics window; overrides days_back. A date alone or a timestamp without an offset is UTC; encode a + offset as %2B. Ignored unless metrics are requested.

until
string <date-time>

ISO8601 end (exclusive) of the metrics window (default: now), same formats as since. Ignored unless metrics are requested.

page
integer

Page number

per
integer

Records per page (default 25, max 100; per_page also accepted)

per_page
integer

Alias for per (same 25/100 bounds)

header Parameters
X-API-Key
string

Private API key for authentication

Authorization
string

Bearer token with the private API key

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Inference APIs

List inference model pricing metadata

Returns a list of all inference models tracked by Coolhand, including per-token pricing metadata.

Authentication: Authentication is optional. Unauthenticated callers are rate-limited (see below). Authenticated callers bypass the rate limit entirely. Pass a public API key via the X-API-Key header or as a Bearer token in the Authorization header. Sign up at coolhand.ai to get a free API key. A private key is not accepted here — this endpoint only recognizes public keys (and rejects a private key the same way it would an invalid one).

How pricing data is gathered: Pricing data is collected by the Godfrey agent, an automated system that periodically fetches and parses pricing pages published by model providers. All records undergo human review before being considered authoritative. The sources field on each record links to the original provider pricing page used as the data source.

Cost units: Every _per_token field is a raw USD cost-per-token value. Multiply by 1,000,000 for the "per 1M tokens" pricing commonly published by providers. per_cache_storage_cost_per_hour_per_token is the one exception: it's priced per hour of storage, not per request — an ongoing hourly charge for holding tokens in a cache (as Gemini's explicit context caching bills), not a one-time per-token cost.

Capabilities: Each record carries a capabilities array of tags for features the model is known to support (e.g. function_calling, vision). A tag is only ever a positive claim; null means the capabilities are not recorded (unknown), not that the model lacks them. Filter with capability (see below).

Rate limiting: Requests are rate limited to 1 request per minute per IP address. Exceeding the limit returns a 429 Too Many Requests response. Requests authenticated with a private API key are never rate limited. A public API key is accepted but does not lift the limit — it is designed to be published in web pages, so it can't prove who is calling.

Authorizations:
publicApiKeybearerAuthNone
query Parameters
q[source_api_eq]
string

Filter by API identifier (e.g. 'openai', 'anthropic'). For some values (e.g. 'opencode', 'openrouter', 'azure', 'bedrock', 'vertex') this is a hosting gateway that re-lists another vendor's models at its own pricing, distinct from the model's actual vendor — see q[provider_eq].

q[model_eq]
string

Filter by the exact model identifier passed in API calls to the source API itself — e.g. if your code calls OpenAI with model: 'gpt-4o', pass q[model_eq]=gpt-4o here, not the display name (see the model field on the response schema below). Combine with q[source_api_eq] to fetch a single model, since (source_api, model) is unique.

q[model_type_eq]
string

Filter by model type: 'llm' for text-generating models, 'system_one' for decision models (e.g. TypeSafe's Jev) that return typed answers instead of generated text.

q[size_tier_eq]
string

Filter LLMs by size tier: 'micro' (on-device), 'small', 'medium', 'large' or 'extra_large' (frontier flagship). Unclassified and System One models are excluded.

q[provider_eq]
string

Filter by the model's underlying vendor, regardless of which source_api/gateway serves it. Values are lowercase (e.g. 'openai', 'anthropic', 'google') and matched case-sensitively.

q[context_window_gteq]
integer

Only models whose context window is at least this many tokens. Models whose context window is unknown (null) are excluded. Other integer predicates (_lteq, _gt, _lt, _eq) work the same way; text predicates such as _cont return 400.

q[max_output_tokens_gteq]
integer

Only models that can generate at least this many tokens in one response. Models whose limit is unknown (null) are excluded. Other integer predicates (_lteq, _gt, _lt, _eq) work the same way; text predicates such as _cont return 400.

q[weights_access_eq]
string

Filter by whether the model's weights can be downloaded: 'closed', 'open_permissive' (OSI-style license such as Apache-2.0 or MIT) or 'open_restricted' (custom or community license with use limits, e.g. Llama). Models whose status is unknown (null) are excluded. To list every open-weights model pass q[weights_access_in][]=open_permissive&q[weights_access_in][]=open_restricted.

q[knowledge_cutoff_gteq]
string

Only models whose knowledge cutoff is on or after this date (YYYY-MM-DD). Models with an unknown (null) cutoff are excluded. The _lteq, _lt, _gt, _eq and _null predicates work the same way.

q[s]
string

Sort expression, e.g. 'model desc'. Defaults to source_api then model. Other ransackable columns (pricing, max_output_tokens, deprecation_notes, sources, not_human_reviewed, slug) can be filtered the same q[column_predicate] way; not enumerated individually here since #index only ever shows the summary fields documented in the response schema below — see GET /api/v2/inference_apis/{id} for the full field set those filter.

available_for_bakeoff
boolean

If true, only return bakeoff-eligible models. Not a Ransack predicate (not under q[...]) — like include_deprecated below, it changes which set is returned rather than narrowing a search.

capability
Array of arrays

Only return models tagged with ALL of the given capabilities. Repeat the param (capability[]=function_calling&capability[]=vision) or comma-separate it (capability=function_calling,vision). Models whose capabilities are not recorded (null) never match. An unrecognized tag returns 400. Not a Ransack predicate: q[capabilities_cont] and similar are rejected. Tags: function_calling (Accepts tool/function definitions and can return tool calls); structured_output (Can constrain responses to a JSON schema (Gemini's schema-less JSON mode counts too)); reasoning (Produces reasoning/thinking output before the final answer); vision (Accepts image input); streaming (Can stream responses incrementally); caching (Supports prompt/context caching).

input_modality
string

Only return models whose input_modalities contain every value given. Comma-separated (input_modality=image,pdf) or repeated (input_modality[]=image). Allowed values: text, image, audio, video, pdf, file. Models whose modalities are unknown (null) are excluded. An unrecognized value returns 400. Not a Ransack predicate.

output_modality
string

Only return models whose output_modalities contain every value given. Same syntax as input_modality. Allowed values: text, image, audio, video, embeddings, rerank, decision.

include_deprecated
boolean

Include deprecated models. Defaults to true. Pass false to hide them. Not a Ransack predicate — it changes which set is returned rather than narrowing it.

q[id_eq]
string

NOT a supported filter — id was intentionally removed from ransackable_attributes. Documented here only to show the resulting contract: an unrecognized predicate returns 400 rather than silently ignoring the filter and returning the whole catalog.

header Parameters
X-API-Key
string

API key for authentication

Authorization
string

Bearer token with API key

Responses

Response samples

Content type
application/json
[
  • {
    }
]

Fetch a single inference model's full record

Returns every field #index's summary response has, plus the fields #index omits: the internal not_human_reviewed review-status flag, the slug used in this model's public page URL, the batch and reasoning-output per-token pricing columns, edge_model / downstep_model, pricing_tiers, and history (index has never exposed any of these). context_window (maximum context window in tokens; null when unknown) and max_output_tokens (the most tokens the model can generate in one response; reasoning tokens count toward it; null when unknown) are on both, as are weights_access (closed, open_permissive or open_restricted; null when unknown) and weights_license. Same authentication and rate limiting as #index.

history: past pricing/status changes, most recent first, reconstructed from this record's audit trail. Each entry's whodunnit is normalized to "automated" (an automated pricing scan or cross-check) or "manual" (a human edit) — never an internal user/admin id. Each changed attribute in changes includes raw before/after values plus a human-readable description.

last_change_detected_at: the timestamp of the most recently detected pricing/status change, not a literal "last time we checked" timestamp — a scan that finds no change leaves no trace, so this can under-report true scan frequency for a model whose data hasn't moved recently.

edge_model / downstep_model: curated comparison references — a newer "same tier" model, and a cheaper alternative, respectively. Each is a minimal {slug, display_name} object, or null if none is set or the related model has no slug.

pricing_tiers: pricing overrides that replace the base rates for a request once a condition is met: input_token_threshold (a flat higher rate for the whole request past a large input-token threshold, condition_trigger) or utc_time_window (a different rate during recurring UTC hours, e.g. peak-hour pricing, given by active_utc_weekdays and active_utc_hours). An empty array if none are configured for this model.

Authorizations:
publicApiKeybearerAuthNone
path Parameters
id
required
string

The model's slug (e.g. 'openai-gpt-4o'), matching its public page URL at /inference-apis/:slug — not the internal numeric id, which isn't a stable public identifier and isn't filterable via q[...] on #index either.

header Parameters
X-API-Key
string

API key for authentication

Authorization
string

Bearer token with API key

Responses

Response samples

Content type
application/json
{
  • "id": 0,
  • "slug": "string",
  • "source_api": "string",
  • "model": "string",
  • "display_name": "string",
  • "provider": "string",
  • "model_type": "llm",
  • "size_tier": "micro",
  • "context_window": 0,
  • "per_input_cost_per_token": 0.1,
  • "per_output_cost_per_token": 0.1,
  • "per_cached_input_cost_per_token": 0.1,
  • "per_cache_creation_input_cost_per_token": 0.1,
  • "per_cache_creation_5m_input_cost_per_token": 0.1,
  • "per_cache_creation_1h_input_cost_per_token": 0.1,
  • "per_cache_storage_cost_per_hour_per_token": 0.1,
  • "capabilities": [
    ],
  • "max_output_tokens": 0,
  • "weights_access": "closed",
  • "weights_license": "string",
  • "available_for_bakeoff": true,
  • "input_modalities": [
    ],
  • "output_modalities": [
    ],
  • "deprecated_at": "2019-08-24T14:15:22Z",
  • "deprecation_notes": "string",
  • "knowledge_cutoff": "2019-08-24",
  • "sources": "string",
  • "updated_at": "2019-08-24T14:15:22Z",
  • "created_at": "2019-08-24T14:15:22Z",
  • "not_human_reviewed": true,
  • "per_input_cost_batch_per_token": 0.1,
  • "per_output_cost_batch_per_token": 0.1,
  • "per_reasoning_output_cost_per_token": 0.1,
  • "edge_model": {
    },
  • "downstep_model": {
    },
  • "pricing_tiers": [
    ],
  • "last_change_detected_at": "2019-08-24T14:15:22Z",
  • "history": [
    ]
}

Client Files

API endpoints for uploading documents/files associated with a client

Create Client File Private key required

Uploads a document as a ClientFile, e.g. so a tool like coolhand-cli can attach project files to a client. id is the file's hashid. metadata is an optional free-form object for client-supplied context - see the project_path convention in the Understanding an LLM Request Log guide's Metadata section, which applies here too. Uploads always land as draft: status is not settable through this endpoint, since nothing reviews an uploaded file's content - promote a file to published from the admin UI once it's been reviewed. Requires the private API key - the public key cannot upload files. Files are currently proxied through the API and capped at 20MB; larger uploads are not yet supported. Accepted formats: pdf, md/markdown, txt, docx/doc, xlsx/xls, pptx/ppt, odt/ods/odp and html/htm. The type is detected from the file's contents and must match its extension; anything else is rejected with a 422 on file.

Authorizations:
privateApiKeybearerAuth
header Parameters
X-API-Key
string

Private API key for authentication

Authorization
string

Bearer token with the private API key

Request Body schema: multipart/form-data
required

Display name for the file

string

Responses

Response samples

Content type
application/json
{
  • "id": "string",
  • "name": "string",
  • "file_type": "string",
  • "status": "string",
  • "description": "string",
  • "metadata": { },
  • "created_at": "2019-08-24T14:15:22Z"
}

CLI Auth

Show CLI auth form — OAuth-style browser token handoff

Displays the browser-based token authorization page for the Coolhand CLI.

Flow overview:

  1. The CLI starts a local HTTP server and opens the browser to GET /cli/auth?redirect_uri=http://localhost:<port>/cb&state=<random>&scope=<scope>.
  2. If the user is unauthenticated, they are redirected to /users/sign_in, which stores the CLI auth URL as the post-login return URL.
  3. After authenticating, the user sees this page, selects (or creates) a Coolhand account, checks the confirmation box(es), and clicks Connect.
  4. The server redirects to redirect_uri with token(s) in the query string (see POST /cli/auth).

scope parameter:

  • Omitted entirely — defaults to public only (backward compatible with callers written before scope was configurable).
  • public — returns only the public API key (token). Use for log ingest and REST API calls.
  • private — returns only the private MCP key (private_token). The UI displays an owner-consent confirmation checkbox; the CLI Connect button is disabled until it's checked, and the form is rejected with 422 if it is submitted without it. Only request private scope when the CLI needs to call the /mcp endpoint.

Regardless of which scope is requested, the page always renders both the public and private key checkboxes and pre-checks only the requested one(s), so the user can grant more than was asked for without re-running the CLI with a different --scope.

  • public,private (or repeated/space-separated) — returns both keys. Requesting an explicit scope with zero valid values (e.g. scope=) is rejected with 400 Bad Request — a CLI must ask for at least one of public or private.

Security: redirect_uri must be an http:// URL with host localhost, 127.0.0.1, or ::1. HTTPS and non-localhost hosts are rejected with 400 Bad Request.

query Parameters
redirect_uri
required
string

Localhost HTTP callback URL for the CLI (e.g. http://localhost:59123/cb)

state
required
string

Opaque CSRF state string echoed back in the callback. Max 256 characters.

scope
string

Key scope(s) to request: public (default), private, or public,private

Responses

Submit CLI auth form — issues token callback redirect

Processes the CLI auth form submission and redirects the browser to the CLI's local callback URL with API key(s) in the query string.

Callback query parameters:

Parameter Always present Description
token Only when scope[] includes public Public API key (ch_pub_*). Use for log ingest and REST API calls.
private_token Only when scope[] includes private Private MCP key (ch_priv_*). Required for POST /mcp.
state ✓ Echoed back from the request — verify this matches your initial state to prevent CSRF.
client_name ✓ Human-readable name of the selected Coolhand account.
client_id ✓ Hashid of the selected Coolhand account.

At least one of token / private_token is always present — submitting an empty scope[] is rejected with 422 Unprocessable Entity.

Using the private key: Pass private_token as the X-API-Key header when calling POST /mcp. The public token is used everywhere else (SDK configuration, ingest endpoint). Never embed the private key in client-side code or commit it to source control.

New client creation: Pass new_client=1 and new_client_name=<name> instead of client_id to create a new Coolhand account on the fly. The callback will carry the new account's freshly generated keys.

Request Body schema: application/x-www-form-urlencoded
required

Localhost HTTP callback URL (must match the GET request value)

string

Responses

Cancel a CLI auth request the user did not initiate

Handles the "I didn't request this" escape hatch on the CLI auth screen. Never redirects to the CLI's redirect_uri — instead sends the user to their own dashboard and posts a Slack alert so the team can investigate a possible spoofed or malicious CLI auth link. No token is issued.

The request URI included in that alert is rebuilt server-side from the validated redirect_uri / state / scope params, so there is no caller-supplied field for it — a tampered form cannot inject text into the alert.

Request Body schema: application/x-www-form-urlencoded
required

Localhost HTTP callback URL (must match the GET request value)

string

Responses

LLM Request Log Model

id
integer

LLM Request Log ID

collector
string or null

Optional name/stamp identifying the collection method (e.g., 'nodejs-sdk-v1.2.0', 'ruby-gem-v2.1.3')

metadata
object

Optional free-form client-supplied context (e.g. { "project_path": "..." })

created_at
string <date-time> (shared_created_at)

Date and time when object was created formatted according to RFC 3339. Timezone is UTC

updated_at
string <date-time> (shared_updated_at)

Date and time when object was updated formatted according to RFC 3339. Timezone is UTC

ingest_evidence
object

Facts Coolhand extracted while processing the log, grouped by topic (max_output_tokens, context_window, modalities, capabilities). A group appears only when the payload carried that signal. Always {} in the create response, since processing happens afterwards.

{
  • "id": 78933,
  • "collector": "nodejs-sdk-v1.2.0",
  • "metadata": {
    },
  • "created_at": "2026-10-07T18:06:07+00:00",
  • "updated_at": "2026-10-07T18:06:07+00:00",
  • "ingest_evidence": {
    }
}

LLM Request Log Feedback Model

id
string

Unique hashid identifier for the feedback entry. Use this to reference the feedback in update requests.

client_id
string

Hashid of the client that owns this feedback entry.

llm_request_log_id
string or null

🎯 Exact Match - Hashid of the LLM request log this feedback is linked to. Provides exact matching to connect feedback to a specific logged request.

workload_id
string or null

📦 Workload Association - Hashid of the workload this feedback is associated with. Set automatically when a valid workload_hashid is provided in the request.

like
boolean or null

👍 Low Signal - Boolean like/dislike rating (deprecated, use sentiment instead). Computed from sentiment for backwards compatibility: like=true, dislike=false, neutral/nil=nil.

sentiment
string or null
Enum: "like" "dislike" "neutral"

🎭 Sentiment Rating - String sentiment value: 'like', 'dislike', or 'neutral'. Preferred over boolean 'like' field. Takes precedence if both are provided.

creator_type
string or null
Enum: "human" "agent" "unknown"

🧑‍🤝‍🤖 Creator Type - What kind of creator submitted this feedback: 'human' (a person), 'agent' (an AI agent or automated tool), or 'unknown'. Defaults to 'unknown' when omitted.

explanation
string or null

💬 Medium Signal - End user explanation of why the response was good or bad. Valuable qualitative data for understanding user preferences and improving model performance.

revised_output
string or null

⭐ Best Signal - End user revision of the LLM response. The highest value data for improving quality scores. This is the user's improved version of what the AI should have said.

llm_provider_unique_id
string or null

🎯 Exact Match - The x-request-id from the LLM API response (e.g., 'req_xxxxxxx'). Provides exact matching to connect feedback to the specific LLM provider request.

original_output
string or null

🔍 Fuzzy Match - The original LLM response text. Provides fuzzy matching but isn't 100% reliable. Use when you don't have llm_provider_unique_id or llm_request_log_id.

client_unique_id
string or null

🔗 Your Internal Matcher - Connect to an identifier from your system for internal matching. This helps you correlate feedback with your own request tracking system.

creator_unique_id
string or null

👤 User Tracking - Unique ID to match feedback to the end user who created it. Useful for analyzing feedback patterns by user and preventing duplicate feedback.

collector
string or null

🏷️ Metadata - Optional name/stamp identifying the collection method or SDK version. Helps track which system or version collected the feedback.

coolhand_fingerprint_id
string or null

🔒 Optional - Unique fingerprint ID set by the CoolhandJS SDK (https://github.com/Coolhand-Labs/coolhand-js). DO NOT set this field when calling the API directly - it is reserved for CoolhandJS.

created_at
string <date-time> (shared_created_at)

Date and time when object was created formatted according to RFC 3339. Timezone is UTC

updated_at
string <date-time> (shared_updated_at)

Date and time when object was updated formatted according to RFC 3339. Timezone is UTC

{
  • "id": "abc123xyz789",
  • "client_id": "xyz789abc123",
  • "llm_request_log_id": "abc123xyz789",
  • "workload_id": "xyz789abc123",
  • "like": true,
  • "sentiment": "like",
  • "creator_type": "human",
  • "explanation": "Great response! Very helpful and accurate.",
  • "revised_output": "This is a dummy revised output",
  • "llm_provider_unique_id": "req_1234567890abcdef",
  • "original_output": "Dear John Doe,\n\nThank you for reaching out to us regarding the issue with your recent order #12345...",
  • "client_unique_id": "email-service-1760025972258",
  • "creator_unique_id": "user-789",
  • "collector": "coolhand-node-0.1.0",
  • "coolhand_fingerprint_id": "fp_abc123xyz789",
  • "created_at": "2026-10-07T18:06:07+00:00",
  • "updated_at": "2026-10-07T18:06:07+00:00"
}

Client File Model

id
string

Client file hashid

name
string

Display name for the file

file_type
string

One of slide_deck, report, document

status
string

One of draft, published, archived

description
string or null

Optional free-text description

object

Optional free-form client-supplied context (e.g. { "project_path": "..." })

created_at
string <date-time> (shared_created_at)

Date and time when object was created formatted according to RFC 3339. Timezone is UTC

{
  • "id": "abc123",
  • "name": "Q1 Report",
  • "file_type": "document",
  • "status": "draft",
  • "description": "string",
  • "metadata": {
    },
  • "created_at": "2026-10-07T18:06:07+00:00"
}