Coolhand LLM Request Logging API (v2)

Creates a new LLM request log entry for monitoring and analysis purposes. See the Understanding an LLM Request Log guide for a full description of the fields Coolhand captures, which fields each provider exposes, and common gotchas. id is the log's hashid (a string), not the internal integer primary key — every log identifier returned by this API is a hashid, matching the GET endpoints below; consumers that previously parsed id as an integer need to treat it as an opaque string instead.

Returns this client's LLM request logs as a bare JSON array (unlike GET /api/v2/llm_request_log_feedbacks's { feedback:, pagination: } envelope - coolhand-cli's fetchLastSync relies on this response being Array.isArray, so the body shape is intentionally not wrapped). Pagination totals are exposed via X-Page/X-Per-Page response headers, plus X-Total-Count/X-Total-Pages when include_total=true is passed (omitted by default to skip the COUNT(*) query - analyze-claude-sessions's frequent per=1 dedup-cutoff poll never needs a total). Supports Ransack search via q[...] params (e.g. q[source_api_in][]=claude_code), sorting via q[s] (e.g. created_at desc for newest first; when no q[s] is supplied, the endpoint applies a stable default sort of id desc for newest first so pagination is deterministic - pass q[s] explicitly to override), and pagination via page/per (per_page also accepted; default 25, max 100). Requires the private API key - the public key is write-only and cannot read. id is always the log's hashid (not the internal integer PK) - the sort itself still orders by the real primary key internally, but a cutoff/cursor derived from results should key off created_at, not id, since hashids don't sort lexically in creation order. analyze-claude-sessions uses this to derive its dedup cutoff by requesting the newest claude_code/claude_cowork record (per=1), instead of a bespoke endpoint. Also supports named filters (applied on top of any q[...] filtering, not in place of it): template_id/workload_id (hashid), system_prompt_contains/user_prompt_contains (case-insensitive substring), model, source_api, source_api_result, project_path (exact match against metadata.project_path), unmatched_only, days_back (unset by default - unlike search_logs's MCP tool, index has always returned unrestricted results and does not implicitly apply a 30-day window), and include_prompts (adds system_prompt/user_prompt, truncated to 500 chars, to each result).

Fetches full input/output content for a single log by its ID (hashid). Accepts any of this client's private credentials: the client's own private API key, a team member's UserClient private key, an AdminClientCredential, or the scoped, rotating key of the client's agent user (see Clients::AgentUserProvisioner) used by the change-suggestion coding agent to ground a fix in the log it was told to look at. Supports section/max_chars for large logs and search_query for snippet search. Only directly-collected logs are fetchable here (matching index's .client_logs restriction) - internally generated records (evals, synthetic logs) 404 even when the hashid is known.

Creates a new feedback entry for LLM request monitoring and improvement purposes.

Field Guide - All fields are optional, but here's how to get the best results:

Matching Fields

• llm_request_log_id 🎯 Exact Match - Hashid from the Coolhand API response when the original LLM request was logged. Provides exact matching.
• llm_provider_unique_id 🎯 Exact Match - The x-request-id from the LLM API response (e.g., "req_xxxxxxx")
• original_output 🔍 Fuzzy Match - The original LLM response text. Provides fuzzy matching but isn't 100% reliable.
• client_unique_id 🔗 Your Internal Matcher - Connect to an identifier from your system for internal matching
• workload_hashid 📦 Workload Association - Optional. Associate feedback with a specific workload to increase fuzzy match success rate

Quality Data

• revised_output ⭐ Best Signal - End user revision of the LLM response. The highest value data for improving quality scores.
• explanation 💬 Medium Signal - End user explanation of why the response was good or bad. Valuable qualitative data.
• like 👍 Low Signal - Boolean like/dislike. Lower quality signal but easy for users to provide.
• creator_unique_id 👤 User Tracking - Unique ID to match feedback to the end user who created it
• collector 🏷️ Metadata - Optional name/stamp identifying the collection method (e.g., "coolhand-node-0.1.0")

Returns this client's feedback entries. Supports Ransack search via q[...] params (e.g. q[sentiment_eq]=2 - sentiment is stored as the integer codes 0=dislike/1=neutral/2=like, which is what responses render as the strings 'dislike'/'neutral'/'like' - or q[created_at_gteq]=...), sorting via q[s] (e.g. created_at desc; when no q[s] is supplied, the endpoint applies a stable default sort of id desc so pagination is deterministic), and pagination via page/per (default 25, max 100). Filtering and sorting on original_output/revised_output are not supported. Accepts any of this client's private credentials - the client's own private API key, a team member's UserClient private key, an AdminClientCredential, or the scoped, rotating key of the client's agent user (see Clients::AgentUserProvisioner) used by the change-suggestion coding agent to ground a fix in the feedback it was told to look at - but not the public key, which is write-only and cannot read. List items omit original_output/revised_output (which can each hold up to 1GB) - fetch a specific item via GET /api/v2/llm_request_log_feedbacks/{id} to get the full text and any feedback_partials. Also sets X-Total-Count/X-Page/X-Per-Page/X-Total-Pages response headers - the same pagination signal every paginated v2 endpoint exposes (see GET /api/v2/llm_request_logs); the body's pagination object is kept for backward compatibility, not the pattern to rely on for new integrations.

Updates an existing feedback entry. Only specific fields can be updated - identity fields are immutable.

Partial Feedback Updates: When partial_id is provided in the request body, the update is applied to that specific FeedbackPartial instead of the parent feedback. This is useful when adding explanations to partial feedback (highlighted text sections). The response will include updated_partial_id to confirm which partial was updated.

Updatable Fields (Parent Feedback) - public key:

• like - Boolean like/dislike rating
• sentiment - Sentiment value (like/dislike/neutral)
• explanation - End user explanation
• revised_output - End user revision of the LLM response

Updatable Fields (Parent Feedback) - private key / OAuth Bearer token: Everything above, plus:

• creator_type - Who supplied the feedback (human/agent/unknown)
• original_output - The original LLM response text
• llm_provider_unique_id - The x-request-id from the LLM API response
• collector - Collection method identifier
• client_unique_id - Your internal request identifier
• coolhand_fingerprint_id - Fingerprint id used for dedup/matching
• workload_hashid - Associate with a workload (or clear by setting to null/empty)

A public-key request that includes a private-key-only field does not error - that field is silently ignored, and only the public-key-allowed fields in the request are applied.

Response shape: for a public-key caller the response omits original_output, revised_output and the feedback_partials array (reading a record back is a private-credential operation - see GET /api/v2/llm_request_log_feedbacks/{id}). updated_partial_id is still returned so a partial update can be confirmed. Private key and OAuth Bearer callers get the full payload documented below. Any field the request sent that the credential may not write is reported back in warnings.

Updatable Fields (FeedbackPartial - when partial_id is provided) - all credential types:

• explanation - End user explanation for this specific partial
• sentiment - Sentiment value (like/dislike/neutral)
• like - Boolean (automatically converted to sentiment)

Partials have no creator_type - it is a parent-feedback field only, and is ignored here for every credential type.

Immutable Fields (cannot be updated):

• creator_unique_id
• llm_request_log_id

Note: After a successful update, if the feedback is not already matched to a log and has original_output, the fuzzy matching job will be re-triggered to attempt matching.

Fetches a single feedback entry by hashid, including original_output, revised_output, and any feedback_partials - the full record that GET /api/v2/llm_request_log_feedbacks omits from list results. Accepts any of this client's private credentials - the client's own private API key, a team member's UserClient private key, an AdminClientCredential, or the scoped, rotating key of the client's agent user (see Clients::AgentUserProvisioner) used by the change-suggestion coding agent to ground a fix in the feedback it was told to look at - but not the public key, which is write-only and cannot read.

Returns this client's LLM request templates as a bare JSON array (the same body shape as GET /api/v2/llm_request_logs, not the { feedback:, pagination: } envelope used by GET /api/v2/llm_request_log_feedbacks). Pagination metadata is always exposed via the X-Page, X-Per-Page, X-Total-Count and X-Total-Pages response headers — unlike /llm_request_logs, this endpoint has no include_total opt-out, because a client's template count is small enough that counting the templates themselves is cheap.

log_count is the expensive part of this response, not the pagination total. It aggregates over llm_request_logs, so its cost scales with how many logs the matched templates hold rather than with how many templates come back — the Unmatched bucket reached via include_system=true can hold every log that never matched a template.

The 10-second guard is per statement, not per response — read this before sizing a client timeout. #index issues three statements inside it (the page load, the pagination COUNT, and the grouped log count), and Postgres applies statement_timeout to each one independently. Database time for a single response can therefore reach three times that figure before anything is cancelled, and a 504 here means one query exceeded 10 seconds — never that the response as a whole did. Total response time is not bounded by this endpoint. Do not treat the absence of a 504 as evidence a response was fast, and do not derive a client timeout from the 10-second number: size it from observed end-to-end latency instead. Narrow with workload_id, search, or a smaller per if you do hit a 504.

log_count counts only directly-collected client logs — the same records GET /api/v2/llm_request_logs?template_id=… returns — so the two numbers agree. Internally generated records (evals, bakeoff comparisons, synthetic logs) are excluded, which is why this can be lower than the count the search_templates MCP tool reports.

The client is always derived from the authenticating API key. There is no client_id parameter and one cannot be supplied.

Requires the private API key — the public key is write-only and cannot read data back.

Results are ordered newest first (created_at DESC, with the internal primary key as a tiebreaker so paging is deterministic when two templates share a timestamp).

Prompt patterns are not included here. Fetch a single template via GET /api/v2/llm_request_templates/{id} for user_prompt_pattern / system_prompt_pattern.

System templates. Every client has two system buckets, Unmatched and Ignored API Calls (LlmRequestTemplate::SYSTEM_TEMPLATE_NAMES), created automatically with the client. They are excluded from this list by default and returned when include_system=true is passed — the Unmatched bucket is what you inspect when logs are misrouting. Each row carries a system_template boolean so callers do not have to match on names.

Deprecated templates are excluded by default and returned when include_deprecated=true is passed. A deprecated row is identifiable by a non-null deprecated_at. (The originating issue called this filter deprecated; it is named include_deprecated here to match both include_system above and the existing search_templates MCP parameter, and because deprecated=true would be ambiguous between "only deprecated" and "also deprecated".)

Archived workloads are not filtered out. Unlike the search_templates MCP tool, which hides templates whose workload has been archived, this endpoint returns them — otherwise the list would disagree with GET /api/v2/llm_request_templates/{id}, which can always fetch such a template by id. Use workload_id to narrow instead.

Not included in this endpoint: template version history. LlmRequestTemplate versions user_prompt_pattern, system_prompt_pattern, name, status, workload_id and the deprecation fields via PaperTrail, and a versions sub-resource was considered alongside these two endpoints, but it is deliberately deferred to its own change rather than shipped here — it is a distinct resource with its own response shape and its own disclosure surface (historical prompt patterns). Template mutation likewise stays on the MCP surface; this REST surface is read-only.

Returns a single template belonging to the authenticating client, including user_prompt_pattern and system_prompt_pattern — the full untruncated regexes, which the list endpoint omits.

Unlike the list endpoint, this action applies no default filtering beyond client ownership: a deprecated template or a system template (Unmatched, Ignored API Calls) is reachable by id without any opt-in flag, because inspecting one of those is the usual reason to fetch a template directly.

A template belonging to another client returns 404, not 403 — its existence is not disclosed.

As on the list endpoint, log_count is the expensive part: it aggregates over llm_request_logs, and fetching the Unmatched bucket by id counts every log that never matched a template. That count is the one statement bounded by the 10-second statement_timeout, so on this action the guard does correspond to a single query — but it still bounds only that query, not the whole response (the record lookup and authentication run outside it). A 504 means the count exceeded 10 seconds. See the list endpoint for why total response time is not bounded either. log_count counts only directly-collected client logs, matching GET /api/v2/llm_request_logs?template_id=….

Requires the private API key.

Returns a list of all inference models tracked by Coolhand, including per-token pricing metadata.

Authentication: Authentication is optional. Unauthenticated callers are rate-limited (see below). Authenticated callers bypass the rate limit entirely. Pass a public API key via the X-API-Key header or as a Bearer token in the Authorization header. Sign up at coolhand.ai to get a free API key. A private key is not accepted here — this endpoint only recognizes public keys (and rejects a private key the same way it would an invalid one).

How pricing data is gathered: Pricing data is collected by the Godfrey agent, an automated system that periodically fetches and parses pricing pages published by model providers. All records undergo human review before being considered authoritative. The sources field on each record links to the original provider pricing page used as the data source.

Cost units: Every _per_token field is a raw USD cost-per-token value. Multiply by 1,000,000 for the "per 1M tokens" pricing commonly published by providers. per_cache_storage_cost_per_hour_per_token is the one exception: it's priced per hour of storage, not per request — an ongoing hourly charge for holding tokens in a cache (as Gemini's explicit context caching bills), not a one-time per-token cost.

Rate limiting: Unauthenticated requests are rate limited to 1 request per minute per IP address. Exceeding the limit returns a 429 Too Many Requests response. Authenticated requests are never rate limited.

Returns every field #index's summary response has, plus the fields #index omits: the internal not_human_reviewed review-status flag, the slug used in this model's public page URL, the batch and reasoning-output per-token pricing columns, edge_model / downstep_model, pricing_tiers, and history (index has never exposed any of these). Same authentication and rate limiting as #index.

history: past pricing/status changes, most recent first, reconstructed from this record's audit trail. Each entry's whodunnit is normalized to "automated" (an automated pricing scan or cross-check) or "manual" (a human edit) — never an internal user/admin id. Each changed attribute in changes includes raw before/after values plus a human-readable description.

last_change_detected_at: the timestamp of the most recently detected pricing/status change, not a literal "last time we checked" timestamp — a scan that finds no change leaves no trace, so this can under-report true scan frequency for a model whose data hasn't moved recently.

edge_model / downstep_model: curated comparison references — a newer "same tier" model, and a cheaper alternative, respectively. Each is a minimal {slug, display_name} object, or null if none is set or the related model has no slug.

pricing_tiers: cliff-style pricing overrides that replace the base rates once a condition is met (e.g. a flat higher rate for the whole request past a large input-token threshold). An empty array if none are configured for this model.

Attaches an existing LlmRequestLogFeedback to an optimization as supporting evidence. Requires a private credential (the client's own private API key, a team member's UserClient private key, or an AdminClientCredential) or a Doorkeeper Bearer token - the public api_key is rejected on this endpoint. Breaking change: the response id field is now the evidence link's hashid (a string), not the internal integer primary key - pass it back as-is to the corresponding DELETE .../{id} endpoint; consumers that previously parsed or stored it as an integer need to treat it as an opaque string.

Removes a feedback-evidence link from an optimization. Requires a private credential (see the create operation's description) - the public api_key is rejected.

Attaches an existing LlmRequestLog to an optimization as supporting evidence. Requires a private credential (see the feedback-links create operation's description) - the public api_key is rejected on this endpoint.

Removes a log-evidence link from an optimization. Requires a private credential (see the create operation's description) - the public api_key is rejected.

Attaches an existing LlmRequestEval to an optimization as supporting evidence. Requires a private credential (see the feedback-links create operation's description) - the public api_key is rejected on this endpoint.

Removes an eval-evidence link from an optimization. Requires a private credential (see the create operation's description) - the public api_key is rejected.

Records a change suggestion (a proposed fix/PR) against the authenticated client.

suggestion_type - one of setup, optimization_fix.

status - one of draft, open, merged, closed, awaiting_client_review, incomplete, awaiting_repo_selection.

The suggestion is always scoped to the authenticated client - any client_id in the request body is ignored.

Requires a private credential (the client's own private API key, a team member's UserClient private key, or an AdminClientCredential) or a Doorkeeper Bearer token - the public api_key is rejected on this endpoint.

Breaking change: the response id field is now the change suggestion's hashid (a string), not the internal integer primary key - consumers that previously parsed it as an integer need to treat it as an opaque string.

Uploads a document as a ClientFile, e.g. so a tool like coolhand-cli can attach project files to a client. id is the file's hashid. metadata is an optional free-form object for client-supplied context - see the project_path convention in the Understanding an LLM Request Log guide's Metadata section, which applies here too. Uploads always land as draft: status is not settable through this endpoint, since nothing validates an uploaded file's content - promote a file to published from the admin UI once it's been reviewed. Requires the private API key - the public key cannot upload files. Files are currently proxied through the API and capped at 20MB; larger uploads are not yet supported.

Displays the browser-based token authorization page for the Coolhand CLI.

Flow overview:

1. The CLI starts a local HTTP server and opens the browser to GET /cli/auth?redirect_uri=http://localhost:<port>/cb&state=<random>&scope=<scope>.
2. If the user is unauthenticated, they are redirected to /users/sign_in, which stores the CLI auth URL as the post-login return URL.
3. After authenticating, the user sees this page, selects (or creates) a Coolhand account, checks the confirmation box(es), and clicks Connect.
4. The server redirects to redirect_uri with token(s) in the query string (see POST /cli/auth).

scope parameter:

• Omitted entirely — defaults to public only (backward compatible with callers written before scope was configurable).
• public — returns only the public API key (token). Use for log ingest and REST API calls.
• private — returns only the private MCP key (private_token). The UI displays an owner-consent confirmation checkbox; the CLI Connect button is disabled until it's checked, and the form is rejected with 422 if it is submitted without it. Only request private scope when the CLI needs to call the /mcp endpoint.

Regardless of which scope is requested, the page always renders both the public and private key checkboxes and pre-checks only the requested one(s), so the user can grant more than was asked for without re-running the CLI with a different --scope.

• public,private (or repeated/space-separated) — returns both keys. Requesting an explicit scope with zero valid values (e.g. scope=) is rejected with 400 Bad Request — a CLI must ask for at least one of public or private.

Security: redirect_uri must be an http:// URL with host localhost, 127.0.0.1, or ::1. HTTPS and non-localhost hosts are rejected with 400 Bad Request.

Processes the CLI auth form submission and redirects the browser to the CLI's local callback URL with API key(s) in the query string.

Callback query parameters:

At least one of token / private_token is always present — submitting an empty scope[] is rejected with 422 Unprocessable Entity.

Using the private key: Pass private_token as the X-API-Key header when calling POST /mcp. The public token is used everywhere else (SDK configuration, ingest endpoint). Never embed the private key in client-side code or commit it to source control.

New client creation: Pass new_client=1 and new_client_name=<name> instead of client_id to create a new Coolhand account on the fly. The callback will carry the new account's freshly generated keys.

Handles the "I didn't request this" escape hatch on the CLI auth screen. Never redirects to the CLI's redirect_uri — instead sends the user to their own dashboard and posts a Slack alert so the team can investigate a possible spoofed or malicious CLI auth link. No token is issued.

The request URI included in that alert is rebuilt server-side from the validated redirect_uri / state / scope params, so there is no caller-supplied field for it — a tampered form cannot inject text into the alert.