API docs by Redocly

Coolhand LLM Request Logging API (v2)

Download OpenAPI specification:Download

LLM Request Logging API Documentation.

Introduction

This API allows clients to log and store request/response data from Large Language Model (LLM) API calls for monitoring, analysis, and debugging purposes. The API accepts any JSON structure in the raw_request field, making it flexible for different LLM providers and use cases.

Authentication

API uses API Key authentication. Clients are issued unique API keys that must be included in requests using one of these methods:

  • X-API-Key header: X-API-Key: your_api_key_here
  • Authorization Bearer header: Authorization: Bearer your_api_key_here
  • Query parameter: ?api_key=your_api_key_here

Contact your administrator to obtain an API key.

apiKeyAuth

API key for client authentication

Security Scheme Type: API Key
Header parameter name: X-API-Key

bearerAuth

Bearer token authentication using API key

Security Scheme Type: HTTP
HTTP Authorization Scheme: bearer

Common Use Cases

OpenAI API Logging

Log requests and responses from OpenAI's chat completions, embeddings, and other endpoints.

Anthropic Claude Logging

Store conversation data and model responses from Claude API calls.

Custom LLM Provider Logging

The flexible JSON structure supports any LLM provider's request/response format.

Analytics and Monitoring

Aggregate usage statistics, token consumption, and performance metrics across LLM calls.

API Error Handling

Error Response Structure

When interacting with the LLM Request Logging API, you'll encounter standard error response formats for different types of failures.

object

Contains error details.

{
  • "errors": {
    }
}

HTTP Status Codes

Understanding the HTTP status codes for this API:

HTTP Status Code Description
201 Created LLM request log successfully created
400 Bad Request Malformed request or invalid JSON structure
401 Unauthorized Missing, invalid, or expired API key
422 Unprocessable Entity Validation error - required fields missing or invalid format
5xx Server Errors Server-side issues - please report these for investigation

LLM Request Logs

API endpoints for logging and storing LLM (Large Language Model) request/response data

Create LLM Request Log

Creates a new LLM request log entry for monitoring and analysis purposes.

query Parameters
api_key
string

API key as query parameter

header Parameters
X-API-Key
string

API key for authentication

Authorization
string

Bearer token with API key

Request Body schema: application/json
required
required
object

Responses

Request samples

Content type
application/json
{
  • "llm_request_log": {
    }
}

Response samples

Content type
application/json
{
  • "id": 0,
  • "collector": "string",
  • "created_at": "2019-08-24T14:15:22Z",
  • "updated_at": "2019-08-24T14:15:22Z"
}

LLM Request Log Feedbacks

API endpoints for collecting user feedback on LLM request/response quality and accuracy

Create LLM Request Log Feedback

Creates a new feedback entry for LLM request monitoring and improvement purposes.

Field Guide - All fields are optional, but here's how to get the best results:

Matching Fields

  • llm_request_log_id 🎯 Exact Match - ID from the Coolhand API response when the original LLM request was logged. Provides exact matching.
  • llm_provider_unique_id 🎯 Exact Match - The x-request-id from the LLM API response (e.g., "req_xxxxxxx")
  • original_output 🔍 Fuzzy Match - The original LLM response text. Provides fuzzy matching but isn't 100% reliable.
  • client_unique_id 🔗 Your Internal Matcher - Connect to an identifier from your system for internal matching
  • workload_hashid 📦 Workload Association - Optional. Associate feedback with a specific workload to increase fuzzy match success rate

Quality Data

  • revised_output ⭐ Best Signal - End user revision of the LLM response. The highest value data for improving quality scores.
  • explanation 💬 Medium Signal - End user explanation of why the response was good or bad. Valuable qualitative data.
  • like 👍 Low Signal - Boolean like/dislike. Lower quality signal but easy for users to provide.
  • creator_unique_id 👤 User Tracking - Unique ID to match feedback to the end user who created it
  • collector 🏷️ Metadata - Optional name/stamp identifying the collection method (e.g., "coolhand-node-0.1.0")
query Parameters
api_key
string

API key as query parameter

collector
string

🏷️ Optional collector identifier (can also be provided in request body)

header Parameters
X-API-Key
string

API key for authentication

Authorization
string

Bearer token with API key

Request Body schema: application/json
required
required
object

Responses

Request samples

Content type
application/json
{
  • "llm_request_log_feedback": {
    }
}

Response samples

Content type
application/json
{
  • "id": "string",
  • "client_id": 0,
  • "llm_request_log_id": 0,
  • "workload_id": 0,
  • "like": true,
  • "explanation": "string",
  • "revised_output": "string",
  • "llm_provider_unique_id": "string",
  • "original_output": "string",
  • "client_unique_id": "string",
  • "creator_unique_id": "string",
  • "collector": "string",
  • "coolhand_fingerprint_id": "string",
  • "created_at": "2019-08-24T14:15:22Z",
  • "updated_at": "2019-08-24T14:15:22Z",
  • "warnings": [
    ]
}

Update LLM Request Log Feedback

Updates an existing feedback entry. Only specific fields can be updated - identity fields are immutable.

Updatable Fields:

  • like - Boolean like/dislike rating
  • explanation - End user explanation
  • revised_output - End user revision of the LLM response
  • original_output - The original LLM response text
  • llm_provider_unique_id - The x-request-id from the LLM API response
  • collector - Collection method identifier
  • client_unique_id - Your internal request identifier
  • workload_hashid - Associate with a workload (or clear by setting to null/empty)

Immutable Fields (cannot be updated):

  • creator_unique_id
  • llm_request_log_id

Note: After a successful update, if the feedback is not already matched to a log and has original_output, the fuzzy matching job will be re-triggered to attempt matching.

path Parameters
id
required
string

Feedback hashid

header Parameters
X-API-Key
string

API key for authentication

Authorization
string

Bearer token with API key

Request Body schema: application/json
required
object

Responses

Request samples

Content type
application/json
{
  • "llm_request_log_feedback": {
    }
}

Response samples

Content type
application/json
{
  • "id": "string",
  • "client_id": 0,
  • "llm_request_log_id": 0,
  • "workload_id": 0,
  • "like": true,
  • "explanation": "string",
  • "revised_output": "string",
  • "llm_provider_unique_id": "string",
  • "original_output": "string",
  • "client_unique_id": "string",
  • "creator_unique_id": "string",
  • "collector": "string",
  • "coolhand_fingerprint_id": "string",
  • "created_at": "2019-08-24T14:15:22Z",
  • "updated_at": "2019-08-24T14:15:22Z"
}

LLM Request Log Model

id
integer

LLM Request Log ID

collector
string or null

Optional name/stamp identifying the collection method (e.g., 'nodejs-sdk-v1.2.0', 'ruby-gem-v2.1.3')

created_at
string <date-time> (shared_created_at)

Date and time when object was created formatted according to RFC 3339. Timezone is UTC

updated_at
string <date-time> (shared_updated_at)

Date and time when object was updated formatted according to RFC 3339. Timezone is UTC

{
  • "id": 97917,
  • "collector": "nodejs-sdk-v1.2.0",
  • "created_at": "2026-01-13T16:55:06+00:00",
  • "updated_at": "2026-01-13T16:55:06+00:00"
}

LLM Request Log Feedback Model

id
string

Unique hashid identifier for the feedback entry. Use this to reference the feedback in update requests.

llm_request_log_id
integer or null

🎯 Exact Match - ID from the Coolhand API response when the original LLM request was logged. Provides exact matching to connect feedback to a specific logged request.

workload_id
integer or null

📦 Workload Association - ID of the workload this feedback is associated with. Set automatically when a valid workload_hashid is provided in the request.

like
boolean or null

👍 Low Signal - Boolean like/dislike rating. Lower quality signal but easy for users to provide. Use true for positive feedback, false for negative, or null/omit for neutral.

explanation
string or null

💬 Medium Signal - End user explanation of why the response was good or bad. Valuable qualitative data for understanding user preferences and improving model performance.

revised_output
string or null

⭐ Best Signal - End user revision of the LLM response. The highest value data for improving quality scores. This is the user's improved version of what the AI should have said.

llm_provider_unique_id
string or null

🎯 Exact Match - The x-request-id from the LLM API response (e.g., 'req_xxxxxxx'). Provides exact matching to connect feedback to the specific LLM provider request.

original_output
string or null

🔍 Fuzzy Match - The original LLM response text. Provides fuzzy matching but isn't 100% reliable. Use when you don't have llm_provider_unique_id or llm_request_log_id.

client_unique_id
string or null

🔗 Your Internal Matcher - Connect to an identifier from your system for internal matching. This helps you correlate feedback with your own request tracking system.

creator_unique_id
string or null

👤 User Tracking - Unique ID to match feedback to the end user who created it. Useful for analyzing feedback patterns by user and preventing duplicate feedback.

collector
string or null

🏷️ Metadata - Optional name/stamp identifying the collection method or SDK version. Helps track which system or version collected the feedback.

coolhand_fingerprint_id
string or null

🔒 Optional - Unique fingerprint ID set by the CoolhandJS SDK (https://github.com/Coolhand-Labs/coolhand-js). DO NOT set this field when calling the API directly - it is reserved for CoolhandJS.

created_at
string <date-time> (shared_created_at)

Date and time when object was created formatted according to RFC 3339. Timezone is UTC

updated_at
string <date-time> (shared_updated_at)

Date and time when object was updated formatted according to RFC 3339. Timezone is UTC

{
  • "id": "abc123xyz789",
  • "llm_request_log_id": 84272,
  • "workload_id": 82548,
  • "like": true,
  • "explanation": "Great response! Very helpful and accurate.",
  • "revised_output": "This is a dummy revised output",
  • "llm_provider_unique_id": "req_1234567890abcdef",
  • "original_output": "Dear John Doe,\n\nThank you for reaching out to us regarding the issue with your recent order #12345...",
  • "client_unique_id": "email-service-1760025972258",
  • "creator_unique_id": "user-789",
  • "collector": "coolhand-node-0.1.0",
  • "coolhand_fingerprint_id": "fp_abc123xyz789",
  • "created_at": "2026-01-13T16:55:06+00:00",
  • "updated_at": "2026-01-13T16:55:06+00:00"
}