Skip to main content

How do I list and analyze my workloads using the API?

Last updated 2 min read

A Coolhand workload is a group of prompt templates that make up a single task or agent. Use the `/api/v2/workloads` endpoint to list your workloads and return performance and cost metrics for them. This allows you to programmatically monitor your AI agents and tasks from your own tools.

What is a workload?

A workload is a collection of prompt templates that together form a single logical task or AI agent. Grouping templates into workloads allows you to analyze the performance and cost of an entire agent at once, even if it uses many different prompts to accomplish its goal.

Listing your workloads

To get a list of your account's workloads, send a GET request to the /api/v2/workloads endpoint. This will return a JSON array of workload objects, ordered by name.

curl -H "Authorization: Bearer $COOLHAND_API_KEY" https://coolhand.com/api/v2/workloads

The basic response includes the workload's ID, name, description, and counts of its templates and logs.

[
  {
    "id": "wkld-12345",
    "name": "Invoice Data Extraction",
    "description": "Extracts structured data from invoices.",
    "template_count": 5,
    "draft_template_count": 1,
    "log_count": 1523,
    "last_activity": "2024-08-15T18:45:00Z",
    "archived": false,
    "system": false
  }
]

Including templates and metrics

To get more detailed information, you can include two powerful parameters:

  • include_templates=true: Adds a templates array to each workload object, listing the prompt templates it contains.
  • include_metrics=true: Adds a metrics object with a full cost and performance analysis, calculated over the last 28 days by default.
curl -H "Authorization: Bearer $COOLHAND_API_KEY" \
  "https://coolhand.com/api/v2/workloads?include_templates=true&include_metrics=true"

The response will now include the templates and metrics objects:

[
  {
    "id": "wkld-12345",
    "name": "Invoice Data Extraction",
    // ... other fields
    "templates": [
      {
        "id": "tmpl-abcde",
        "name": "Extract invoice header",
        "status": "active",
        "user_prompt_pattern": "Extract the header from this invoice: ...",
        "system_prompt_pattern": null
      }
    ],
    "metrics": {
      "request_count": 1523,
      "error_rate": 0.02,
      "avg_latency_ms": 1250,
      "total_cost": 4.56,
      "avg_cost_per_request": 0.003,
      "correctness_score": 0.85,
      "sentiment_score": 0.92,
      "days_back": 28
      // ... other metrics
    }
  }
]

You can change the lookback window for metrics by using the days_back parameter (e.g., days_back=90).

Filtering workloads

Use the search parameter to find workloads by name.

curl -H "Authorization: Bearer $COOLHAND_API_KEY" \
  "https://coolhand.com/api/v2/workloads?search=Invoice"

By default, the endpoint excludes archived and internal system workloads (like Unmatched). You can include them by setting include_archived=true or include_system=true.

Related articles