Skip to main content

Creators vs. reviewers

Last updated 5 min read

A creator is one raw identifier attached to a single feedback submission. A reviewer is the person behind it, assembled automatically by grouping every creator identifier and fingerprint belonging to the same individual. Reviewer-level analytics tell you how many people gave feedback, not how many sessions did.

This guide explains the difference between a creator and a reviewer in Coolhand, and why reviewers give you a more accurate picture of who is submitting feedback.

The short version

  • A creator is a single raw identifier you attach to one feedback submission.
  • A reviewer is the canonical person, automatically assembled by grouping all of the creator identifiers (and Coolhand fingerprints) that belong to the same individual.

One reviewer can have many creators. Filtering or reporting by reviewer gives you everything that person submitted; filtering by a single creator only gives you the slice tagged with that one identifier.

What is a creator?

When you submit feedback, you can include a creator_unique_id — any string that identifies who gave the feedback in your system. Common choices are your application's user ID, an email, or a session handle.

POST /api/v2/llm_request_log_feedbacks
{
  "llm_request_log_feedback": {
    "sentiment": "like",
    "creator_unique_id": "user_8842"
  }
}

A creator identifier is taken at face value. If the same person submits feedback from your web app as user_8842 and from your mobile app as mobile:8842, those are two different creators even though it's one human.

Some feedback carries no creator_unique_id at all and is identified only by an automatically captured Coolhand fingerprint (coolhand_fingerprint_id).

What is a reviewer?

A reviewer is the deduplicated person behind the feedback. Coolhand automatically links each incoming submission to a reviewer based on the identifiers it carries:

  • Every creator_unique_id seen for a reviewer becomes one of their aliases.
  • Every coolhand_fingerprint_id seen for a reviewer becomes one of their aliases.

Because a reviewer can hold several aliases, the two submissions above (user_8842 and mobile:8842) — and any fingerprint-only submissions from the same person — roll up to one reviewer. This happens for you; you do not need to send a reviewer ID.

A reviewer therefore answers the question "who is this person, across everything they've sent?", while a creator answers "what label was on this one submission?".

When to use which

You want to… Use
See every submission from one person, no matter which app or identifier they used reviewer
Look up the exact submissions tagged with one specific raw identifier creator
Report on how many distinct people are giving feedback reviewer
Trace where a particular creator_unique_id value is coming from creator

In the MCP tools, reviewer filters expect a Reviewer ID (the canonical reviewer's identifier, available from reviewer stats), and they return all feedback across that reviewer's aliases. Creator filters match the raw creator_unique_id string directly.

Why it matters

If you measure engagement or quality per creator, a single power user who submits under three identifiers will look like three light users, and any fingerprint-only feedback they left won't be counted with the rest. Measuring per reviewer collapses those back into the one person they represent, so counts, frequency, and history reflect reality.

Swarmer and Captain badges

Reviewer badges appear on the workload analytics page and on each reviewer's profile. They describe how a reviewer engages with feedback, not just how much they submit.

Swarmer

A Swarmer is a reviewer who consistently gives feedback quickly — within 5 minutes of a log being created. Fast feedback is valuable because it captures a reviewer's immediate reaction before they've been influenced by other outputs.

The Swarmer % for a reviewer is:

(number of feedback items submitted within 5 minutes of the matched log's creation) ÷ (total matched feedback items) × 100

Only feedback that is matched to a log contributes — unmatched feedback has no creation timestamp to compare against.

Grade levels:

Badge Threshold
Swarmer Bronze ≥ 50% of matched feedback submitted within 5 minutes
Swarmer Silver ≥ 70%
Swarmer Gold ≥ 90%

No badge is shown if the reviewer is below 50%.

Captain

A Captain is a reviewer who reviews in focused sessions — submitting several pieces of feedback in a row with less than 5 minutes between consecutive items. Batch reviewing signals intentional, systematic evaluation.

The Captain % for a reviewer is:

(number of feedback items that belong to a session of 2 or more) ÷ (total feedback items) × 100

A new session starts whenever a reviewer's gap between consecutive feedback items exceeds 5 minutes.

Grade levels:

Badge Threshold
Captain Bronze ≥ 50% of feedback submitted in batch sessions
Captain Silver ≥ 70%
Captain Gold ≥ 90%

No badge is shown if the reviewer is below 50%.

Minimum sample size

Badges are only computed for reviewers with at least 5 feedback items on a given workload. Reviewers below this threshold are excluded from badge counts in the workload summary to avoid misleading 100% rates from single data points.

Related articles