What is partial feedback?

This guide explains how to submit feedback on specific sections of an LLM response using partial feedback, rather than rating the entire response.

---

What is Partial Feedback?#

Partial feedback allows users to provide targeted feedback on specific portions of an AI response. Instead of rating an entire response as good or bad, users can highlight and rate individual sections.

Use Cases#

• Highlighting inaccuracies: Mark specific sentences that contain factual errors
• Praising good sections: Call out particularly helpful parts of a response
• Granular improvement signals: Help identify which parts of responses need work
• A/B comparison: Compare different sections of the same response

---

How Partial Feedback Works#

Parent-Child Relationship#

Partial feedbacks are organized hierarchically:

Parent Feedback (full response)
├── Child Partial (section 1 - positive)
├── Child Partial (section 2 - negative)
└── Child Partial (section 3 - neutral)

• The parent feedback represents the full AI response
• Child partials are feedback on specific sections within that response
• Multiple partials can be associated with the same parent

Automatic Parent Matching#

When you submit partial feedback, the system automatically finds or creates a parent:

1. Direct reference: If you provide parent_feedback_hashid, that parent is used
2. Creator matching: Finds existing feedback with same creator_unique_id and response content
3. Fingerprint matching: Falls back to matching by coolhand_fingerprint_id
4. Auto-creation: Creates a placeholder parent if no match is found

---

API Reference#

Creating Partial Feedback#

POST /api/v2/llm_request_log_feedbacks
{
  "llm_request_log_feedback": {
    "original_output": "The full AI response text here with multiple sections and content.",
    "focus_section": "multiple sections",
    "focus_range": { "start": 40, "end": 57 },
    "like": false,
    "explanation": "This part contains inaccurate information",
    "creator_unique_id": "user-123",
    "coolhand_fingerprint_id": "fp-xyz"
  }
}

Parameters#

Note: The sentiment parameter takes precedence over like if both are provided. The like parameter is deprecated but maintained for backwards compatibility.

Response#

{
  "id": "abc123",
  "parent_feedback_id": "xyz789",
  "focus_section": "multiple sections",
  "focus_range": { "start": 40, "end": 57 },
  "like": false,
  "sentiment": "dislike",
  "original_output": "The full AI response text here...",
  "explanation": "This part contains inaccurate information",
  "partial_feedback_count": 1,
  "partial_positive_count": 0,
  "partial_negative_count": 1,
  "computed_sentiment_score": -0.5,
  "computed_sentiment_updated_at": "2026-01-31T12:00:00Z",
  "created_at": "2026-01-31T12:00:00Z",
  "updated_at": "2026-01-31T12:00:00Z",
  "created_partial_id": "def456"
}

---

Best Practices#

1. Always Include original_output#

The full response text is required for partial feedback. This enables:

• Automatic parent matching
• Validation that focus_section exists in the output
• Content normalization for deduplication

2. Provide User Identifiers#

Include creator_unique_id and/or coolhand_fingerprint_id for better parent matching:

{
  "llm_request_log_feedback": {
    "original_output": "...",
    "focus_section": "...",
    "creator_unique_id": "user_abc123",
    "coolhand_fingerprint_id": "session_xyz789"
  }
}

3. Use focus_range for Precision#

When the same text appears multiple times, use focus_range to specify exactly which occurrence:

{
  "focus_section": "the",
  "focus_range": { "start": 100, "end": 103 }
}

4. Direct Parent Reference#

If you already have a parent feedback ID, reference it directly:

{
  "llm_request_log_feedback": {
    "parent_feedback_hashid": "xyz789",
    "focus_section": "specific section",
    "like": true
  }
}

---

Aggregating Partial Feedback#

Parent Counts#

Parents automatically track their children's sentiment:

Weighted Sentiment Rollup Formula#

The computed_sentiment_score is calculated using a weighted formula that considers both the parent's sentiment and all partial feedbacks:

computed_sentiment_score = Σ(weight × value) / Σ(weight)

Example Calculations#

Score Interpretation#

Sentiment Summary (Legacy)#

The parent also provides a legacy sentiment summary:

{
  "partial_sentiment_summary": {
    "total": 5,
    "positive": 3,
    "negative": 1,
    "neutral": 1
  }
}

---

Warnings#

The API returns warnings (not errors) for common issues:

focus_section Not Found#

If the focus_section text doesn't exist in original_output:

{
  "warnings": [{
    "message": "focus_section text was not found within original_output. The feedback was saved but the section may not match the intended content.",
    "timestamp": "2026-01-31T12:00:00Z"
  }]
}

Invalid Parent Reference#

If parent_feedback_hashid doesn't exist:

{
  "warnings": [{
    "message": "Invalid parent_feedback_hashid 'invalid_id' - does not exist or does not belong to this client. Feedback saved without parent association.",
    "timestamp": "2026-01-31T12:00:00Z"
  }]
}

Placeholder Parent Created#

When no existing parent is found:

{
  "warnings": [{
    "message": "Created placeholder parent feedback (ID: abc123) for partial feedback association.",
    "timestamp": "2026-01-31T12:00:00Z"
  }]
}

---

Example Workflow#

1. User highlights a section in the UI#

const selectedText = window.getSelection().toString();
const fullResponse = document.querySelector('.ai-response').textContent;

2. Submit partial feedback#

await fetch('/api/v2/llm_request_log_feedbacks', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer YOUR_API_KEY',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    llm_request_log_feedback: {
      original_output: fullResponse,
      focus_section: selectedText,
      like: false,
      explanation: "This part is incorrect",
      creator_unique_id: currentUser.id
    }
  })
});

3. Query parent with all partials#

Fetch a single feedback record by its id, using your private API key. The response always includes the parent's feedback_partials:

GET /api/v2/llm_request_log_feedbacks/{id}
Authorization: Bearer YOUR_PRIVATE_API_KEY

To list or search feedback (newest first, paginated), use the collection endpoint. List results omit original_output/revised_output — fetch a specific record with the call above to get the full text and partials. Filter with Ransack q[...] params; sentiment is stored as an integer code (0=dislike, 1=neutral, 2=like), so q[sentiment_eq]=2 returns liked feedback:

GET /api/v2/llm_request_log_feedbacks?q[sentiment_eq]=2&page=1&per=25
Authorization: Bearer YOUR_PRIVATE_API_KEY

To filter by workload, pass its hashid (the same value returned as each record's workload_id) via q[workload_id_eq]. An unrecognized hashid returns 404 Workload not found rather than an empty list, so a typo is surfaced instead of silently matching nothing:

GET /api/v2/llm_request_log_feedbacks?q[workload_id_eq]=WORKLOAD_HASHID&page=1&per=25
Authorization: Bearer YOUR_PRIVATE_API_KEY

How workload_id gets set#

You don't have to send workload_hashid for feedback to be attributed to a workload. When you submit feedback tied to a request log (or when Coolhand later matches it to one automatically), the feedback inherits the workload of that log's template, so it shows up in workload-level views and in the q[workload_id_eq] filter above without any extra work on your side.

Precedence and edge cases:

• An explicit workload_hashid always wins. Inheritance only fills in a workload you left unset.
• Merged workloads resolve to their destination. If the inherited workload has since been merged into another, the feedback is attributed to the workload it was merged into.
• Cross-client references are never followed. If a workload doesn't belong to your client, the feedback is saved unassigned rather than pointing at another client's workload.
• A rejected workload_hashid stays rejected. If you send a workload_hashid we can't honor, the response carries a warning and the feedback is saved with no workload — we won't quietly substitute a different one.

---

Technical Details#

Data Model#

Partial feedbacks are now stored in a separate FeedbackPartial model:

FeedbackPartial Table | Field | Type | Description | |——-|——|————-| | llm_request_log_feedback_id | bigint | FK to parent feedback | | client_id | bigint | FK to client | | focus_section | text | The highlighted text (required) | | focus_range | jsonb | {start: int, end: int} positions | | sentiment | integer | 0=dislike, 1=neutral, 2=like | | explanation | text | Why this section is good/bad | | creator_unique_id | string | User identifier | | coolhand_fingerprint_id | string | Session/device identifier |

LlmRequestLogFeedback (Parent) | Field | Type | Description | |——-|——|————-| | sentiment | integer | 0=dislike, 1=neutral, 2=like | | computed_sentiment_score | decimal | Weighted score from -1.0 to 1.0 | | computed_sentiment_updated_at | datetime | When score was last calculated |

Scopes#

# Parent feedbacks
LlmRequestLogFeedback.non_partial  # No focus_section (parent/full feedback)

# Sentiment scopes (both models)
feedback.sentiment_like?     # Is sentiment "like"?
feedback.sentiment_dislike?  # Is sentiment "dislike"?
feedback.sentiment_neutral?  # Is sentiment "neutral"?

Methods#

# LlmRequestLogFeedback
feedback.parent?              # Has feedback_partials?
feedback.sentiment_value      # Returns 1.0, 0.0, or -1.0
feedback.feedback_partials    # Collection of FeedbackPartial records

# FeedbackPartial
partial.sentiment_value       # Returns 1.0, 0.0, or -1.0
partial.llm_request_log_feedback  # Parent feedback

Async Rollup#

computed_sentiment_score is kept up to date automatically. A background job recalculates it whenever any of the following events occur:

Timing: The recalculation runs asynchronously via Sidekiq, typically completing within a few seconds of the triggering event. There is no polling interval — each change schedules its own recalculation immediately.

When score is absent: If neither the parent nor any of its partials carry a sentiment value (e.g. a feedback submitted with only revised_output and no rating), computed_sentiment_score will be null.

Idempotency: Submitting the same sentiment value again does not re-trigger recalculation.