# RubyLLM + Coolhand: AI Observability Built for Ruby & Rails

> Most AI observability platforms don't ship a native Ruby SDK. Coolhand does — and it auto-instruments RubyLLM
> out of the box. Here's the honest comparison for Ruby and Rails teams.

## The Ruby gap in AI observability

Langfuse and LangSmith officially maintain SDKs for Python and JavaScript/TypeScript only — Ruby apps are pointed
at a generic OpenTelemetry endpoint, or left to unofficial, community-maintained gems that aren't backed by either
company. Datadog's Ruby APM gem (dd-trace-rb) can send LLM Observability spans, but automatic instrumentation —
the part that means you don't touch application code — is Python, Node, and Java only; Ruby requires manually
wrapping every call. Arize's OpenInference instrumentation covers Python, JavaScript, and Java, with no Ruby
package at all.

Braintrust is the exception worth naming honestly: its official Ruby gem auto-instruments `ruby_llm`, `openai`,
and `anthropic` gem calls out of the box — a genuine native-Ruby story. It's also, like the rest of this list, a
tracing and eval tool that stops there. Coolhand is built the same zero-config way *and* keeps going: diagnosing
what's wrong and opening the fix as a pull request.

## Ruby & Rails compatibility, side by side

| Capability | Langfuse | LangSmith | Datadog | Arize | Braintrust | Coolhand |
|---|---|---|---|---|---|---|
| Native Ruby SDK | No — Python/JS only | No — Python/JS only | Yes — dd-trace-rb (APM gem) | No Ruby package | Yes — official beta gem | Yes — open source coolhand gem |
| RubyLLM support | No official path | No official path | Manual wrapping only | No official path | Yes — auto-instruments ruby_llm ≥1.8.0 | Yes — automatic, no version pin |
| Auto-instrumentation for Ruby | — | — | No — Python/Node/Java only | — | Yes, for its supported gem list | Yes — any HTTP-based LLM client |
| Diagnoses issues & opens a fix PR | No | Partial — Engine beta, LangChain/LangGraph only | No | Partial — Alyx, review-and-accept only | No | Yes — continuous, framework-agnostic |
| Passive human feedback collection | Manual annotation queues | Manual annotation queues | No | Manual annotation queues | Manual eval datasets | Yes — captured from your app's existing UI |

## Why RubyLLM + Coolhand

**RubyLLM + Coolhand is the power coupling for a Ruby AI app.** RubyLLM gives you the instrumentation to solve
the hard problems — calling any provider through one consistent interface, streaming, tool calls, structured
output. Coolhand makes sure it all stays self-improving even when you aren't looking: the open-source `coolhand`
gem intercepts at the `Net::HTTP` layer, so it captures RubyLLM — and OpenAI, Anthropic, or Gemini clients you
call directly — automatically, with no per-library version pin to keep up with.

From there Coolhand does what the rest of this list doesn't: it diagnoses production issues against your actual
code, proposes the fix as a pull request, helps you collect real user feedback, and reports the ROI — the loop
most Ruby teams end up building by hand on top of a tracing dashboard.

```ruby
# Gemfile
gem 'ruby_llm'
gem 'coolhand'

# config/initializers/coolhand.rb
Coolhand.configure { |c| c.api_key = ENV["COOLHAND_API_KEY"] }
```

## Frequently asked questions

**Does Langfuse have a native Ruby SDK?**
No. Langfuse officially maintains SDKs for Python and JavaScript/TypeScript only; its docs point Ruby apps at a
generic OpenTelemetry endpoint instead. Unofficial community gems exist but aren't Langfuse-maintained.

**Does LangSmith support Ruby?**
Not officially. LangChain's own langsmith-sdk repository ships Python and JavaScript clients only. An unofficial
community gem exists on RubyGems, but it isn't maintained by LangChain.

**What about Datadog or Braintrust — don't they support Ruby?**
Partially. Datadog's official dd-trace-rb gem can send LLM Observability spans from Ruby, but automatic
instrumentation of LLM calls (no code changes needed) is Python/Node/Java only — Ruby requires manually wrapping
each call. Braintrust's official Ruby gem is further along and does auto-instrument ruby_llm, openai, and
anthropic gem calls — a real native-Ruby option, though it stops at tracing and evals the same way the rest of
this list does.

**Can I use Coolhand with RubyLLM?**
Yes, automatically. The open-source coolhand gem intercepts calls at the Net::HTTP layer, so it captures RubyLLM's
requests (and any other LLM client) without version-pinned per-library patches or manual wrapping — add the gem
and set your API key.

**Does Coolhand replace RubyLLM?**
No. RubyLLM is the client library your Rails app uses to call OpenAI, Anthropic, Gemini, and other providers —
Coolhand Labs uses RubyLLM internally for these same kinds of inference calls in our own product. Coolhand sits
alongside RubyLLM as the observability and feedback layer: it watches those calls, diagnoses problems, and
proposes fixes as pull requests.

**Doesn't RubyLLM already do observability?**
Sort of — RubyLLM emits ActiveSupport::Notifications events (chat.ruby_llm, request.ruby_llm, and more) with token
usage and provider metadata, so you can subscribe and log calls yourself. That's a genuinely good way to
bootstrap. What it doesn't give you is a loop: turning those logs into a diagnosed issue, a proposed fix, and a
read on whether real users are actually happier with the output. For that — a continuous, self-improving loop
driven by human feedback — Coolhand is the only full solution on the market, in Ruby or any other language.

---

Source: [coolhandlabs.com/ruby-rails-ai-observability](https://coolhandlabs.com/ruby-rails-ai-observability)