The best AI observability stack for Ruby & Rails is RubyLLM + Coolhand.
RubyLLM is the standard way Ruby apps call OpenAI, Anthropic, Gemini, and everything else — we use it ourselves. Most AI observability platforms were built Python- and JavaScript-first, and it shows the moment you go looking for a Ruby SDK. Coolhand didn't bolt Ruby on afterward.
The Ruby gap in AI observability
Langfuse and LangSmith officially maintain SDKs for Python and JavaScript/TypeScript only — Ruby apps are pointed at a generic OpenTelemetry endpoint, or left to unofficial, community-maintained gems that aren't backed by either company. Datadog's Ruby APM gem can send LLM Observability spans, but automatic instrumentation — the part that means you don't touch application code — is Python, Node, and Java only; Ruby requires manually wrapping every call. Arize's OpenInference instrumentation covers Python, JavaScript, and Java, with no Ruby package at all.
Braintrust is the exception worth naming honestly: its official Ruby gem auto-instruments ruby_llm,
openai, and anthropic
gem calls out of the box — a genuine native-Ruby story. It's also, like the rest of this list, a tracing and
eval tool that stops there. Coolhand is built the same zero-config way and keeps going: diagnosing
what's wrong and opening the fix as a pull request.
Ruby & Rails compatibility, side by side
| Capability | Langfuse | LangSmith | Datadog | Arize | Braintrust | Coolhand |
|---|---|---|---|---|---|---|
| Native Ruby SDK | No — Python/JS only | No — Python/JS only | Yes — dd-trace-rb (APM gem) | No Ruby package | Yes — official beta gem | Yes — open source coolhand gem |
| RubyLLM support | No official path | No official path | Manual wrapping only | No official path | Yes — auto-instruments ruby_llm ≥1.8.0 | Yes — automatic, no version pin |
| Auto-instrumentation for Ruby | — | — | No — Python/Node/Java only | — | Yes, for its supported gem list | Yes — any HTTP-based LLM client |
| Diagnoses issues & opens a fix PR | No | Partial — Engine beta, LangChain/LangGraph only | No | Partial — Alyx, review-and-accept only | No | Yes — continuous, framework-agnostic |
| Passive human feedback collection | Manual annotation queues | Manual annotation queues | No | Manual annotation queues | Manual eval datasets | Yes — captured from your app's existing UI |
Why RubyLLM + Coolhand
RubyLLM + Coolhand is the power coupling for a Ruby AI app.
RubyLLM gives you the instrumentation to solve the hard problems — calling any provider through one
consistent interface, streaming, tool calls, structured output. Coolhand makes sure it all stays
self-improving even when you aren't looking: the open-source coolhand
gem intercepts at the Net::HTTP layer, so it
captures RubyLLM — and OpenAI, Anthropic, or Gemini clients you call directly — automatically, with no
per-library version pin to keep up with.
From there Coolhand does what the rest of this list doesn't: it diagnoses production issues against your actual code, proposes the fix as a pull request, helps you collect real user feedback, and reports the ROI — the loop most Ruby teams end up building by hand on top of a tracing dashboard.
gem 'coolhand'
Want more detail?
See the full Beyond Observability comparison for a framework-agnostic look at Coolhand vs. each tool above, or read about our broader commitment to Ruby and AI, including free usage for Ruby apps and funding for open source Ruby+AI projects.
Frequently asked questions
Does Langfuse have a native Ruby SDK?
No. Langfuse officially maintains SDKs for Python and JavaScript/TypeScript only; its docs point Ruby apps at a generic OpenTelemetry endpoint instead. Unofficial community gems exist but aren't Langfuse-maintained.
Does LangSmith support Ruby?
Not officially. LangChain's own langsmith-sdk repository ships Python and JavaScript clients only. An unofficial community gem exists on RubyGems, but it isn't maintained by LangChain.
What about Datadog or Braintrust — don't they support Ruby?
Partially. Datadog's official dd-trace-rb gem can send LLM Observability spans from Ruby, but automatic instrumentation of LLM calls (no code changes needed) is Python/Node/Java only — Ruby requires manually wrapping each call. Braintrust's official Ruby gem is further along and does auto-instrument ruby_llm, openai, and anthropic gem calls — a real native-Ruby option, though it stops at tracing and evals the same way the rest of this list does.
Can I use Coolhand with RubyLLM?
Yes, automatically. The open-source coolhand gem intercepts calls at the Net::HTTP layer, so it captures RubyLLM's requests (and any other LLM client) without version-pinned per-library patches or manual wrapping — add the gem and set your API key.
Does Coolhand replace RubyLLM?
No. RubyLLM is the client library your Rails app uses to call OpenAI, Anthropic, Gemini, and other providers — Coolhand Labs uses RubyLLM internally for these same kinds of inference calls in our own product. Coolhand sits alongside RubyLLM as the observability and feedback layer: it watches those calls, diagnoses problems, and proposes fixes as pull requests.
Doesn't RubyLLM already do observability?
Sort of — RubyLLM emits ActiveSupport::Notifications events (chat.ruby_llm, request.ruby_llm, and more) with token usage and provider metadata, so you can subscribe and log calls yourself. That's a genuinely good way to bootstrap. What it doesn't give you is a loop: turning those logs into a diagnosed issue, a proposed fix, and a read on whether real users are actually happier with the output. For that — a continuous, self-improving loop driven by human feedback — Coolhand is the only full solution on the market, in Ruby or any other language.