Google Gemini best practices

---

System instructions are a top-level field#

Like Vertex AI, the Gemini API uses a top-level system_instruction field (or systemInstruction in camelCase) rather than a system role in the messages array. Coolhand reads from there first, then falls back to checking the contents array for system/developer role messages. Use system_instruction to ensure reliable capture.

---

Structured outputs#

Supported via the response_schema parameter in generation_config. Constrains the response to a JSON schema you define.

Docs: https://ai.google.dev/gemini-api/docs/structured-output

---

Thinking budgets#

Gemini's "thinking" (extended internal reasoning before responding) is configured differently depending on the model generation — and getting it wrong is not a soft degradation, it's a hard HTTP 400 at request time.

Gemini 2.5 models: numeric thinkingBudget#

Set via generation_config.thinking_config.thinking_budget (generationConfig.thinkingConfig.thinkingBudget in camelCase/REST) — an integer token count. The valid range is per-model, not one constant across the family:

A budget outside the model's valid range is rejected outright, not clamped:

HTTP 400: The thinking budget 32000 is invalid. Please choose a value between 512 and 24576.

The most common way to trip this: reusing one hardcoded/shared budget value across model tiers. 32000 is valid for gemini-2.5-pro (max 32768) but exceeds both Flash tiers' 24576 cap — always source the budget from configuration scoped to the specific model being called, not a constant shared across your app's Gemini integrations.

{
  "generationConfig": {
    "thinkingConfig": { "thinkingBudget": 8192 }
  }
}

Docs: https://ai.google.dev/gemini-api/docs/generate-content/thinking

Gemini 3.x models: thinkingLevel replaces the numeric budget#

Gemini 3 models use thinking_config.thinking_level (thinkingConfig.thinkingLevel) instead — an enum (minimal, low, medium, high; exact supported set varies by model) rather than a raw token count, so there's no per-model numeric range to look up.

{
  "generationConfig": {
    "thinkingConfig": { "thinkingLevel": "high" }
  }
}

thinkingBudget and thinkingLevel cannot both be set on a request to a Gemini 3.x model — the request errors if you send both. A caller migrating from a 2.5 model to a 3.x model needs its thinking configuration to change shape, not just its value — swapping the model string alone while still sending thinkingBudget will fail.

Docs: https://ai.google.dev/gemini-api/docs/thinking

Gemini 2.5 is winding down#

Last checked 2026-08-16: the Gemini API's own deprecations page listed no shutdown date for gemini-2.5-pro, gemini-2.5-flash, or gemini-2.5-flash-lite. Do not read that as a guarantee of longevity — retirement timing can differ between the Gemini API and Vertex AI for the same model (see Google Vertex AI Best Practices), and there are developer-forum reports of 2.5 models returning 404s ahead of any published shutdown date. Check the live deprecations page before building a new integration against a 2.5 model, and prefer a 3.x model (with thinkingLevel) for new work.

---

Caching#

Gemini supports two caching modes:

Implicit caching (Gemini 2.5 models)#

Gemini 2.5 Flash and 2.5 Pro cache automatically — no setup required, no storage cost. The model detects repeated prefixes and applies a discount on cache hits.

• Minimum size: 2,048 tokens (both 2.5 Flash and 2.5 Pro)
• No explicit API call required
• No storage billing — you only pay the reduced inference rate on hits

Explicit context caching#

For older models (or when you need deterministic cache control), create a named cache resource and reference it in requests:

# Create a cache resource
POST https://generativelanguage.googleapis.com/v1beta/cachedContents
{ "model": "models/gemini-2.0-flash", "contents": [...], "ttl": "3600s" }

# Use it in a request
{ "model": "models/gemini-2.0-flash", "cachedContent": "cachedContents/<id>", ... }

• Minimum size: 2,048 tokens for Gemini 2-family models; 4,096 tokens for Gemini 3-family models
• Default TTL: 1 hour; configurable (consult current API docs for the maximum — published limits have changed across versions)
• Invalidation: TTL expiry, or explicit deletion via DELETE cachedContents/<id>
• Billing: two separate charges apply — a per-token storage fee, billed hourly for as long as the cache resource exists (regardless of whether it's used), plus the usual reduced per-token read rate on any request that hits it. This is unlike Anthropic's caching (see Anthropic Best Practices), which only ever charges per-token on write/read and never for idle storage time. Forgetting to delete an unused cache resource keeps accruing the storage charge until its TTL expires.

Coolhand's inference API pricing catalog publishes the storage rate ($ per 1M cached tokens per hour) for models where it's confirmed, alongside the cache-read rate.

Note: Coolhand does not capture Gemini cache usage metrics — Cached Input Tokens will always be blank for Gemini logs, regardless of whether a cached context was used. The savings are real but won't be reflected in Coolhand's token fields. (The catalog's storage/read rates above are tracked independently of per-request usage and aren't affected by this gap.)

---

Batch processing#

The Gemini API has a native Batch Mode for generateContent requests — submit a JSONL file, get results asynchronously within 24 hours at approximately 50% of standard pricing.

• Upload a JSONL input file via the Files API, then create a batch job
• Poll for completion; download the JSONL output file when done
• Each request maps via a key field you supply

Docs: https://ai.google.dev/gemini-api/docs/batch-api

For large-scale workloads that need Cloud Storage integration, Vertex AI batch prediction is also an option — see Google Vertex AI Best Practices.

---

Latency#

Collector-derived only. The Gemini API does not return timing information natively. Coolhand reads a duration_ms (or duration in seconds) field that your collector or proxy must provide. Latency will be blank if your collector does not capture wall-clock time.