---
title: "Google Gemini best practices"
answer: "The Gemini API carries system instructions in a top-level `system_instruction` field rather than a system-role message. Coolhand reads that field first and falls back to the contents array, so using `system_instruction` is the dependable path to having your prompt captured and matched to a template."
category: "Provider guides"
tags: ["gemini","google","providers","prompts"]
published: 2026-08-30T13:57:51Z
updated: 2026-08-30T13:57:51Z
canonical_url: https://coolhandlabs.com/help/google-gemini-best-practices
index_url: https://coolhandlabs.com/help.md
---

# Google Gemini best practices

The Gemini API carries system instructions in a top-level `system_instruction` field rather than a system-role message. Coolhand reads that field first and falls back to the contents array, so using `system_instruction` is the dependable path to having your prompt captured and matched to a template.

---

## System instructions are a top-level field

Like Vertex AI, the Gemini API uses a top-level `system_instruction` field (or `systemInstruction` in camelCase) rather than a `system` role in the messages array. Coolhand reads from there first, then falls back to checking the contents array for system/developer role messages. Use `system_instruction` to ensure reliable capture.

---

## Structured outputs

Supported via the `response_schema` parameter in `generation_config`. Constrains the response to a JSON schema you define.

Docs: https://ai.google.dev/gemini-api/docs/structured-output

---

## Thinking budgets

Gemini's "thinking" (extended internal reasoning before responding) is configured differently depending on the model generation — and getting it wrong is not a soft degradation, it's a hard `HTTP 400` at request time.

### Gemini 2.5 models: numeric `thinkingBudget`

Set via `generation_config.thinking_config.thinking_budget` (`generationConfig.thinkingConfig.thinkingBudget` in camelCase/REST) — an integer token count. The valid range is per-model, not one constant across the family:

| Model | Valid range | Default | Disable | Dynamic |
|---|---|---|---|---|
| `gemini-2.5-pro` | 128–32768 | Thinks by default; cannot be disabled | — | `thinking_budget: -1` |
| `gemini-2.5-flash` | 0–24576 (0 disables) | Dynamic (auto) | `thinking_budget: 0` | `thinking_budget: -1` |
| `gemini-2.5-flash-lite` | 512–24576 (or 0 to disable) | Off (does not think) | `thinking_budget: 0` | `thinking_budget: -1` |

A budget outside the model's valid range is rejected outright, not clamped:

```
HTTP 400: The thinking budget 32000 is invalid. Please choose a value between 512 and 24576.
```

The most common way to trip this: reusing one hardcoded/shared budget value across model tiers. 32000 is valid for `gemini-2.5-pro` (max 32768) but exceeds both Flash tiers' 24576 cap — always source the budget from configuration scoped to the specific model being called, not a constant shared across your app's Gemini integrations.

```json
{
  "generationConfig": {
    "thinkingConfig": { "thinkingBudget": 8192 }
  }
}
```

Docs: https://ai.google.dev/gemini-api/docs/generate-content/thinking

### Gemini 3.x models: `thinkingLevel` replaces the numeric budget

Gemini 3 models use `thinking_config.thinking_level` (`thinkingConfig.thinkingLevel`) instead — an enum (`minimal`, `low`, `medium`, `high`; exact supported set varies by model) rather than a raw token count, so there's no per-model numeric range to look up.

```json
{
  "generationConfig": {
    "thinkingConfig": { "thinkingLevel": "high" }
  }
}
```

**`thinkingBudget` and `thinkingLevel` cannot both be set on a request to a Gemini 3.x model** — the request errors if you send both. A caller migrating from a 2.5 model to a 3.x model needs its thinking configuration to change shape, not just its value — swapping the model string alone while still sending `thinkingBudget` will fail.

Docs: https://ai.google.dev/gemini-api/docs/thinking

### Gemini 2.5 is winding down

Last checked 2026-08-16: the Gemini API's own [deprecations page](https://ai.google.dev/gemini-api/docs/deprecations) listed no shutdown date for `gemini-2.5-pro`, `gemini-2.5-flash`, or `gemini-2.5-flash-lite`. Do not read that as a guarantee of longevity — retirement timing can differ between the Gemini API and Vertex AI for the same model (see [Google Vertex AI Best Practices](https://coolhandlabs.com/help/google-vertex-ai-best-practices.md)), and there are developer-forum reports of 2.5 models returning 404s ahead of any published shutdown date. Check the live deprecations page before building a new integration against a 2.5 model, and prefer a 3.x model (with `thinkingLevel`) for new work.

---

## Caching

Gemini supports two caching modes:

### Implicit caching (Gemini 2.5 models)

Gemini 2.5 Flash and 2.5 Pro cache automatically — no setup required, no storage cost. The model detects repeated prefixes and applies a discount on cache hits.

- **Minimum size:** 2,048 tokens (both 2.5 Flash and 2.5 Pro)
- **No explicit API call required**
- **No storage billing** — you only pay the reduced inference rate on hits

### Explicit context caching

For older models (or when you need deterministic cache control), create a named cache resource and reference it in requests:

```
# Create a cache resource
POST https://generativelanguage.googleapis.com/v1beta/cachedContents
{ "model": "models/gemini-2.0-flash", "contents": [...], "ttl": "3600s" }

# Use it in a request
{ "model": "models/gemini-2.0-flash", "cachedContent": "cachedContents/<id>", ... }
```

- **Minimum size:** 2,048 tokens for Gemini 2-family models; 4,096 tokens for Gemini 3-family models
- **Default TTL:** 1 hour; configurable (consult current API docs for the maximum — published limits have changed across versions)
- **Invalidation:** TTL expiry, or explicit deletion via `DELETE cachedContents/<id>`
- **Billing:** two separate charges apply — a per-token **storage** fee, billed hourly for as long as the cache resource exists (regardless of whether it's used), plus the usual reduced per-token **read** rate on any request that hits it. This is unlike Anthropic's caching (see [Anthropic Best Practices](https://coolhandlabs.com/help/anthropic-api-best-practices.md)), which only ever charges per-token on write/read and never for idle storage time. Forgetting to delete an unused cache resource keeps accruing the storage charge until its TTL expires.

Coolhand's [inference API pricing catalog](https://coolhandlabs.com/inference-apis) publishes the storage rate ($ per 1M cached tokens per hour) for models where it's confirmed, alongside the cache-read rate.

**Note:** Coolhand does not capture Gemini cache *usage* metrics — Cached Input Tokens will always be blank for Gemini logs, regardless of whether a cached context was used. The savings are real but won't be reflected in Coolhand's token fields. (The catalog's storage/read *rates* above are tracked independently of per-request usage and aren't affected by this gap.)

---

## Batch processing

The Gemini API has a native **Batch Mode** for `generateContent` requests — submit a JSONL file, get results asynchronously within 24 hours at approximately **50% of standard pricing**.

- Upload a JSONL input file via the Files API, then create a batch job
- Poll for completion; download the JSONL output file when done
- Each request maps via a `key` field you supply

Docs: https://ai.google.dev/gemini-api/docs/batch-api

For large-scale workloads that need Cloud Storage integration, **Vertex AI batch prediction** is also an option — see [Google Vertex AI Best Practices](https://coolhandlabs.com/help/google-vertex-ai-best-practices.md).

---

## Latency

**Collector-derived only.** The Gemini API does not return timing information natively. Coolhand reads a `duration_ms` (or `duration` in seconds) field that your collector or proxy must provide. Latency will be blank if your collector does not capture wall-clock time.


---

Related:
- [Google Vertex AI best practices](https://coolhandlabs.com/help/google-vertex-ai-best-practices.md)
- [OpenAI API best practices](https://coolhandlabs.com/help/openai-api-best-practices.md)
- [Anthropic API best practices](https://coolhandlabs.com/help/anthropic-api-best-practices.md)
- [AWS Bedrock best practices](https://coolhandlabs.com/help/aws-bedrock-best-practices.md)

[← All help articles](https://coolhandlabs.com/help.md) · [Provider guides](https://coolhandlabs.com/help/category/providers.md)