---
title: "OpenAI API best practices"
answer: "OpenAI renamed the system-instruction role to `developer` for its o-series reasoning models and silently maps `system` to it on newer models. Coolhand captures both. Send a system or developer instruction on every request so your prompts group into templates instead of landing on Unmatched."
category: "Provider guides"
tags: ["openai","providers","prompts"]
published: 2026-08-30T13:57:51Z
updated: 2026-08-30T13:57:51Z
canonical_url: https://coolhandlabs.com/help/openai-api-best-practices
index_url: https://coolhandlabs.com/help.md
---

# OpenAI API best practices

OpenAI renamed the system-instruction role to `developer` for its o-series reasoning models and silently maps `system` to it on newer models. Coolhand captures both. Send a system or developer instruction on every request so your prompts group into templates instead of landing on Unmatched.

---

## Role deprecation: `system` → `developer`

OpenAI renamed the system-instruction role for its o-series reasoning models:

- **`system`** — original role, works on all models. On o1-2024-12-17 and later o-series models, OpenAI maps it to `developer` silently. Note: o1-preview and o1-mini support neither role.
- **`developer`** — preferred for o1-2024-12-17 and later. Accepted on GPT-4o too, but `system` is still more widely compatible across integrations.

Coolhand captures both as the **System Prompt** field. No action needed for existing integrations — both roles land in the same place.

---

## Structured outputs

OpenAI offers two JSON output modes:

- **Structured Outputs** (`response_format: { type: "json_schema", json_schema: ... }`) — strict schema enforcement with no extra fields or omissions. Requires `gpt-4o-2024-08-06`, `gpt-4o-mini`, or later models.
- **JSON Mode** (`response_format: { type: "json_object" }`) — guarantees valid JSON but does not enforce a schema. Compatible with older models including `gpt-3.5-turbo-1106` and `gpt-4-*`, but the model may produce any JSON structure.

Use Structured Outputs when schema adherence matters. JSON Mode is a fallback for older deployments that don't support schema constraints.

Docs: https://platform.openai.com/docs/guides/structured-outputs

---

## Caching

Automatic — no explicit API required. OpenAI caches the longest stable prefix of your prompt.

- **Minimum prefix:** 1,024 tokens
- **TTL:** typically 5–10 minutes of inactivity; up to 1 hour maximum (treat as short-lived — a prompt inactive for 10 minutes may already be evicted)
- **Invalidation:** any change to the cached prefix breaks the cache
- **Scope:** per organization and per model version — different organizations or model versions do not share a cache

To maximize hit rate: put your entire system prompt first, followed by any stable context, and keep all variable content at the end of the prompt (or in the user turn). A single token change anywhere before the variable content resets the cache prefix.

Coolhand captures cache hits as **Cached Input Tokens**. Cache creation tokens are not reported by OpenAI (no separate creation event).

---

## Batch processing

The OpenAI Batch API processes up to 50,000 requests from a JSONL file asynchronously, returning results within 24 hours at **50% of standard per-token pricing**. Suitable for any workload that doesn't require real-time responses: evals, document processing, bulk classification.

- Submit a JSONL file via the Files API, then create a batch job referencing the file ID
- Poll the batch job status; download the output JSONL when complete
- Each line in the input/output maps via a `custom_id` you supply

Docs: https://platform.openai.com/docs/guides/batch

---

## Non-standard token fields

| Field | When present |
|-------|-------------|
| **Reasoning Tokens** | o-series models only — internal chain-of-thought tokens, billed as output |
| **Audio Input Tokens** | Multimodal requests that include audio input |

---

## Latency

Three-tier fallback in order of preference:
1. **Assistants API runs** — wall-clock time derived from `completed_at - started_at` timestamps in the run object. Note: the Assistants API is deprecated and [scheduled for sunset on 2026-08-26](https://platform.openai.com/docs/deprecations); the Responses API is now the recommended path for stateful workflows.
2. **Chat Completions** — `openai-processing-ms` response header (server-side processing time, does not include network transit)
3. **Fallback** — collector-provided `duration_ms` when the header is absent (wall-clock)

Most requests use tier 2.


---

Related:
- [Google Gemini best practices](https://coolhandlabs.com/help/google-gemini-best-practices.md)
- [Azure OpenAI best practices](https://coolhandlabs.com/help/azure-openai-best-practices.md)
- [Anthropic API best practices](https://coolhandlabs.com/help/anthropic-api-best-practices.md)
- [Google Vertex AI best practices](https://coolhandlabs.com/help/google-vertex-ai-best-practices.md)

[← All help articles](https://coolhandlabs.com/help.md) · [Provider guides](https://coolhandlabs.com/help/category/providers.md)