Deployment name ≠ model name
This is the most common source of confusion in Azure logs. When you deploy a model in Azure, you choose a deployment name (e.g., my-gpt4o-prod) that may differ from the underlying model version (gpt-4o-2024-11-20). Coolhand logs the deployment name, not the model version.
Practical consequences:
- Two deployments running the same base model appear as different models in Coolhand filters
- Comparing per-model costs across Azure and direct OpenAI requires aligned naming
Best practice: Name deployments after their model version (e.g., gpt-4o-2024-11-20-eastus) so logs are unambiguous.
API version pinning
For the legacy dated-API surface, requests require an explicit api-version parameter (e.g., 2024-10-21). The newer v1 GA API endpoint does not require this. Regardless of which surface you use, older API versions return fewer token-breakdown fields — if you see blank token counts, check whether your API version supports those fields.
Structured outputs
Supported on compatible deployments (GPT-4o and o-series). Same response_format: { type: "json_schema" } parameter as OpenAI directly.
Docs: https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/structured-outputs
Caching
Automatic — no explicit API required. Azure supports two retention tiers:
- Minimum prefix: 1,024 tokens
- In-memory retention: typically 5–10 minutes of inactivity; no extra charge
- Extended retention (
prompt_cache_retention): up to 24 hours on supported models; pricing is the same as in-memory retention — no separate storage charge - Invalidation: any change to the cached prefix; also varies by Azure region and deployment quota
- Scope: per deployment; different deployments do not share a cache
Put your entire system prompt first and keep all variable content at the end of the prompt to maximize cache hit rate. The deployment name must also be consistent — the same prompt sent to two different deployments does not benefit from a shared cache.
Coolhand captures cache hits as Cached Input Tokens, and on GPT-5.6 and later also captures cache writes as Cache Creation Tokens. Both are reported as subsets of the request's input token count rather than as separate additive counts, so Coolhand subtracts them when working out how much of the prompt was billed at the standard input rate. Earlier models leave Cache Creation Tokens blank.
Cache writes are billed at 1.25× the uncached input rate on GPT-5.6 and later; earlier model families do not charge to write to the cache. Two cases leave Cache Creation Tokens blank even on GPT-5.6 and later: Provisioned Throughput managed (PTU-M) deployments still cache but do not report a cache-write count, and requests sent with prompt_cache_options.mode set to explicit and no breakpoints do not use the cache at all.
Batch processing
Azure OpenAI Batch processes requests from a JSONL file asynchronously within 24 hours at 50% of standard per-token pricing, following the same format as OpenAI Batch.
- Upload a JSONL file, create a batch job referencing it, poll for completion
- Results are written to an output file you download when the job finishes
- Supported on GPT-4o and o-series deployments (availability varies by region)
Docs: https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/batch
Non-standard token fields
| Field | When present |
|---|---|
| Audio Input Tokens | Multimodal deployments with audio input |
Latency
Derived from the collector-provided duration_ms (wall-clock including network transit to the Azure region); blank if the collector does not supply it. Unlike direct OpenAI, Azure does not return an openai-processing-ms header.