Deployment name ≠ model name
This is the most common source of confusion in Azure logs. When you deploy a model in Azure, you choose a deployment name (e.g., my-gpt4o-prod) that may differ from the underlying model version (gpt-4o-2024-11-20). Coolhand logs the deployment name, not the model version.
Practical consequences:
- Two deployments running the same base model appear as different models in Coolhand filters
- Comparing per-model costs across Azure and direct OpenAI requires aligned naming
Best practice: Name deployments after their model version (e.g., gpt-4o-2024-11-20-eastus) so logs are unambiguous.
API version pinning
For the legacy dated-API surface, requests require an explicit api-version parameter (e.g., 2024-10-21). The newer v1 GA API endpoint does not require this. Regardless of which surface you use, older API versions return fewer token-breakdown fields — if you see blank token counts, check whether your API version supports those fields.
Structured outputs
Supported on compatible deployments (GPT-4o and o-series). Same response_format: { type: "json_schema" } parameter as OpenAI directly.
Docs: https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/structured-outputs
Caching
Automatic — no explicit API required. Azure supports two retention tiers:
- Minimum prefix: 1,024 tokens
- In-memory retention: typically 5–10 minutes of inactivity; no extra charge
- Extended retention (
prompt_cache_retention): up to 24 hours on supported models; pricing is the same as in-memory retention — no separate storage charge - Invalidation: any change to the cached prefix; also varies by Azure region and deployment quota
- Scope: per deployment; different deployments do not share a cache
Put your entire system prompt first and keep all variable content at the end of the prompt to maximize cache hit rate. The deployment name must also be consistent — the same prompt sent to two different deployments does not benefit from a shared cache.
Coolhand captures cache hits as Cached Input Tokens. Cache creation tokens are not reported.
Batch processing
Azure OpenAI Batch processes requests from a JSONL file asynchronously within 24 hours at 50% of standard per-token pricing, following the same format as OpenAI Batch.
- Upload a JSONL file, create a batch job referencing it, poll for completion
- Results are written to an output file you download when the job finishes
- Supported on GPT-4o and o-series deployments (availability varies by region)
Docs: https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/batch
Non-standard token fields
| Field | When present |
|---|---|
| Audio Input Tokens | Multimodal deployments with audio input |
Latency
Derived from the collector-provided duration_ms (wall-clock including network transit to the Azure region); blank if the collector does not supply it. Unlike direct OpenAI, Azure does not return an openai-processing-ms header.