System instructions are a top-level field
The Vertex AI preferred format uses a top-level system_instruction field (or systemInstruction in camelCase), outside the contents array. Coolhand reads from there first. As a fallback, Coolhand also checks the contents array for messages with a system or developer role — so role-based system messages may be captured, but system_instruction is the reliable path.
Structured outputs
Supported via the response_schema parameter (part of generation_config). Enforces a JSON schema on the response.
Docs: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/control-generated-output
Thinking budgets
Vertex-hosted Gemini models use the same thinking_config.thinking_budget (2.5 models) / thinking_config.thinking_level (3.x models) mechanics as the direct Gemini API — see Google Gemini Best Practices for the per-model valid ranges, the two parameters' incompatibility on 3.x models, and why a shared/hardcoded budget value is the most common way to trip the HTTP 400 "thinking budget … is invalid" error.
Model retirement timing can differ between the two surfaces for the same model — Vertex AI's own model lifecycle documentation has listed retirement dates for some Gemini 2.5 models where the Gemini API's deprecations page had none announced. Check Vertex's model lifecycle page directly rather than assuming the two surfaces retire a model in lockstep.
Caching
Vertex AI supports the same implicit and explicit context caching as the direct Gemini API — see Google Gemini Best Practices for the full mechanics, including the two-part billing (a per-token storage fee charged hourly, plus the discounted per-token rate on cache hits) for explicit caching. Coolhand's inference API pricing catalog tracks both rates for Vertex-hosted Gemini models where confirmed.
Vertex returns a cachedContentTokenCount field, but Coolhand's ingestor does not currently extract it. Cached Input Tokens and Cache Creation Tokens are always blank for Vertex logs even when caching is active — this affects only per-request usage capture, not the catalog's published rates above.
Batch processing
Vertex AI batch prediction submits a JSONL input file to Cloud Storage, processes the requests asynchronously, and writes results to a Cloud Storage output path. There is no fixed completion window — jobs run to completion based on available capacity.
- No per-request overhead; well-suited for large one-off processing jobs
- Input and output stay in your GCP project (Cloud Storage buckets you control)
- Typical completion ~24 hours; maximum queue time is 72 hours
- Supports most Gemini models available on Vertex — tuned models must use a tuned endpoint MODEL_PATH rather than the global endpoint
Docs: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/batch-prediction-gemini
Latency
Not available. Vertex AI does not include timing information in API responses, and Coolhand does not read a collector duration_ms for Vertex text logs. Latency will always be blank.