---
title: "Google Vertex AI best practices"
answer: "Vertex AI's preferred request format puts system instructions in a top-level `system_instruction` field, outside the `contents` array. Coolhand reads there first and falls back to system or developer roles inside `contents`, so role-based instructions may still be captured — but the top-level field is the reliable one."
category: "Provider guides"
tags: ["vertex","google","providers","prompts"]
published: 2026-08-30T13:57:52Z
updated: 2026-08-30T13:57:52Z
canonical_url: https://coolhandlabs.com/help/google-vertex-ai-best-practices
index_url: https://coolhandlabs.com/help.md
---

# Google Vertex AI best practices

Vertex AI's preferred request format puts system instructions in a top-level `system_instruction` field, outside the `contents` array. Coolhand reads there first and falls back to system or developer roles inside `contents`, so role-based instructions may still be captured — but the top-level field is the reliable one.

---

## System instructions are a top-level field

The Vertex AI preferred format uses a top-level `system_instruction` field (or `systemInstruction` in camelCase), outside the `contents` array. Coolhand reads from there first. As a fallback, Coolhand also checks the `contents` array for messages with a system or developer role — so role-based system messages may be captured, but `system_instruction` is the reliable path.

---

## Structured outputs

Supported via the `response_schema` parameter (part of `generation_config`). Enforces a JSON schema on the response.

Docs: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/control-generated-output

---

## Thinking budgets

Vertex-hosted Gemini models use the same `thinking_config.thinking_budget` (2.5 models) / `thinking_config.thinking_level` (3.x models) mechanics as the direct Gemini API — see [Google Gemini Best Practices](https://coolhandlabs.com/help/google-gemini-best-practices.md) for the per-model valid ranges, the two parameters' incompatibility on 3.x models, and why a shared/hardcoded budget value is the most common way to trip the `HTTP 400` "thinking budget ... is invalid" error.

Model retirement timing can differ between the two surfaces for the same model — Vertex AI's own model lifecycle documentation has listed retirement dates for some Gemini 2.5 models where the Gemini API's deprecations page had none announced. Check Vertex's model lifecycle page directly rather than assuming the two surfaces retire a model in lockstep.

---

## Caching

Vertex AI supports the same implicit and explicit context caching as the direct Gemini API — see [Google Gemini Best Practices](https://coolhandlabs.com/help/google-gemini-best-practices.md) for the full mechanics, including the two-part billing (a per-token storage fee charged hourly, plus the discounted per-token rate on cache hits) for explicit caching. Coolhand's [inference API pricing catalog](https://coolhandlabs.com/inference-apis) tracks both rates for Vertex-hosted Gemini models where confirmed.

Vertex returns a `cachedContentTokenCount` field, but Coolhand's ingestor does not currently extract it. **Cached Input Tokens** and **Cache Creation Tokens** are always blank for Vertex logs even when caching is active — this affects only per-request usage capture, not the catalog's published rates above.

---

## Batch processing

Vertex AI batch prediction submits a JSONL input file to Cloud Storage, processes the requests asynchronously, and writes results to a Cloud Storage output path. There is no fixed completion window — jobs run to completion based on available capacity.

- No per-request overhead; well-suited for large one-off processing jobs
- Input and output stay in your GCP project (Cloud Storage buckets you control)
- Typical completion ~24 hours; maximum queue time is 72 hours
- Supports most Gemini models available on Vertex — tuned models must use a tuned endpoint MODEL_PATH rather than the global endpoint

Docs: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/batch-prediction-gemini

---

## Latency

**Not available.** Vertex AI does not include timing information in API responses, and Coolhand does not read a collector `duration_ms` for Vertex text logs. Latency will always be blank.


---

Related:
- [Google Gemini best practices](https://coolhandlabs.com/help/google-gemini-best-practices.md)
- [OpenAI API best practices](https://coolhandlabs.com/help/openai-api-best-practices.md)
- [Anthropic API best practices](https://coolhandlabs.com/help/anthropic-api-best-practices.md)
- [AWS Bedrock best practices](https://coolhandlabs.com/help/aws-bedrock-best-practices.md)

[← All help articles](https://coolhandlabs.com/help.md) · [Provider guides](https://coolhandlabs.com/help/category/providers.md)