Skip to main content

Ollama best practices

Last updated 3 min read

Coolhand captures logs for both text generation and embedding requests from your self-hosted Ollama models. This article covers how to get the most out of your Ollama integration, including how to control the context window, detect truncated outputs, and correctly name your models for clean analytics.

Model naming

Ollama uses a name:tag format (e.g., llama3.1:8b, deepseek-r1:14b, qwen2.5:32b). Different tags for the same base model (e.g., llama3.1:8b vs llama3.1:8b-instruct-q4_K_M) appear as separate models in Coolhand. Standardize tags across your team to keep analytics meaningful.

Embedding logs

Coolhand captures logs from Ollama's /api/embed and /api/embeddings endpoints. No special configuration is needed. Requests made to these endpoints will automatically be logged and available in Coolhand for analysis.

Truncation detection

Coolhand uses the done_reason field from the Ollama API response to detect when an output has been truncated. If the done_reason field has a value of length, Coolhand sets the output_truncated flag on the log to true. This helps you identify and diagnose incomplete model responses.

Context window

To control the context window size for a request, include the num_ctx parameter in the options object of your request. Coolhand reads this parameter and records it as the context window limit for the log, allowing you to see the exact context size used for each request.

Structured outputs

Supported via the format parameter. Pass a JSON schema to enforce structured output.

Docs: https://ollama.com/blog/structured-outputs

Caching

Not supported. Cached Input Tokens and Cache Creation Tokens will always be blank.

Batch processing

No batch API. A single Ollama instance handles concurrent requests in parallel via the OLLAMA_NUM_PARALLEL environment variable (default: 1). Raise it to process multiple requests simultaneously. Requests beyond the parallel limit are queued server-side up to OLLAMA_MAX_QUEUE (default: 512); Ollama returns HTTP 503 only when that queue is full. For higher throughput, increase OLLAMA_NUM_PARALLEL or run multiple instances behind a load balancer.

Latency

Reflects your local hardware and network, not a cloud provider's infrastructure. Expect more variability than hosted APIs, especially on consumer GPUs or CPU inference.

Reasoning tokens

Not supported. Reasoning Tokens will always be blank for Ollama logs, even for models that perform internal reasoning (e.g., DeepSeek-R1). Thinking content may appear in the Thinking Response field, but the token count is not broken out separately.

Cost tracking

Ollama has no per-token pricing — costs are your own compute. Coolhand captures token counts, but there is no dollar-cost mapping for Ollama logs.

Related articles

  • Azure OpenAI best practices

    Azure logs record your deployment name, not the underlying model version, so `my-gpt4o-prod` appears where you might ...

  • OpenRouter best practices

    OpenRouter model IDs are routing aliases in `provider/model` form, not provider-native names. Coolhand records the al...

  • AWS Bedrock best practices

    AWS Bedrock exposes two API surfaces and Coolhand records them as separate sources: `bedrock` for the OpenAI-compatib...

  • Google Gemini best practices

    The Gemini API carries system instructions in a top-level `system_instruction` field rather than a system-role messag...