Cutting LLM Inference Spend by 68% with Prompt Caching and Price-Sorted Routing
Practical architectural patterns for structuring system prompts, leveraging provider.sort="price", and cascading from flash models to frontier reasoning models.
1. Structuring Static Prefixes for Maximum Cache Hits
Upstream prompt caches match exact token prefixes. Placing dynamic timestamps, user metadata, or per-request variables at the top of your system prompt invalidates the cache on every call.
Keep your system instructions, tool schemas, and reference documents strictly static in the first messages, and append dynamic user context at the end of the message array.
2. Dynamic Price Sorting & Model Cascading
By passing provider: { sort: "price" } in your request body, Zopyr’s router automatically orders healthy candidate endpoints from lowest to highest blended token price while still respecting your context window and tool-calling requirements.
{
"model": "qwen/qwen-3.8-max-prime",
"provider": {
"sort": "price",
"allow_fallbacks": true
},
"messages": [
{ "role": "system", "content": "You are a financial document parser." },
{ "role": "user", "content": "Extract line items as JSON." }
]
}