Provider feature that reuses computation for repeated long prefixes—cuts latency and cost for stable system prompts and documents.
Prompt caching (sometimes “context caching”) is different from the KV cache inside a single request: it amortizes work across calls when prefixes match provider rules. It rewards stable system prompts and careful ordering of variable user content.
Caveats
Caches can affect privacy and debugging—know what is retained and for how long.