Slash costs and latency by caching massive prompts. Part of the free Vertex AI Academy — every lesson below is open to everyone, no signup required.
Slashing Costs by 70% When you cache a large prompt (like a codebase or a 1-hour video), Google processes the input and stores the Key-Value (KV) cach...
Querying the Cache Once a cache is created, you instantiate a GenerativeModel pointing to the cache instead of providing the massive context again. fr...
Time-To-Live (TTL) Caches are not free; you are billed per hour based on the number of tokens stored in the cache. Therefore, you must specify a TTL (...