Using a Cached Content — Context Caching

Querying & Sharing the KV Cache

Once a context cache resource is created on Vertex AI, it is assigned a globally unique resource name (e.g., projects/PROJECT_ID/locations/REGION/cachedContents/CACHE_ID). The underlying Key-Value (KV) attention tensors reside in high-speed accelerator memory across Google's TPU infrastructure, ready to be referenced by thousands of concurrent downstream requests.

Instead of passing the massive document, video, or codebase in the request payload, downstream calls simply pass the cached_content resource reference. This dramatically slashes network payload bandwidth, cuts Time-To-First-Token (TTFT) by up to 80%, and guarantees that all users query the exact same foundational context state.

from google import genai
from google.genai import types

client = genai.Client(vertexai=True, project="my-gcp-project", location="global")

# 1. Create the persistent context cache (e.g., 200-page regulatory handbook)
cache = client.caches.create(
    model="gemini-3.8-flash",
    config=types.CreateCachedContentConfig(
        contents=[regulatory_pdf_part],
        display_name="fca_compliance_handbook_2026",
        system_instruction="You are an expert regulatory compliance auditor.",
        ttl="3600s", # 1-hour initial TTL
    )
)

# 2. Query against the active cache resource
response = client.models.generate_content(
    model="gemini-3.8-flash",
    contents="Under section 4.2, what are the mandatory reporting deadlines for cyber incidents?",
    config=types.GenerateContentConfig(
        cached_content=cache.name,
        temperature=0.1,
    )
)
print(response.text)