Constrained Decoding & Token-Level Guarantees

Production enterprise pipelines cannot tolerate malformed JSON, truncated markdown backticks, or missing mandatory fields. Gemini on Vertex AI guarantees 100% schema compliance through Constrained Decoding.

Rather than relying on prompt engineering (e.g., "Please return valid JSON only"), Vertex AI compiles the provided schema into a grammar constraint automaton directly at the logits sampling level. Tokens that violate the schema receive a probability of zero, mathematically guaranteeing valid parsing.

from pydantic import BaseModel, Field
from google import genai
from google.genai import types

class AuditFinding(BaseModel):
    severity: str = Field(description="CRITICAL, HIGH, MEDIUM, or LOW")
    cve_id: str = Field(description="CVE identifier or N/A")
    description: str = Field(description="Concise description of vulnerability")
    remediation_steps: list[str] = Field(description="Actionable remediation steps")

class SecurityReport(BaseModel):
    target_host: str
    scan_timestamp: str
    findings: list[AuditFinding]

client = genai.Client()

response = client.models.generate_content(
    model="gemini-2.0-flash",
    contents="Scan log excerpt: OpenSSH 7.4 buffer overflow detected on 192.168.1.50.",
    config=types.GenerateContentConfig(
        response_mime_type="application/json",
        response_schema=SecurityReport,
    )
)

# Guaranteed to parse into strongly typed Pydantic object
report = SecurityReport.model_validate_json(response.text)
print(f"Parsed {len(report.findings)} findings for {report.target_host}")