Serverless Foundation Model APIs vs Dedicated Endpoints

On Vertex AI, developers choose between two fundamentally distinct architectural patterns depending on whether they are serving managed proprietary foundation models or hosting specialized open-source weights.

1. Serverless Foundation Model APIs (Gemini)

For Google's frontier models (Gemini 3.8 Flash, Gemini 2.5 Pro, Gemini 2.0 Flash Thinking), Vertex AI operates in a purely serverless paradigm:

• Zero Cluster Management: No VMs, GPUs, or TPU pods to provision, patch, or autoscale.
• Pay-Per-Token Billing: Incur costs strictly for input tokens processed and output tokens generated. Idle applications cost exactly $0.00.
• High-Throughput Provisioned Throughput (PT): For predictable enterprise workloads requiring strict latency SLAs and guaranteed throughput, enterprises can reserve dedicated throughput capacity units rather than sharing standard multi-tenant pools.

2. Dedicated Prediction Endpoints (Model Garden & Custom Weights)

When deploying custom fine-tuned weights, open-source models (such as Llama 3.3 70B, Mistral Large, or DeepSeek-R1), or proprietary PyTorch/JAX artifacts from the Vertex AI Model Garden, you deploy to a Dedicated Endpoint:

• Hardware Allocation: You specify exact Compute Engine machine types (e.g., g2-standard-48 with 4x NVIDIA L4 GPUs or a2-ultragpu-8g with 8x NVIDIA A100 80GB GPUs).
• Container Runtimes: Uses pre-built optimized serving containers (vLLM, Hugging Face TGI, or Triton Inference Server).
• Autoscaling & Hourly Billing: You pay continuous hourly VM and GPU rates for as long as instances are active. Autoscaling scales between min_replica_count and max_replica_count based on CPU/GPU utilization or queue depth.

import { GoogleGenAI } from "@google/genai";

// 1. Serverless Gemini API via unified Google Gen AI SDK
const ai = new GoogleGenAI({ vertexAI: true, project: "my-gcp-project", location: "us-central1" });
const response = await ai.models.generateContent({
  model: "gemini-3.8-flash",
  contents: "Explain the architectural trade-offs of dedicated endpoints vs serverless APIs."
});
console.log(response.text);