Retrieval-Augmented Generation

RAG is the pattern of retrieving relevant documents from a knowledge base and injecting them into the LLM's context before generating a response. This eliminates hallucinations for domain-specific questions.

The RAG Pipeline

1. Ingest: Split documents into chunks → Embed each chunk → Store in vector database
2. Query: Embed the user's question → Search vector DB for similar chunks
3. Generate: Pass retrieved chunks + question to GPT → Get grounded answer

OpenAI Vector Stores

OpenAI provides a fully managed vector store via the API. Upload files, and OpenAI handles chunking, embedding, and search automatically.

// Create a vector store
const vs = await openai.vectorStores.create({ name: "product-docs" });

// Upload files
await openai.vectorStores.files.create(vs.id, {
  file_id: "file-abc123"  // Previously uploaded file
});

// Use in Responses API
const response = await openai.responses.create({
  model: "o3-mini",
  tools: [{ type: "file_search", vector_store_ids: [vs.id] }],
  input: "What is our refund policy?"
});