[ ABORT TO HUD ]
SEQ. 1
SEQ. 2

Running Models with Ollama

🧠 Applied Local AI & RAG⏱ 15 min⭐ 150 BASE XP⌨ HANDS-ON LAB

Ollama & Modelfiles

Ollama has revolutionized local AI by providing an incredibly simple, Docker-like experience for running large language models via the command line.

Core Commands

  • ollama run <model-name>: Automatically pulls the model (if missing) and drops you into an interactive chat session.
  • ollama list (or ls): Lists all models currently downloaded to your system.
  • ollama ps: Shows which models are currently loaded into memory and running.

The Modelfile

Similar to a Dockerfile, you can customize models using a Modelfile. This allows you to bake in system prompts and temperature parameters.

FROM llama3
SYSTEM "You are a highly skilled Rust engineer. Only answer in valid Rust code."
PARAMETER temperature 0.1

You can then create your custom model using ollama create custom-rust-bot -f ./Modelfile.

⌨ HANDS-ON LABRun Your First Local Model
⭐ +150 XP

Verify Ollama is installed, pull the Llama 3 model, and list your installed models.

1Check if the Ollama CLI is installed.
2Pull the Llama 3 model weights without starting a chat.
3List all local models to verify it downloaded.
lab-sandbox — simulated environment
INFINITY LAB SANDBOX v2.6 — simulated shell
Type the command for the current objective. Helpers: "hint", "solution", "clear".
$
OBJECTIVE 1 / 3 — type "hint" if stuck
KNOWLEDGE CHECK
QUERY 1 // 1
What is the primary privacy advantage of running open-weight models locally?
They use less electricity.
Your data never leaves your machine.
They are always smarter than proprietary APIs.
They require an internet connection to work.
Running Models with Ollama Tutorial | Applied Local AI & RAG — Open Source AI Academy