[ ABORT TO HUD ]
SEQ. 1
SEQ. 2

Running Models with Ollama

🧠 Applied Local AI & RAG⏱ 15 min⭐ 150 BASE XP⌨ HANDS-ON LAB

Ollama & Modelfiles

Ollama has revolutionized local AI by providing an incredibly simple, Docker-like experience for running large language models via the command line.

Core Commands

  • ollama run <model-name>: Automatically pulls the model (if missing) and drops you into an interactive chat session.
  • ollama list (or ls): Lists all models currently downloaded to your system.
  • ollama ps: Shows which models are currently loaded into memory and running.

The Modelfile

Similar to a Dockerfile, you can customize models using a Modelfile. This allows you to bake in system prompts and temperature parameters.

FROM llama3
SYSTEM "You are a highly skilled Rust engineer. Only answer in valid Rust code."
PARAMETER temperature 0.1

You can then create your custom model using ollama create custom-rust-bot -f ./Modelfile.

⌨ HANDS-ON LABRun Your First Local Model
⭐ +150 XP

Verify Ollama is installed, pull the Llama 3 model, and list your installed models.

1Check if the Ollama CLI is installed.
2Pull the Llama 3 model weights without starting a chat.
3List all local models to verify it downloaded.
lab-sandbox — simulated environment
INFINITY LAB SANDBOX v2.6 — simulated shell
Type the command for the current objective. Helpers: "hint", "solution", "clear".
$
OBJECTIVE 1 / 3 — type "hint" if stuck
KNOWLEDGE CHECK
QUERY 1 // 1
What is the primary privacy advantage of running open-weight models locally?
They use less electricity.
Your data never leaves your machine.
They are always smarter than proprietary APIs.
They require an internet connection to work.
Watch: 139x Rust Speedup
Running Models with Ollama Tutorial | Applied Local AI & RAG — Open Source AI Academy