Is the Open Source AI Academy free?

Yes, 100% free. No sign-up, no credit card. All 18 modules, 31 lessons, and 73 quiz questions are accessible immediately. Your progress saves locally in your browser.

How is Infinity Open Source AI Academy different from vendor-specific open-weight tutorials?

Unlike generic or single-model tutorials, Infinity Tech Stack's Open Source AI Academy bridges global open weights (Llama 3.3, Mistral) and China Frontier Models (DeepSeek R1, Qwen 2.5, Kimi K3, GLM) with 18 comprehensive modules and simulated in-browser terminal labs for Ollama, llama.cpp, vLLM, and GGUF quantization. It requires zero GPU hardware, no cloud subscriptions, and no sign-up.

Does the Open Source AI Academy cover China's latest frontier open models?

Yes. The academy includes a dedicated China Frontier Models module covering Qwen, DeepSeek, Z.ai GLM, Moonshot Kimi, InternLM, Baichuan, and Yi, plus a practical model-radar workflow for weekly tracking and gated production adoption.

Did Kimi K3 score higher than Claude?

On several benchmarks, yes. Kimi K3 (July 14, 2026: 2.8T open-weight MoE, 1M context) beat Claude Fable 5 on 6 of 14 shared benchmarks — including BrowseComp (91.2 vs 88.0), SWE Marathon (42.0 vs 35.0), and Terminal-Bench 2.1 (88.3 vs 84.6) — topped the Arena.ai Frontend Code Arena, and edged Claude Opus 4.8 on the Artificial Analysis Intelligence Index (57 vs 56) at about 3.3x lower pricing. Claude Fable 5 won the other 8 benchmarks and leads that index at 60. The academy's China Frontier Models module has a dedicated lesson on the full scoreboard.

What topics does the Open Source AI Academy cover?

The course covers Transformer architecture, Meta Llama 4, Mistral, DeepSeek, Qwen, Gemma, Hugging Face, quantization (GGUF, AWQ, GPTQ, EXL2), Ollama, llama.cpp, vLLM, SGLang, LoRA/QLoRA fine-tuning, pre-training from scratch, RLHF/DPO/GRPO alignment, and production MLOps with Docker and Kubernetes.

vLLM is the industry-standard engine for high-throughput production LLM serving. It uses PagedAttention for efficient GPU memory management and continuous batching for >90% GPU utilization.

Do I need expensive hardware to run open-source models?

No. Many models can run on consumer GPUs (e.g., RTX 4090) or even CPUs via llama.cpp. The course covers deployment strategies for every budget, from Raspberry Pi edge devices to multi-GPU production clusters.

Does the Open Source AI Academy include hands-on labs?

Yes. Hands-on terminal labs are built into the lessons - ollama pull, run, and list models, quantize a model to Q4_K_M, and serve it with vLLM using real, vendor-verified commands, with objectives, hints, solutions, and XP rewards. No GPU or sign-up needed.

What is speculative decoding?

A draft model proposes tokens that the main model verifies in batch — Speculative decoding uses a small draft model to propose tokens, then the main model verifies them in a single forward pass - faster than sequential generation.

llama.cpp Deep Dive Tutorial | llama.cpp Engine

[ ABORT TO HUD ]

SEQ. 1

llama.cpp Deep Dive

⚡ llama.cpp Engine⏱ 15 min⭐ 125 BASE XP⌨ HANDS-ON LAB

The Universal Inference Engine

llama.cpp is the industry-standard engine for running LLMs on any hardware - from Raspberry Pis to multi-GPU servers.

Architecture

GGML: Custom tensor library optimized for quantized inference
GGUF: Universal model format supporting all major architectures
Backends: CUDA, Metal, ROCm, Vulkan, OpenVINO (Intel NPUs)

Inference Optimization Techniques

Technique	What It Does	Speedup
GPU Layer Offloading	Offload N layers to GPU, rest on CPU	2-10x vs CPU-only
Speculative Decoding	Draft model proposes tokens, main model verifies	1.5-3x throughput
Speculative Checkpointing	Extends speculative decoding to MoE models	Variable (MoE-specific)
Flash Attention	Memory-efficient attention computation	2x+ for long contexts
Batch Processing	Process multiple requests simultaneously	Linear with batch size
Mmap Loading	Memory-map model files (instant cold start)	Near-zero startup

llama-server (HTTP API)

# Basic server
llama-server -m model.gguf --host 0.0.0.0 --port 8080

# Optimized production server
llama-server -m model.gguf \
  --host 0.0.0.0 --port 8080 \
  -ngl 99 \           # Offload all layers to GPU
  --ctx-size 32768 \  # Context window
  -np 4 \             # 4 parallel request slots
  --flash-attn \      # Enable Flash Attention
  --cont-batching      # Continuous batching

MCP Integration

llama-server now supports Model Context Protocol natively - enabling direct tool calling from your local model.

🐳 Production Container:

docker run -d --gpus all \
  -v ./models:/models \
  -p 8080:8080 \
  --name llama-server \
  ghcr.io/ggml-org/llama.cpp:server \
  -m /models/mistral-large-Q4_K_M.gguf \
  --host 0.0.0.0 -ngl 99 --flash-attn \
  -np 8 --cont-batching

⌨ HANDS-ON LABBuild llama.cpp from Source

⭐ +200 XP

Ollama wraps it; now own it. Clone llama.cpp, compile the engine with CMake, and run a GGUF model directly with llama-cli - no wrapper, full control over every flag.

1Clone the llama.cpp repository and enter it.

2Configure and compile with CMake in Release mode.

3Run inference on a GGUF model straight from the binary you just built.

lab-sandbox — simulated environment

INFINITY LAB SANDBOX v2.6 — simulated shell
Type the command for the current objective. Helpers: "hint", "solution", "clear".
$

OBJECTIVE 1 / 3 — type "hint" if stuck

KNOWLEDGE CHECK

QUERY 1 // 2

What does the -ngl 99 flag do in llama-server?

Limits to 99 tokens

Sets 99 parallel slots

Offloads all layers to GPU

Sets context to 99K