From Language Models to Robot Policies

LeRobot is Hugging Face's open-source framework for physical AI - training and deploying robot policies (Vision-Language-Action models, or VLAs) with the same Hub-first workflow used for LLMs. The v0.6.0 "Imagine, Evaluate, Improve" release (July 2026, now at patch v0.6.1) covers the full robot-learning loop: record data, train a policy, evaluate in simulation, deploy on real hardware, and feed corrections back in.

World Models: Policies That Imagine

v0.6 introduces three policy families that learn to predict the future as part of training, then drop that prediction at inference for zero extra cost:

The Growing VLA Model Zoo

• GR00T N1.7: NVIDIA's cross-embodiment foundation model, now built on Cosmos-Reason2-2B (Qwen3-VL) with a flow-matching action head - replaces GR00T N1.5
• MolmoAct2: Allen Institute for AI's VLA, zero-shot ready on SO-100/101 arms
• EO-1: Qwen2.5-VL-3B backbone pretrained on interleaved vision-text-action data
• Multitask DiT: ~450M-parameter diffusion transformer (TRI Large Behavior Models recipe), one model for many language-selected tasks
• EVO1: compact 0.77B-parameter VLA (InternVL3-1B backbone) that fine-tunes and runs in real time on modest GPUs

Reward Models & Evaluation

A new unified lerobot.rewards API adds Robometer (a pretrained Qwen3-VL-4B reward model scoring task progress from raw video, trained on 1M+ trajectories) and TOPReward (fully zero-shot - reads the log-probability of "True" from any off-the-shelf VLM). Six new simulation benchmarks - LIBERO-plus, RoboTwin 2.0, RoboCasa365, RoboCerebra, RoboMME, and VLABench - all run through one lerobot-eval CLI, bringing the total to nine benchmark families alongside LIBERO, Meta-World, and NVIDIA IsaacLab-Arena.

Deployment & Training at Scale

# Deploy with the new lerobot-rollout CLI (DAgger-style corrections)
lerobot-rollout \
  --strategy.type=dagger \
  --policy.path=your-username/my_policy \
  --robot.type=so100_follower --robot.port=/dev/ttyACM0 \
  --teleop.type=so101_leader --teleop.port=/dev/ttyACM1 \
  --dataset.repo_id=your-username/dagger_corrections

# Train in the cloud on HF Jobs (no local GPU needed)
lerobot-train \
  --dataset.repo_id=your-username/so101_test \
  --policy.type=act --policy.repo_id=your-username/my_policy \
  --job.target=a10g-small

Training now supports FSDP (via Accelerate) for policies bigger than one GPU, and pip install lerobot is leaner - roughly 40% fewer base dependencies, with feature-scoped extras like lerobot[training] and lerobot[groot] covering the rest.