The Computer Use Agent

AI That Operates Your Computer

The Computer Use Agent (CUA) enables AI models to interact with any software through a screenshot-action loop: the model views a screenshot, decides what to click/type/scroll, and the action is executed in a virtual environment.

How CUA Works

1. Screenshot: Capture the current screen state
2. Reasoning: The model analyzes the screenshot and decides the next action
3. Action: Execute the action (click, type, scroll, drag)
4. Repeat: Capture new screenshot, continue until task is complete

Supported Actions

const response = await openai.responses.create({
  model: "computer-use-preview",
  tools: [{
    type: "computer_use_preview",
    display_width: 1024,
    display_height: 768,
    environment: "browser"
  }],
  input: "Go to Hacker News and find today's top story"
});