Navigating the Unintegrated World
Not every enterprise system has an API. Legacy ERP systems, government portals, and decades-old line-of-business software often present only a graphical user interface. In 2026, Copilot Studio agents can interact with these systems using Computer Use — the same capability available in Claude's API, now integrated into the Power Platform ecosystem.
How Copilot Studio Computer Use Works
- Screenshot Capture: The agent takes a screenshot of the target application (running in a secure sandbox).
- Visual Reasoning: The model analyzes the screenshot to identify UI elements, buttons, fields, and forms.
- Action Execution: The agent moves the mouse, types text, clicks buttons, and navigates menus — just like a human operator.
- Self-Correction: If an action fails (element not found), the agent re-analyzes and adapts its approach.
Use Cases
| Industry | System | Automation |
|---|
| Finance | Legacy banking ERP | Extract account balances, process transactions |
| Government | Citizen portals | Submit forms, check application status |
| Healthcare | Clinical systems (HL7/FHIR-less) | Enter patient data, retrieve records |
| Manufacturing | SCADA/HMI systems | Monitor parameters, adjust settings |
Safety & Governance Requirements
Computer Use in enterprise requires strict sandboxing:
- Isolated VM: The target application runs in a dedicated, network-restricted virtual machine
- Action logging: Every mouse click, keystroke, and screenshot is logged to Purview for audit
- Human-in-the-loop: High-stakes actions (form submissions, data deletion) require human confirmation
- Scope restrictions: Agents can only interact with pre-approved applications; arbitrary web browsing is blocked
🔒 Security Note: Computer Use should only be deployed in isolated sandboxes. Never allow a Computer Use agent network access to production systems without strict firewall rules — prompt injection via UI content could potentially command the agent to perform unauthorized actions.