Computer Use Protocol

Example Use Cases

• 1

  Perceive and interact with desktop UIs across multiple platforms using a single unified format.

• 2

  A compact, LLM-optimized encoding of UI accessibility trees to fit complex interfaces into context windows.

• 3

  Perform standardized actions on UI elements with cross-platform compatibility.

Description

Computer Use Protocol (CUP) provides a unified JSON schema and a compact text format to represent UI accessibility trees consistently across Windows, macOS, Linux, Web, Android, and iOS. It enables AI agents to understand and act on desktop UIs with minimal token usage, supporting 59 ARIA-derived roles, 16 states, and 15 canonical actions. CUP also offers SDKs for capturing native UI trees and executing actions, facilitating cross-platform AI agent integration without information loss.