PC-Agent: A Multi-Agent Framework for Automating Complex Tasks on PC 192

What is PC-Agent?

PC-Agent is a new hierarchical AI agent framework based on multimodal large models (MLLM), jointly developed by Alibaba Tongyi Lab and Shanghai Jiao Tong University. By simulating human cognitive processes, it enables automated operation of complex PC tasks. It can control Chrome, Word, WeChat, and other applications based on instructions, improving the task success rate by 32% compared to existing methods, and addressing the limitations of traditional approaches on complex PC tasks.

Core Features of PC-Agent

• Active Perception Module (APM): By combining multimodal large models with screen capture content, PC-Agent achieves fine-grained perception and operation of screen content.

• Hierarchical Multi-Agent Collaboration Structure: PC-Agent decomposes complex instructions into three layers—instructions, subtasks, and actions—and assigns four agents accordingly: Manager, Progress, Decision, and Reflection. This structure enables top-down task decomposition and bottom-up precise feedback.

• Dynamic Decision-Making Mechanism: The Reflection Agent provides real-time feedback and adjustment of execution results, ensuring task accuracy and adaptability.

Application Scenarios of PC-Agent

• Task Automation: PC-Agent can automate complex digital tasks, such as organizing research materials, writing reports, and creating presentations.

• Cross-Application Workflow: The framework can handle complex cross-application tasks, such as data interaction and operations between different software.

• Efficient Data Utilization: Even when trained on a small amount of high-quality data, PC-Agent can handle complex workflows of up to 50 steps.

Advantages of PC-Agent

• High Task Success Rate: The success rate on complex tasks is 32% higher than existing methods.

• High Data Efficiency: Only a small amount of data is needed to learn complex tasks.

• Strong Operational Capability: It can accurately perceive screen content and complete complex editing and cross-application operations.

• Wide Range of Applicable Scenarios: It can automatically handle office tasks such as documents, spreadsheets, presentations, and schedule management.

• High User Satisfaction: 92% accuracy, fast response, and a user rating of 4.5 out of 5.

• Performance Improvement: In experimental evaluations on complex tasks, PC-Agent's task success rate is 32% higher than existing methods.

• Strong Adaptability: By simulating human cognitive processes, PC-Agent can better adapt to complex interactive environments.

GitHub Repository:https://github.com/X-PLUG/MobileAgent/tree/main/PC-Agent