NON906/omniparser-autogui-mcp
Free屏幕 GUI 的自动操作。
FreeFree tier
About NON906/omniparser-autogui-mcp
A Model Context Protocol (MCP) server that uses OmniParser to analyze the on-screen GUI and perform automated operations. It captures the screen, interprets the visual content via OmniParser, and executes actions such as mouse clicks and keystrokes based on natural language instructions or task descriptions. Confirmed working on Windows, it integrates with MCP-compatible clients like Claude Desktop to enable AI-driven desktop automation. Supports optional window targeting, remote OmniParser processing, and SSE communication.
Key Features
Screen analysis and GUI automation using OmniParser
Native MCP protocol integration for LLM-driven automation
Optional target window specification for focused operations
Configurable OCR language via environment variable
Support for remote OmniParser processing on a separate device
SSE (Server-Sent Events) communication mode
Customizable OmniParser model and box threshold settings
Confirmed working on Windows
Pros & Cons
Pros
- Open source (MIT license, with submodule exceptions)
- Integrates seamlessly with the MCP ecosystem (e.g., Claude Desktop)
- Flexible configuration: window targeting, remote server, SSE
- Supports multiple OCR languages
- Actively maintained on GitHub with community stars and forks
Cons
- Only confirmed on Windows (cross-platform testing limited)
- Requires downloading OmniParser models (may need GPU)
- Setup involves multiple steps: clone, sync submodules, install dependencies, download models
- Relies on OmniParser submodule, which has its own license constraints (CC-BY-4.0)
Best For
Automating desktop GUI tasks with AI agents (e.g., searching in a browser)Testing and interacting with on-screen applications programmaticallyEnabling LLMs to perform multi-step GUI workflows via MCP
FAQ
What operating systems does OmniParser Autogui MCP support?
The tool has been confirmed working on Windows. It may work on other platforms but official confirmation is limited.
How do I install and set up this MCP server?
Clone the repository with submodules, use 'uv sync' to install dependencies, set the OCR language, run 'download_models.py' to fetch OmniParser models, and configure the MCP client (e.g., Claude Desktop) with appropriate settings in claude_desktop_config.json.
Can I run OmniParser on a different machine?
Yes, by setting the OMNI_PARSER_SERVER environment variable to the server's address and port, you can offload OmniParser processing to another device.
What license does this tool use?
The main repository is MIT licensed, but it includes submodules and sub-packages that may have different licenses (e.g., OmniParser is CC-BY-4.0).