NakaokaRei/swift-mcp-gui
Free可以执行键盘输入和鼠标移动等命令的 MCP 服务器。
FreeFree tier
About NakaokaRei/swift-mcp-gui
swift-mcp-gui is an open-source Model Context Protocol (MCP) server that enables programmatic control of macOS through SwiftAutoGUI. It provides a suite of tools for mouse movement, clicks, keyboard input, scrolling, screen capture, and pixel color retrieval. Designed for integration with MCP clients such as Claude, it allows AI agents to directly interact with the macOS graphical interface. The server requires macOS 15.0 or later, Swift 6.0, and Xcode 16.0. Installation is available via Homebrew or from source. Tools include moveMouse, mouseClick, sendKeys, scroll, getScreenSize, getPixelColor, captureScreen, and captureRegion, with flexible input options and output formats (file path or inline image data).
Key Features
Mouse movement to specified coordinates (x, y)
Mouse clicks (left or right button) at current cursor position
Keyboard input supporting key combinations and shortcuts
Scrolling in four directions (up, down, left, right) with configurable clicks
Screen size retrieval returning width and height
Pixel color capture returning RGBA values (0-255) at given coordinates
Screen capture with adjustable JPEG quality (0.0-1.0) and scale factor (0.1-1.0)
Region capture for specific screen areas with configurable quality, scale, and output format
Pros & Cons
Pros
- Free and open-source with permissive license
- Easy installation via Homebrew or from source
- Comprehensive set of GUI automation tools
- Supports both file-based and inline image output to reduce token consumption
- Actively maintained on GitHub with community engagement
Cons
- Only works on macOS 15.0 or later
- Requires Swift 6.0 and Xcode 16.0 for building from source
- No built-in support for Windows or Linux
- Limited to GUI control; no file management or system commands beyond provided tools
Best For
Automating GUI interactions on macOS via AI agentsTesting and debugging macOS applications programmaticallyIntegrating screen capture and analysis into MCP-based workflowsControlling external devices or simulations requiring mouse/keyboard inputBuilding accessibility or automation tools for macOS
FAQ
What are the system requirements for swift-mcp-gui?
It requires macOS 15.0 or later, Swift 6.0 or later, and Xcode 16.0 or later.
How do I install swift-mcp-gui?
The recommended method is via Homebrew: `brew install NakaokaRei/tap/swift-mcp-gui`. Alternatively, you can clone the repository and build from source using Swift Package Manager.
What tools are provided by swift-mcp-gui?
The server provides eight tools: moveMouse, mouseClick, sendKeys, scroll, getScreenSize, getPixelColor, captureScreen, and captureRegion.
Can I capture a screen region instead of the full screen?
Yes, the captureRegion tool allows you to specify x, y, width, height, and optionally quality, scale, and output format.