gui-agent
FastMCP server for desktop computer use and GUI automation with mouse, keyboard, OCR and video.
Not verified yetKnown vulnerabilities
How far pickfit got
pickfit hasn’t verified gui-agent yet, so there are no tested install steps.
What follows comes from its README only.
Use it
Before you install: known vulnerabilities
pickfit found 28 known high or critical vulnerabilities in the packages a fresh install brings (OSV.dev, 3 Nov 2026). Weigh this before you install it.
pypi:pillow 12.3.0Pillow: Heap buffer overflow in _tiffWriteProc when saving TIFF to BytesIO (GHSA-332p-7hrc-92wf, critical, fixed in 13.0.0)
See 9 more
pypi:pillow 12.3.0— GHSA-5x8w-8jj5-q52h (critical), fixed in 13.0.0pypi:pillow 12.3.0— GHSA-8g5g-8vmr-9782 (critical), fixed in 13.0.0pypi:pillow 12.3.0— GHSA-v9pv-5gc8-w2m6 (critical), fixed in 13.0.0pypi:pillow 12.3.0— GHSA-w889-685w-c8fh (critical), fixed in 13.0.0pypi:pillow 12.3.0— GHSA-22hg-r758-rrw9 (high), fixed in 13.0.0pypi:pillow 12.3.0— GHSA-27h8-gf7j-v5hc (high), fixed in 13.0.0pypi:pillow 12.3.0— GHSA-3j72-p6qg-6xp7 (high), fixed in 13.0.0pypi:pillow 12.3.0— GHSA-58r4-6rv3-gfh8 (high), fixed in 13.0.0pypi:pillow 12.3.0— GHSA-5xm2-6676-qcp9 (high), fixed in 13.0.0
18 more findings not listed here.
You need first
- Runtimes:
python 3.10+ - System packages:
xdotool,wmctrl,ffmpeg,tesseract-ocr
What it does
pickfit hasn’t run it yet, so these come from the README only.
- Click and move the mouse cursorNot tested
- Send keystrokes and hotkeysNot tested
- Find text on the screen with OCRNot tested
- Take screenshots with a coordinate gridNot tested