A little proof-of-concept agent that uses text-only LLMs to navigate Linux desktop

useless-agent

What can this agent do? Basically nothing, that’s why it is the useless-agent.

Why is it interesting?

• Uses text-only LLMs.
• Cheap: I spent about $4.57 playing with it for about 7 evenings.
• Single binary. (almost, see todo list)
• Easy to use: run the binary and copy the IP address.
• No telemetry, no bullshit.
• IPv4 & IPv6(*should work, not tested yet)

:star: Would you like me to work more on this project? Please consider giving this repository a star!

[!CAUTION]

• Only use this on a disposable virtual machine.
• It can, and most likely will, destroy your system.
• The LLM API provider has a realistic ability to inject malicious commands/actions/data into the ingested API responses.
• The video is not compressed. If you are connected to a virtual machine in the cloud, be aware of high internet traffic.
• The video stream and everything else, except the API queries, are not encrypted. If connecting to a remote machine, use an SSH tunnel with port forwarding.

[!NOTE]
It is super slow. Right now, speed is not a priority. If your only problem is speed, you have already won the agents game.

Currently supported llm providers:

• DeepSeek
• Z.AI

Currently supported models:

• deepseek-chat
• deepseek-reasoner
• glm-5
• glm-4.6
• glm-4.5
• glm-4.5-air
• glm-4.5-flash

Environment: Only works on - Linux + xfce + X11.

Changelog

v0.0.3

• User-assist feature works.
• Faster task cancellation.
• Extract css and js from the html file.
• Fixed routing of the line which connects chat and session window.
• Fixed media buttons overlay resizing.
• Tasks destined to the same session now queued.
• Use more API of the environment where it's possible.
• Break main.go into separate packages.
• Other small fixes.

v0.0.2

• Manage multiple sessions at once.
• Fullscreen & maximize control elements for the video stream. Hotkeys F and M.