A step by step implementation of building an AI agent that plays 3d shooting game

AI Agent that plays 3D shooting game

AI agents are now also becoming part of games. The way they interact with the environment, take actions, and make decisions all depends on proper thinking and proper usage of LLMs/multimodals. So, to learn how AI agents work in a gaming environment:

1. First, we are going to create a first-person 3D action shooting game using the Python Ursina engine.
2. Then we will create a multi-AI agent based on sub-agents using LLM multimodals, like Gemini V3 for text and Mistral 3.1 for vision capability, combined together to form a powerful multi-agent system that will play the game for us and win it.

Take a look at how our AI agent (Gemini + Mistral) is playing our own 3D action game, failing at the first attempt but winning after the second attempt.

We will code everything from scratch, all the way from building the 3D game engine to creating the AI agent using an LLM and a vision-language model.