DeepMind Explores AI-Enhanced Pointer for Computers
A team at Google DeepMind, led by Adrien Baranes and Rob Marchant, works on better ways for people to interact with AI. The standard mouse pointer has stayed the same for over 50 years. It appears on every screen, website, and document. Now, engineers add AI features so the pointer grasps both the target and the reason behind the point.
Users often face issues with AI tools that require copying content into separate windows. DeepMind aims to fix this by bringing AI directly into existing apps. Picture pointing at a building photo and saying, "Show me directions." The AI handles the rest using built-in context. On May 12, 2026, the team shared core ideas for new interfaces and demoed a pointer powered by Gemini. Users can try it in Google AI Studio to edit images or locate spots on maps through simple points and voice commands.
Google DeepMind, formed in London in 2010 and bought by Google in 2014, leads in AI research. It has advanced models like Gemini, a family of large language models that process text, images, and more. This project builds on that to change how people use computers.
Four Key Interaction Principles
DeepMind sets four rules to move context and intent handling from users to machines. These cut down on long text prompts and favor easy actions.
Maintain the Flow
AI should operate in any app without pulling users away. The test pointer works everywhere. For instance, point to a PDF for a bullet-point summary to copy into email. Hover over data tables for a pie chart. Highlight a recipe to double ingredients.
Show and Tell
AI today needs exact descriptions for best results. The new pointer grabs visual and meaning details around it. Point to a word, image section, paragraph, or code, and the AI focuses there precisely.
Stay ahead of the AI curve
The most important updates, news, and content — delivered weekly.
No spam. Unsubscribe anytime.
Embrace 'This' and 'That'
People use short phrases like "Fix this," "Move that," or "What does this mean?" with gestures. The pointer combines point, speech, and context for natural requests without detailed setup.
Turn Pixels into Actionable Entities
Computers once just noted pointer position. AI now identifies objects like places, dates, or items. A note photo turns into a to-do list. A video pause creates a restaurant booking link.
These steps make AI collaboration feel natural and smooth. DeepMind plans to add them to daily products.
Integration into Google Products
The principles now shape pointer use in Chrome and the Googlebook laptop. From today, point in Chrome to query Gemini about page parts. Select products to compare them. Point to visualize a couch in a room. Magic Pointer rolls out soon in Googlebook for fingertip Gemini access. Tests continue on platforms like Google Labs' Disco.
Try the AI Pointer in Google AI Studio
Users can test editing images or finding map locations by pointing and speaking. Videos show the setup, though sequences are sped up.