When you buy through affiliate links in our content, we may earn a commission at no extra cost to you. Learn 
Microsoft has introduced a host of experimental Copilot features that significantly expand its generative AI’s abilities, including the ability to look at what’s on your screen.
The company has launched a Copilot Labs hub for Pro subscribers with Copilot Vision as its initial centerpiece. If you’re using the Edge browser, the AI can now study the page you’re on, answer questions, help complete tasks, and suggest your next actions. If you’re looking for a place to stay during a trip, for instance, you can have Copilot find something on the page that meets your criteria.
Copilot Vision is only available in the US at present.
Microsoft is also roughly matching Google’s Gemini Live and OpenAI’s Advanced Voice Mode with Copilot Voice. You can not only talk to Copilot and expect an audible answer, but interrupt it mid-response and have your tone of voice influence the conversation.
The voice option is initially available for English speakers in Australia, Canada, New Zealand, the UK, and the US.
There’s also a feature for users who are more interested in the most insightful answer than the quickest one. Think Deeper uses the “very latest reasoning models” (possibly OpenAI’s o1) to provide more detailed and explicit answers to difficult queries, including challenging math. It’s available now to a “limited” group of Copilot Pro customers in Australia, Canada, New Zealand, the UK, and the US.
And if you’d like a more customized experience, Microsoft is giving you the choice to personalize Copilot with your past history (including other Microsoft apps) and produce new recommendations. It should become available in the US this afternoon.
In all cases, Microsoft is drawing on the lessons learned from the botched Recall debut. Copilot Labs itself is a space where Microsoft can get feedback from early adopters before making advanced AI features broadly available. Copilot Vision is strictly opt-in, and the preview version doesn’t store or train on the data from your conversations.
The company also stresses that it’s limiting the AI to training on a “pre-approved” website list, and blocks it from using paywalls and sensitive material. Microsoft vows that Vision will respect sites’ readable restrictions on AI, and answer questions rather than directly controlling sites.
The concepts are far from new. Google Assistant can now draw on Lens to search your Android phone’s screen, while Gemini is also aware. In 2025, Apple Intelligence should give Siri screen awareness. However, Copilot Vision is important simply because it could reach most computer owners, if at first solely though Pro subscriptions.
