Copilot’s New Capabilities: Voice and Vision
Microsoft is unveiling a major overhaul of its Copilot experience, introducing voice and vision capabilities. Copilot now offers a more personalized AI assistant experience, including a virtual news presenter mode to read headlines and the ability to see what you’re looking at. A new voice feature allows for natural conversations, similar to OpenAI’s Advanced Voice Mode.
Inflection AI’s Influence on Copilot’s Redesign
The new Copilot is designed with a card-based user interface, similar to the personalized Pi AI assistant from Inflection AI. Mustafa Suleyman, cofounder of Google DeepMind, now leads Microsoft AI and has overseen these transformative changes since joining earlier this year. His vision for Copilot is to create an AI companion for everyone, emphasizing calm, helpful, and supportive interactions.
A New Look for Copilot: Warmer and More Inviting
The redesigned Copilot interface departs significantly from previous designs. It now features a personalized Copilot Discover page, customized based on conversation history, and offers useful searches, tips, and relevant information. This change brings a more engaging and personalized user experience.
Enhanced Voice Capabilities: Conversational AI
Microsoft is doubling down on voice interaction, allowing users to chat with Copilot, ask questions, and interrupt conversations naturally. Copilot now offers four voice options to choose from when first using the new experience. This reflects Microsoft’s belief that voice interaction will be central to long-term AI development.
Copilot Vision: AI That Sees What You See
One of the most significant updates is Copilot Vision, allowing the AI assistant to see what’s on your screen. This enables users to ask questions about web pages, images, and content they’re viewing. For instance, while shopping online, Copilot can suggest product recommendations based on what it sees.
Privacy and Safety in Copilot Vision
Copilot Vision sessions are opt-in and Microsoft ensures that none of the content is stored or used for training. Currently, this feature is limited to a select list of popular websites and won’t work on paywalled or sensitive content during its preview phase.
Copilot Daily: AI-Powered News and Weather Recaps
Copilot Daily is a new feature that provides an audio summary of the latest news and weather, designed to be a short morning update. Initially, it will use content from Reuters, Axel Springer, Hearst, and the Financial Times, with plans to expand its sources over time.
Think Deeper: Handling Complex Questions
Microsoft is also introducing Think Deeper, a feature that allows Copilot to take more time to answer complex questions with step-by-step guidance. This feature is part of Copilot Labs, where users can test experimental functions like Copilot Vision and Think Deeper before they are fully released.
Enhanced Privacy and Security in Recall
After facing criticism over Recall security and privacy issues, Microsoft has overhauled the feature with improved options. Users now have the ability to uninstall Recall or choose not to enable it altogether.
Availability of the Updated Copilot Experience
The updated Copilot experience will be available starting today on iOS and Android apps, on the web at copilot.microsoft.com, and through the Windows app. Copilot Voice will initially launch in English in Australia, Canada, New Zealand, the UK, and the US, with more regions and languages to follow. Copilot Daily and Copilot Vision will roll out in select regions and to Pro subscribers.
The Future of Copilot: An Evolving AI Assistant
Looking ahead, Microsoft’s AI CEO Suleyman envisions Copilot becoming more adaptive and dynamic. Over time, Copilot will learn from users’ preferences and behaviors, helping with tasks such as taking notes during doctor’s appointments, planning events, and aiding in decision-making.
