In a major upgrade to ChatGPT’s Advanced Voice Mode, OpenAI has now introduced vision capabilities. With this, ChatGPT can now analyse and respond to live video input and even share screens during conversations. While the feature was introduced with GPT-4o, so far only the audio feature was live. However, now users can chat with ChatGPT using their smartphone camera; essentially, the chatbot can see what you see.
With this update, ChatGPT can understand and discuss the visual context in real time. The feature works via a new video icon in the mobile app, where screen sharing is available through a separate menu option. The new updates are available to ChatGPT Plus, Pro, and Team subscribers, with Enterprise and Edu users getting access in January. The AI powerhouse has also introduced a festive voice option enabling users to chat with Santa, which is essentially a limited-time seasonal addition through early January.
OpenAI CPO Kevin Weil, along with other team members at the AI startup, showed a demo of ChatGPT helping with how to make pour-over coffee. During the demo, the members pointed the camera at the action, and the bot showed that it understood the principle of the coffee maker and explained brewing to the team. The team also showed how ChatGPT now supports screensharing with its understanding of an open message on a smartphone where it understands Weil was wearing a Santa beard.
The latest update comes a day after Google introduced its latest Gemini 2.0 model, capable of processing visual and audio inputs. The Gemini 2.0 is capable of multi-step tasks through three research projects, such as Project Astra (universal assistant), Project Mariner, and Project Jules, which are essentially developer tools. On the other hand, OpenAI’s latest demo shows ChatGPT’s vision modality, which, as of now, is capable of identifying objects and seamless interactions.
Why should you buy our Subscription?
You want to be the smartest in the room.
You want access to our award-winning journalism.
You don’t want to be misled and misinformed.
Choose your subscription package
