OpenAI Just Killed Google Translate with GPT-4o
At the OpenAI Spring Update, OpenAI CTO Mira Murati unveiled GPT-4o, a new flagship model that enriches its suite with ‘omni’ capabilities across text, vision, and audio, promising iterative rollouts to enhance both developer and consumer products in the coming weeks. “They are releasing a combined text-audio-vision model that processes all three modalities in one single neural network, which can then do real-time voice translation as a special case afterthought, if you ask it to,” said former OpenAI computer scientist Andrej Karpathy, who was quick to respond to the release. They are releasing a combined text-audio-vision model that processes all three modalities in one single neural network, which can then do real-time voice translation as a special case afterthought, if you ask it to.(fixed it for you) https://t.co/0y36OId88h— Andrej Karpathy (@karpathy) May 13, 2024 “The new voice (and video) mode is the best compute interface I’ve ever used. It feels like AI from the movies; and it’s still a bit surprising to me that it’s real. Getting to human-level response times and expressiveness turns out to be a big change,” said OpenAI chief Sam Altman, who wants to bring ‘Universal Basic Compute’ to everyone in the world. her— Sam Altman (@sama) May 13, 2024 Further, he said that the original ChatGPT hinted at what was possible with language interfaces; “this new thing feels viscerally different. It is fast, smart, fun, natural, and helpful.” Altman said that talking to a computer has never felt really natural for him. “Now it does,” he said, hopeful about the future where people will be using computers to do more than ever before. What’s really interesting about GPT-4o is that it will be available to ChatGPT Plus (with some personalisation features) and ChatGPT free users soon. “We are a business and will find plenty of things to charge for, and that will help us provide free, outstanding AI service to (hopefully) billions of people,” said Altman. “Thanks to Jensen and the NVIDIA team for bringing us the most advanced GPUs to make this demo possible today,” said Murati during her closing remarks. Meanwhile, OpenAI president and co-founder Greg Brockman also demonstrated human-computer interaction (and even human-computer-computer), giving users a glimpse of pre-AGI vibes. Introducing GPT-4o, our new model which can reason across text, audio, and video in real time.It's extremely versatile, fun to play with, and is a step towards a much more natural form of human-computer interaction (and even human-computer-computer interaction): pic.twitter.com/VLG7TJ1JQx— Greg Brockman (@gdb) May 13, 2024 RIP Google Translate? In the demonstration of GPT-4o’s real-time translation capabilities, the model seamlessly translated between English and Italian, exemplifying its sophisticated linguistic adaptability. Many believe that this new feature of OpenAI is likely to replace Google Translate. “OpenAI just killed Google Translate with their real-time translator (near 0 delay in response),” said Fraser, https://twitter.com/tomedwards/status/1790070724482175021 Meanwhile, Google is getting ready to make some major announcements tomorrow at Google I/O. “Super excited for my first Google I/O tomorrow and to share what we’ve been working on!,” shared Google DeepMind chief Demis Hassabis, sharing a similar glimpse of its multi-modal AI assistant. Super excited for my first #GoogleIO tomorrow and to share what we've been working on! https://t.co/hRXvNlZSrV— Demis Hassabis (@demishassabis) May 13, 2024 Not just Google, but many were quick to point out the end of many AI startups offering similar solutions and features. “OpenAI just shot Rabbit in the face,” said AI developer Benjamin De Kraker. Interestingly, OpenAI also announced the launch of the GPT-4o API, which developers can use to build new products and solutions. ok we got the API. the game is not killing each other, it's everyone working on the same foundations. once the foundations are solid, the building on top of it will be much different. pic.twitter.com/FDyqlEZzOb— Jesse Lyu (@jessechenglyu) May 13, 2024 Meanwhile, Hume AI, which released EVI (Empathetic Voice Interface), also felt the pressure, making them launch its API today, alongside other future improvements. The future of empathic AI is looking bright! Emotionally intelligent AI will be standard for all voice-based applications in the future. EVI is available today through our API, so you can start building immediately. We have a lot of exciting improvements coming soon — this is…— Hume (@hume_ai) May 13, 2024 Improves Non-English Language Performance Interestingly, OpenAI has also expanded its language capabilities, supporting over 50 languages, including Indian languages. GPT-4o has significantly optimised token usage for Indian languages, reducing Gujarati by 4.4x, Telugu by 3.5x, Tamil by 3.3x, and Marathi and Hindi by 2.9x. We also have significantly improved non-English language performance quite a lot, including improving the tokenizer to better compress many of them: pic.twitter.com/hE92x1qmM1— Greg Brockman (@gdb) May 13, 2024 GPT-4o can engage in natural, real-time voice conversations and has the ability to converse with ChatGPT via real-time video. It also understands the emotional tone of the speaker and can adjust its tone and modulation accordingly. Moreover, the latest model can understand and discuss images, allowing users to take a picture of a menu in a foreign language and translate it, learn about the food’s history and significance, and receive recommendations. One Step Closer to Autonomous Agents Another interesting update was OpenAI’s announcement of the ChatGPT (GPT-4o) desktop app, which can read your screen in real-time. The app allows for voice conversations, screenshot discussions, and instant access to ChatGPT. desktop app and new UI pic.twitter.com/k8ukzCCeH4— Sam Altman (@sama) May 13, 2024 When will GPT-4 ‘Omni’ Arrive? (Source: X) GPT-4o’s text and image capabilities are starting to roll out today in ChatGPT. Developers can now access GPT-4o in the API as a text and vision model. The company is rolling out GPT-4o to ChatGPT Plus and Team users, with Enterprise users to follow soon. ChatGPT Free users will also have access to advanced tools, including features like GPT-4 level intelligence, web responses, data analysis, and file uploads. However, ChatGPT Free users will have a message limit, which will increase as usage and demand grow. When the limit is reached, the app will automatically switch to GPT-3.5 to ensure uninterrupted conversations.Last but not least, the company has also introduced a simplified look and feel for ChatGPT, featuring a new home screen, message layout, and more. The new design is designed to be friendlier and more conversational.
GPT-4o has significantly optimised token usage for Indian languages, reducing Gujarati by 4.4x, Telugu by 3.5x, Tamil by 3.3x, and Marathi and Hindi by 2.9x.
At the OpenAI Spring Update, OpenAI CTO Mira Murati unveiled GPT-4o, a new flagship model that enriches its suite with ‘omni’ capabilities across text, vision, and audio, promising iterative rollouts to enhance both developer and consumer products in the coming weeks. 
["OpenAI"]
["Go", "ChatGPT", "API", "OpenAI", "AI", "neural network", "GPT-4o", "ML", "autonomous agents", "R"]
["AI", "ML", "neural network", "GPT-4o", "ChatGPT", "OpenAI", "autonomous agents", "R", "Go", "API"]
Siddharth is a media graduate who loves to explore tech through journalism and putting forward ideas worth pondering about in the era of artificial intelligence.