Meta Llama 3.1 is Officially Out, Dethrones GPT-4o
After Llama 3.1 was leaked, Meta has officially released Llama 3.1 405B, a new frontier-level open source AI model, alongside its 70B and 8B versions. Meta is offering developers free access to its weights and code, and enabling fine-tuning, distillation, and deployment. https://twitter.com/astonzhangAZ/status/1815763885380747422 The Llama 3.1 405B model performs on par with the best closed models. It supports a context length of 128k, eight languages, and offers robust capabilities in code generation, complex reasoning, and tool use. “Meta AI is on track to reach our goal of becoming the most used AI assistant in the world by the end of the year,” said Meta chief Mark Zuckerberg. He also dropped hints about the upcoming features in Meta AI. “Over the next couple of weeks, we’re also adding a couple of new features. Meta AI Imagine, which generates images as fast as you type, will also let you put yourself in your images in any style and do almost anything that you can think of. So I think that’s going to be quite a bit of fun. We’re also getting closer to releasing a tool that you can use to create your own AIs to interact with across our apps. More on that soon,” added Zuckerberg. Meta has introduced the Llama Stack API for easy integration, supported by an ecosystem of over 25 partners, including AWS, NVIDIA, Databricks, Groq, Dell, Azure, and Google Cloud. For this release, Meta evaluated performance on over 150 benchmark datasets across various languages. Extensive human evaluations were also conducted, comparing Llama 3.1 with competing models in real-world scenarios. The evaluation suggests that Llama 3.1 is competitive with leading foundation models such as GPT-4, GPT-4o, and Claude 3.5 Sonnet. Additionally, the smaller models in the Llama 3.1 series are competitive with closed and open models with similar parameter counts. With today’s launch of our Llama 3.1 collection of models we’re making history with the largest and most capable open source AI model ever released. 128K context length, multilingual support, and new safety tools. Download 405B and our improved 8B & 70B here.… pic.twitter.com/rf5yBxElYx— Ahmad Al-Dahle (@Ahmad_Al_Dahle) July 23, 2024 Zuckerberg said that this is the Linux-like moment in the world of AI. “I think that open source AI is going to become the industry standard just like Linux did. It gives you control to customize and run your own models. You don’t have to send your data to another company, and it’s more affordable,” he said. Llama 3.1 is designed to enable new workflows like synthetic data generation and model distillation. It is available for testing in the US on WhatsApp and at meta.ai, where users can pose challenging math or coding questions. Llama-3 405B is now openly available and beats GPT-4o on most metrics!– Sonnet 3.5 barely inches out 405-b!– It's the best open-source model in the world – At 128K context open source length, it is a pretty good model for RAGThanks to Meta for this excellent model!! It can… pic.twitter.com/95I3MZUGx7— Bindu Reddy (@bindureddy) July 23, 2024 The model was trained on over 15 trillion tokens using over 16,000 H100 GPUs, making it Meta’s largest and most ambitious model to date. The training process involved significant optimisations, including a standard decoder-only transformer model architecture and iterative post-training procedures. These efforts aimed to maximize training stability and improve the quality of synthetic data. To support large-scale production inference, Meta quantised the models from 16-bit to 8-bit numerics, reducing compute requirements and enabling the model to run within a single server node. This advancement is expected to drive innovation and exploration in AI applications, offering unprecedented growth opportunities.
“I think that open source AI is going to become the industry standard just like Linux did. “
“I think that open source AI is going to become the industry standard just like Linux did. "
["Meta AI"]
["Meta AI", "AWS", "AI", "GPT-4o", "RAG", "Aim", "foundation models", "Claude 3.5", "Azure", "Databricks"]
["AI", "foundation models", "GPT-4o", "Claude 3.5", "Meta AI", "Aim", "RAG", "AWS", "Azure", "Databricks"]
Siddharth is a media graduate who loves to explore tech through journalism and putting forward ideas worth pondering about in the era of artificial intelligence.