Meta also developed its own test set covering various use cases, where Llama 3 70B came out on top against other models. The company has used a much larger dataset for training, seven times the size of the Llama 2 training set, and has also used synthetic data to create longer documents for training. Meta claims to have improved in areas of toxicity and bias, common problems with generative AI models, by developing new data-filtering pipelines and updating its AI safety suites. The Llama 3 models are now available for download and will soon be hosted on various cloud platforms.
Key takeaways:
- Meta has released two new models in its Llama 3 series of open source generative AI models, which are described as a major leap in performance compared to the previous Llama models.
- The new models, Llama 3 8B and Llama 3 70B, have performed well on popular AI benchmarks, outperforming other open source models like Mistral’s Mistral 7B and Google’s Gemma 7B on at least nine benchmarks.
- Meta has developed new data-filtering pipelines to improve the quality of its model training data and updated its generative AI safety suites to prevent misuse and unwanted text generations.
- The Llama 3 models are now available for download and are being used to power Meta’s AI assistant on various platforms. In the future, versions of the models optimized for hardware from various companies will be made available.