Apple's upgraded AI models underwhelm on performance

Apple has announced updates to its AI models, which power features across iOS and macOS. However, according to Apple's benchmarks, these models underperform compared to older models from competitors like OpenAI. The "Apple On-Device" model, which operates offline on devices such as the iPhone, was rated by human testers as comparable to models from Google and Alibaba but not superior. The more advanced "Apple Server" model, designed for data centers, was rated lower than OpenAI's GPT-4o. Additionally, in image analysis tests, Meta's Llama 4 Scout model was preferred over Apple Server, highlighting Apple's struggle to keep pace with competitors in AI development.

Despite these challenges, Apple has made its models available to third-party developers through the Foundation Models framework. The Apple On-Device model, with approximately 3 billion parameters, supports features like summarization and text analysis. Both Apple On-Device and Apple Server models have improved tool-use and efficiency, understanding around 15 languages due to an expanded training dataset. However, Apple's AI capabilities have been underwhelming, with delays in promised upgrades like Siri, leading to customer dissatisfaction and lawsuits over undelivered AI features.

Key takeaways:

Apple's new AI models underperform compared to older models from rivals like OpenAI, Google, and Alibaba according to Apple's benchmarks.
The Apple On-Device model, which runs offline, is rated comparably to similarly-sized models from Google and Alibaba, but not better.
Apple's Server model, designed for data centers, is rated behind OpenAI's GPT-4o and Meta's Llama 4 Scout in text and image analysis tests.
Apple's AI models, available to third-party developers, have improved tool-use and efficiency, understanding around 15 languages with an expanded training dataset.

Apple's upgraded AI models underwhelm on performance | TechCrunch

Key takeaways:

Comments (0)

Newsletter