Despite some executives advocating for this shift, others, such as Microsoft's CTO, believe AI has not yet hit a scaling wall. OpenAI's o1, a model that spends more time on inference before answering a question, is an example of attempts to improve existing large language models. However, this model requires more computational power, making it slower and more expensive.
Key takeaways:
- AI leaders are reconsidering the traditional approach of using large amounts of data to train large language models, as this method may have limitations.
- Smaller, more efficient models and new training methods are gaining support in the industry, with some advocating for models that translate questions into computer code to generate answers.
- Despite concerns, some industry leaders, like Microsoft's CTO, believe that AI has not yet hit a scaling wall and that there are still benefits to be gained from scaling up.
- OpenAI's new model, o1, is designed to better handle quantitative questions and spends more time on inference before answering a question, but it requires more computational power, making it slower and more expensive.