In response to this, OpenAI has formed a foundations team to explore ways to continue enhancing its models despite a decreasing amount of new training data. Some strategies being considered include training Orion on synthetic data generated by AI models and refining models during the post-training process. OpenAI has not confirmed plans to release Orion this year.
Key takeaways:
- OpenAI's next flagship model, code-named Orion, might not represent as significant an advancement as its previous models, according to a report in The Information.
- Employees who tested Orion found that its performance exceeds existing models, but the rate of improvement seems to be slowing down.
- Orion might not be reliably better than previous models in some areas, such as coding.
- In response to this, OpenAI has created a foundations team to explore new strategies for improving its models, including training Orion on synthetic data produced by AI models.