Researchers trained an OpenAI rival in half an hour for less than $50

Researchers from Stanford and the University of Washington have developed a low-cost AI reasoning model named s1, which rivals OpenAI's models. Using a method called distillation, they refined s1 with answers from Google's AI model, Gemini 2.0 Flash Thinking Experimental, despite Google's terms prohibiting such use. The model was trained on a small dataset of 1,000 questions using 16 Nvidia H100 GPUs, costing under $50. Techniques like test-time scaling were employed to enhance the model's reasoning capabilities, allowing it to double-check and correct its answers.

The s1 model, based on Alibaba Cloud's open-source Qwen2.5, reportedly outperforms OpenAI's o1-preview on competition math questions by up to 27%. This development highlights the potential for smaller, cost-effective AI models to challenge industry giants like OpenAI, Microsoft, Meta, and Google, suggesting that massive investments in AI infrastructure may not be necessary. The emergence of such models could significantly disrupt the AI industry by demonstrating that high performance can be achieved without extensive resources.

Key takeaways:

Researchers created a low-cost AI reasoning model called s1 in just 26 minutes using a small dataset of 1,000 questions and for under $50.
The s1 model was refined using distillation from Google's AI reasoning model, Gemini 2.0, despite Google's terms prohibiting such use.
The model uses test-time scaling to improve reasoning, allowing it to double-check answers by adding "Wait" to responses.
The success of smaller, cheaper AI models like s1 challenges the need for major companies to spend billions on AI development.

Researchers trained an OpenAI rival in half an hour for less than $50

Key takeaways:

Comments (0)

Newsletter