The article further discusses the rise of open models and the shift from "source" to "weights". The author suggests that for a model to be truly open source, it would need to share all its training code, pre-training dataset, fine-tuning preferences, etc. However, due to the cost-prohibitive nature of training models from scratch, having access to the final weights is preferred. The author concludes by stating that the terms "open source" and "open weights" will be used interchangeably in the future, and that the openness of the work is what's most important.
Key takeaways:
- The author argues that while LLaMA2 isn't technically open source due to certain restrictions, it doesn't matter because the term "open source" is evolving, especially in the world of AI models.
- The history of the open source movement is traced, highlighting the tension between commercial interests and the ethos of free and open software. The author notes that the term "open source" is increasingly becoming synonymous with "source available".
- With the rise of open models, the term "open source" is often used to mean "downloadable weights". The author suggests that for a model to be truly open source, creators would need to share all their training code, pre-training dataset, fine-tuning preferences, etc. However, the cost of training runs makes this prohibitive for most developers and companies.
- The author categorizes different levels of openness in the LLMs space, including open models, open weights, restricted weights, and contaminated weights. The author predicts that the terms "open source" and "open weights" will continue to be used interchangeably, and that this is acceptable as long as work is done as openly as possible.