The new models, known as The Stack, have been trained on a dataset seven times larger than the previous one and can understand and generate low-resource programming languages. The smallest model was trained using ServiceNow’s Fast LLM framework, the 7B model with Hugging Face’s nanotron framework, and the largest model with Nvidia's NeMo cloud-native framework and TensorRT-LLM software. The models can be used for various tasks, including application source code generation, workflow generation, and text summarization.
Key takeaways:
- Nvidia, Hugging Face and ServiceNow have developed StarCoder2, a new family of open-access large language models (LLMs) for code generation, available in three different sizes and trained on over 600 programming languages.
- The models have been developed under the open BigCode Project, a joint effort of ServiceNow and Hugging Face, and are available royalty-free under Open Responsible AI Licenses (OpenRAIL).
- The models can understand and generate low-resource programming languages and can be used for a variety of tasks including application source code generation, workflow generation, text summarization, code completion, advanced code summarization and code snippets retrieval.
- The models are available on the BigCode project’s GitHub repository and can also be downloaded from Hugging Face. The 15B model trained by Nvidia is also available on Nvidia AI Foundation.