LightGPT

LightGPT-instruct-6B Overview

LightGPT-instruct-6B is a transformer-based language model developed by AWS Contributors. It is fine-tuned from EleutherAI's GPT-J 6B model and trained on a high-quality Apache-2.0 licensed instruction dataset. The model is designed to generate responses to instructions in English and can be deployed on Amazon SageMaker. However, it may struggle with long inputs and often gives incorrect answers to math and reasoning questions.

LightGPT-instruct-6B Highlights

• Developed by AWS Contributors and fine-tuned from EleutherAI's GPT-J 6B model.
• Trained on a high-quality Apache-2.0 licensed instruction dataset with approximately 200K examples.
• Can be deployed on Amazon SageMaker for practical applications.