The findings underscore the risk of introducing new factual knowledge through fine-tuning, suggesting that large language models primarily acquire factual knowledge during pre-training. Fine-tuning, on the other hand, helps them utilize this pre-existing knowledge more efficiently. The study supports the view that the introduction of new knowledge during fine-tuning can lead to the generation of ungrounded facts.
Key takeaways:
- Large language models may struggle to acquire new factual knowledge through fine-tuning, learning new information significantly slower than information consistent with the model's pre-existing knowledge.
- As the examples with new knowledge are eventually learned, they linearly increase the model's tendency to hallucinate, or generate factually incorrect responses.
- There is a risk in introducing new factual knowledge through fine-tuning, as it can lead to the generation of incorrect information.
- Large language models mostly acquire factual knowledge through pre-training, and fine-tuning helps them to use this knowledge more efficiently.