The metric can be computed based on the model's output obtained through special iterative prompting based on previous responses. This allows for the detection of hallucinations (high epistemic uncertainty) in both single and multi-answer responses. This is unlike standard uncertainty quantification strategies that cannot detect hallucinations in multi-answer cases. The study also sheds light on how the probabilities assigned to a given output by an LLM can be amplified by iterative prompting.
Key takeaways:
- The study explores uncertainty quantification in large language models (LLMs), aiming to identify when uncertainty in responses given a query is large.
- The research considers both epistemic and aleatoric uncertainties, deriving an information-theoretic metric to detect when only epistemic uncertainty is large, indicating unreliable model output.
- The condition for large epistemic uncertainty can be computed based solely on the model's output obtained by some special iterative prompting based on previous responses.
- The study's approach allows for the detection of hallucinations (cases when epistemic uncertainty is high) in both single- and multi-answer responses, which is not possible with many standard uncertainty quantification strategies.