Sign up to save tools and stay up to date with the latest in AI
bg
bg
1

Training Large Language Models to Reason in a Continuous Latent Space

Dec 10, 2024 - arxiv.org
The article introduces a new reasoning paradigm called Coconut (Chain of Continuous Thought) for large language models (LLMs), which operates in an unrestricted latent space rather than the traditional language space. This approach utilizes the last hidden state of the LLM as a representation of the reasoning state, termed "continuous thought," and feeds it back into the model as the next input embedding. This method allows the model to explore multiple alternative reasoning paths simultaneously, akin to a breadth-first search, rather than committing to a single deterministic path as in the chain-of-thought (CoT) approach.

Experiments demonstrate that Coconut can effectively enhance LLM performance on various reasoning tasks, particularly those requiring significant backtracking and complex planning. Coconut outperforms CoT in specific logical reasoning tasks with fewer thinking tokens during inference, highlighting the potential of latent reasoning. The findings suggest that this novel approach can lead to advanced reasoning patterns and offer valuable insights for future research in the field of LLM reasoning.

Key takeaways:

  • Large language models (LLMs) are traditionally limited to reasoning within the "language space," which may not always be optimal for complex reasoning tasks.
  • The new paradigm, Coconut (Chain of Continuous Thought), utilizes the LLM's last hidden state as a "continuous thought" to represent the reasoning state, bypassing the need for word token decoding.
  • Coconut allows for a breadth-first search (BFS) approach in reasoning, enabling the model to explore multiple alternative reasoning steps simultaneously.
  • Experiments show that Coconut outperforms traditional chain-of-thought (CoT) reasoning in certain logical tasks, particularly those requiring substantial backtracking, with fewer thinking tokens during inference.
View Full Article

Comments (0)

Be the first to comment!