Johnson compares early language models to the case of Henry Molaison, a man with a severe short-term memory deficit, suggesting that these models were similarly trapped in a perpetual present, unable to assimilate new information or carry on a coherent conversation. However, with the expansion of the context window, these models have become more reliable and capable of complex tasks such as hosting factually-grounded role-playing games, answering intricate questions about a book's narrative techniques, and even generating a playable and faithful interactive adventure based on a book.
Key takeaways:
- Language models like Gemini Pro 1.5 can transform linear narratives into interactive games, using a large context window to maintain a coherent narrative and respond to user inputs.
- The expansion of the context window in AI models has significantly improved their ability to maintain continuity in conversations and accurately retrieve information from large bodies of text.
- The author uses the example of his book, _The Infernal Machine_, to demonstrate how a language model can understand and interact with a complex narrative, even identifying literary techniques such as foreshadowing.
- Despite the lack of consciousness or self-awareness in AI, the ability of these models to understand and respond to complex narratives suggests a level of understanding similar to human comprehension.