Superposition is a phenomenon where an entity exists in multiple states simultaneously. But what does this have to do with LLMs?...
In this live coding session we will explore superposition by combining two distinct pieces of text into a single superposed text. Then, we'll test if a frozen LLM can still access and utilize the information from the original texts, when exposed to the superposed text! If the model succeeds in the task, it means that superposition is inherent to the way LLMs work.
Join us as we uncover the hidden workings of these AI systems!