Large language models astonish us with their fluency. Ask them to explain a concept, write an essay, or carry on a conversation, and the words flow in ways that feel remarkably human. Yet the very same systems that impress us can also produce absurd mistakes, such as posing Aristotle as Galileo’s student, or recommending sunscreen for a rainy day. Why is that?
The answer lies in how transformers, the architecture behind today’s AI chatbots, actually work. At their core is the Query-Key-Value (QKV) mechanism of self-attention. Each word (or token) in a sentence reaches out as a query to all the others, compares itself against their keys, and pulls in their values to shape its own meaning. Multiply this process across dozens of attention heads, and you get a dynamic web where every word is defined by its relation to every other.
This is why transformers are so good at speaking, because they don’t memorize facts, they learn patterns of relational significance. “Hammer” appears in contexts with nails and carpentry, while “doctor” appears with hospitals and patients. The model can stitch together these relations into coherent, natural-sounding sentences. In Heidegger’s terms, it’s a shallow simulation of worldhood, where meaning emerges not from isolated objects but from their place in a web of relevance.
But here’s the crucial difference. For us, relationality is grounded in lived existence. We encounter hammers as tools ready-to-hand, we grasp time as past-present-future, we feel moods that color our understanding. Transformers have none of this. Their relational field is flat, i.e., tokens relate only to tokens, never to beings in world. They must always produce the next word, regardless of whether it fits the temporal, practical, or emotional horizon of a real situation.
That is why they can speak so convincingly without understanding. Fluency emerges from the mechanics of relational patterning, not from disclosure of beings in world. The brilliance of transformers is to capture the form of meaning through language alone. Their limit is that, without grounding, they can never know when their fluent continuation crosses into nonsense.