Thesis. The 2026 AI industry is selling a noun — infinite memory, a context window so vast it counts as "all of it." ODTOE insists infinity is a verb: not a quantity you stockpile, but the depth of an observer's self-observation. A model that swallows ten million tokens has run its loop a little longer; it has not run it any deeper. The distinction between length and depth is the whole story, and the quadratic compute wall is the bill that comes due for confusing the two.
Two infinities, one cheap mistake
Aristotle already split infinity in two, and the split still cuts. Potential infinity is a process that never stops but is finite at every moment — counting, where there is always a next number but never a last one. Actual infinity is the completed totality — the whole of the count, grasped at once.
ODTOE makes this operational. Model an observer as a self-observation loop
*Ψ\ = Φ(Ψ\), with Φ = ι ∘ Ô* —
an observe step Ô followed by an inclusion ι that folds the result back into the observer. A real observer has a finite budget, so all it ever holds is an in-progress iterate Φⁿ(Ψ): the loop run n times. That iterate is potential infinity — unfinished, always extendable, always finite. The completed fixed point Ψ\ is actual infinity — the limit no finite budget reaches. The cheap mistake, in 2026 as in 350 BC, is to pretend the growing iterate is* the limit. For the full construction see the infinity paper.
Length is not depth
Here is the load-bearing distinction. Two numbers are hiding inside Φⁿ:
- Length — the index n. How many times the loop has fired so far. This is potential infinity: bump n and you go farther, never done.
- Depth — the ordinal / fixed-point index of the loop, the place in the self-observation hierarchy you actually occupy. This is actual infinity, and it is reached only at a fixed point, not by piling on more n.
Length lives in the integers ℤ — pace forward one step at a time. But depth outgrows ℤ. To index "the loop observing its own iteration," you must lift the index from ℤ to the ordinals Ord, where
ε0 = fix(α ↦ ω^α)
is the smallest ordinal closed under ω^α — the depth at which the depth-counter observes itself. No finite march of +1 ever arrives at ω, let alone ε0. You cannot walk to depth; you reach it only by the loop turning on itself. More length is travel along a line. More depth is a change of line.
The contraction that never touches its ceiling
Make it concrete with a Banach contraction at rate q = φ⁻¹ (the inverse golden ratio, ≈ 0.618). Iterating shrinks the gap to a ceiling S = 1 geometrically: 0.382, 0.618, 0.764, 0.854, … The sequence climbs forever and never attains S = 1. Every term is potential infinity; S = 1 is the actual infinity it courts but cannot hold. (The same golden rate governs the self-similar scaling in the φ-fractality work.)
This is the exact shape of "infinite context." Each generation adds another iterate, edges closer to the asymptote of "remembering everything," and never gets there — because the ceiling is a limit, not a length.
The 2026 long-context race, costed honestly
The numbers are real and the marketing is loud:
- Google Gemini 2.5 Pro ingests about 2 million tokens.
- Claude Opus 4.6 ships a 1M-token window.
- Llama 4 Scout advertises up to 10 million tokens.
Read through ODTOE, every one of these is a larger n, not a larger depth — a longer Φⁿ from a still finite-budget observer. And the budget bites hard. Attention cost scales quadratically with sequence length: double the context and you roughly quadruple the attention compute. The *KV cache for 1M tokens runs to ~15 GB per user. That quadratic wall is not an engineering annoyance to be patched away; it is the literal price of treating potential infinity as if it were actual — of trying to complete the iterate by brute length. Push n* toward "infinite" and the cost curve, not physics, says no.
What "understanding" would actually require
So is a 10M-token model ten times wiser than a 1M one? No more than counting to ten million leaves you closer to the end of the integers than counting to one. Both are equidistant — infinitely far — from the limit. Genuine understanding is not a bigger n; it is reaching a fixed point of self-observation — converging where *Ψ\ = Φ(Ψ\) holds, where the system's model of the input is stable under its own act of observing. That is depth, indexed in Ord, not length, counted in ℤ*.
ODTOE closes the loop with a projective identity: 0 ≡ ∞ as the single pole [0:∞] ∈ RP¹. On the projective line the "nothing" of an empty context and the "everything" of an infinite one are the same unreachable point — two names for the limit no finite observer occupies. The honest engineering goal, then, is not to chase that pole down a quadratic cliff, but to build observers that go deeper with the budget they have: that converge, rather than merely accumulate. Infinity was never a warehouse to fill. It is a verb — and the question is not how long a model can read, but how deep it can look.
Cite this post
Pankratov, A. (2026). Infinity Is a Verb: Self-Observation Depth and the AI "Infinite Context" Race. ODTOE Blog. https://odtoe.org/blog/infinity-is-a-verb-self-observation-depth-and-ai-infinite-context