We live in three dimensions, yet AI thinks in thousands. The geometry that Euclid codified for flat planes extends into spaces with 768, 4096, even 12,288 dimensions — and in those vast spaces, meaning itself has shape. The journey from ancient geometry to modern latent spaces reveals that mathematics can describe worlds far beyond what our eyes can see.
The Timeline
Euclid of Alexandria
Euclid’s five postulates built all of geometry from the ground up. For 2,000 years, geometry meant Euclidean geometry. The fifth postulate — the parallel postulate (through a point not on a line, exactly one parallel exists) — seemed obvious. Attempts to prove it would eventually shatter our understanding of space itself.
Euclid’s distance formula extends directly to any number of dimensions. In 768-D space: $d = \sqrt{\sum_{i=1}^{768}(a_i - b_i)^2}$
René Descartes
Descartes merged algebra and geometry by introducing coordinates. Every geometric shape became an equation; every equation became a shape. This union — analytic geometry — made it possible to do geometry with numbers, which is exactly what computers need.
Descartes’ insight: geometry IS algebra with a different notation. This duality is why neural networks (algebra) can learn geometric structures.
János Bolyai, Nikolai Lobachevsky, Carl Friedrich Gauss
What if Euclid’s parallel postulate is wrong? Lobachevsky and Bolyai independently discovered consistent geometries where multiple parallels exist (hyperbolic) or none exist (spherical/elliptic). Gauss had known secretly for years. This shattered the belief that Euclidean geometry was the only geometry — opening the door to curved spaces and, eventually, the geometry of neural network loss landscapes.
Euclidean: angles sum to $180°$. Hyperbolic: $< 180°$. Spherical: $> 180°$
Hyperbolic geometry is now used in AI: Poincaré embeddings represent hierarchical data (like taxonomies) more efficiently than Euclidean space.
Bernhard Riemann
In his legendary 1854 lecture, Riemann generalized geometry to spaces of any dimension and any curvature. A manifold is a space that locally looks flat (Euclidean) but globally can be curved. Riemann’s work led to Einstein’s general relativity — and today, the manifold hypothesis is central to understanding how AI organizes knowledge.
Riemann was 28 when he revolutionized geometry. Einstein used Riemannian geometry 60 years later for general relativity. AI uses it 170 years later for understanding data.
Richard Bellman
Bellman discovered that as dimensions increase, space becomes overwhelmingly empty. In 100 dimensions, 99.99% of a hypercube’s volume is in a thin shell near the surface. Points that seem nearby in low dimensions are astronomically far apart in high dimensions. Data becomes sparse, distance becomes meaningless, and traditional algorithms fail catastrophically.
The curse of dimensionality says high-dimensional spaces are mostly empty. Yet LLMs thrive in spaces with thousands of dimensions. How? The answer is the manifold hypothesis.
Yann LeCun, Geoffrey Hinton, Yoshua Bengio
The manifold hypothesis proposes that real-world high-dimensional data (images, text, speech) actually lies on or near a much lower-dimensional manifold. Think of it like a crumpled sheet of paper in 3D — the paper itself is 2D. Neural networks learn to “uncrumple” this manifold, finding the true low-dimensional structure hidden in high-dimensional data.
A 512×512 image lives in a space of 786,432 dimensions. But the manifold of “natural images” is estimated to have only ~100 true dimensions. Neural networks learn this structure.
Laurens van der Maaten, Leland McInnes
t-SNE (2008) and UMAP (2018) are dimensionality reduction algorithms that project high-dimensional data into 2D or 3D for visualization, preserving the manifold structure. They revealed that neural networks organize knowledge into clusters, hierarchies, and continuous spectra — showing that AI’s internal representations have beautiful, meaningful geometric structure.
When you visualize what GPT “sees” internally, t-SNE and UMAP reveal clusters: similar concepts group together, forming a geometric map of knowledge.
Modern LLM Research
The internal representations of LLMs form rich geometric structures. “King - Man + Woman = Queen” is a geometric operation in embedding space. Researchers have found that concepts like truth/falsehood, past/future, and positive/negative correspond to directions in the latent space. The geometry of meaning is becoming a scientific discipline.
Researchers at Anthropic discovered that Claude’s internal representations encode truthfulness as a linear direction in geometry — AI has a literal “compass for truth” encoded in high-dimensional space.
The Thread That Connects
From Euclid’s flat plane to 12,288-dimensional latent spaces, geometry has always been about understanding shape and structure. AI has revealed that meaning itself has geometry — that words, concepts, and ideas occupy positions in vast mathematical spaces where distance equals similarity and direction equals relationship.
Connections to Other Lectures
- Lecture 2: Linear Algebra & Transformations — Embeddings and dot products are the algebraic machinery that makes high-dimensional geometry computable.
- Lecture 9: Harmonic Analysis & Signals — Fourier methods provide spectral geometry — analyzing shape through frequencies rather than coordinates.
- Lecture 3: Calculus & Optimization — Loss landscapes are high-dimensional geometric surfaces that gradient descent navigates during training.