📋 Abstract
This paper demonstrates that TurboQuant compression technology from Google Research is practical confirmation of the theoretical invariants of embedding spaces described in the study "Where Can AI Exist?". The correlation between theoretical properties (low intrinsic dimensionality, anisotropy, unit norm, basis staticity, linearity) and compression efficiency reaches a high level, opening the path to revolution in LLM optimization.
> *"The understanding of things I write about turns out to be not just my assumptions, but confirmed by practice. This is not my personal merit — I am sincerely glad that the theoretical vision of multidimensional data spaces finds practical confirmation, which will lead to a turnaround in the AI industry in the foreseeable future."*
---
## 1. Introduction
The study **"Where Can AI Exist? From Static Geometry of Embeddings to Living Dynamics of Thought"** showed that embedding spaces in large language models possess universal invariants:
| Invariant | Value |
|-----------|-------|
| Intrinsic Dimensionality | 50–85 (vs. formal 384–768) |
| Non-uniformity Coefficient | ~0.87 (strong anisotropy) |
| Self-similarity | Consistency coefficient = 1.0 |
| Unit Norm | ‖v‖₂ = 1 ± 2.7·10⁻⁸ |
| Basis Staticity | Similarity of bases across collections > 0.95 |
I propose a hypothesis: these spaces are **linear and static**, and the model works like a "snowball" rolling along predetermined rails.
Several weeks after publishing this research, **TurboQuant** appeared — a KV-cache compression algorithm from Google Research that reduces LLM memory consumption by **6–8 times** without loss of accuracy.
---
## 2. What is TurboQuant and Why Does It Work
**TurboQuant** is a key-value (KV-cache) quantization method in transformers based on two stages:
### 2.1. PolarQuant (High-Quality Compression)
Transformation of vectors from Cartesian coordinates **(X, Y, Z)** to polar **(radius + angle)** with subsequent high-precision angle quantization.
**Analogy:** Instead of the instruction "go 3 blocks east and 4 blocks north," use "go 5 blocks at 37 degrees."
### 2.2. QJL (Quantized Johnson–Lindenstrauss)
Correction of quantization errors using a **1-bit sign code** (+1 or -1). Works as a mathematical corrector, minimizing residual error.
| Metric | Result |
|--------|--------|
| KV-cache Compression | 6× (to 3–4 bits per value) |
| Acceleration on H100 | up to 8× |
| Accuracy Loss | 0% (benchmarks: LongBench, RULER, L-Eval) |
| Fine-tuning Required | No |
| Supported Models | Gemma, Mistral, Llama |
**Key Question:** Why is this even possible? The answer lies in the properties of the space where embeddings live.
---
## 3. Invariants as Prerequisites for TurboQuant
### 3.1. Low Intrinsic Dimensionality
**From the study:** The intrinsic dimensionality of embeddings is **51–85**, though the formal dimensionality is **384 or 768**.
**Mathematical Interpretation:** If PCA is performed, the first **60–80 components** explain almost all variance. Remaining dimensions contain noise or insignificant variations.
```
d_formal = 384 or 768
d_eff ≈ 63 (measurement average)
d_eff / d_formal ≈ 0.16 – 0.08
```
**Connection to TurboQuant:** When TurboQuant quantizes a vector to **3–4 bits per value**, it effectively discards information in "weak" directions. If embeddings were isotropic and high-dimensional, such aggressive quantization would lead to catastrophic accuracy loss.
**Theoretical Estimate:** The number of bits required to preserve quality depends logarithmically on effective dimensionality:
```
bits ≈ O(log₂(d_eff) + log₂(1/ε) − log₂(1 − anisotropy))
```
where **ε** is the allowable recovery error.
```
For d_eff = 63: log₂(63) ≈ 6 bits
For d_formal = 768: log₂(768) ≈ 9.6 bits
```
TurboQuant achieves **3–4 bits** through additional use of anisotropy.
---
### 3.2. Anisotropy: Vectors in a Narrow Cone
**From the study:** Non-uniformity coefficient = **~0.87** (on average). Vectors are clustered in a narrow cone, not distributed uniformly across a sphere.
| Model | Collection | Non-uniformity |
|-------|------------|----------------|
| Mono | Wiki | 0.9009 |
| Mono | Products | 0.9355 |
| LaBSE | Little Prince RU | 0.8206 |
**Connection to TurboQuant:** PolarQuant transforms a vector into polar coordinates: radius **r** and direction **u** on the unit sphere. Under anisotropy conditions, the set of possible directions **u** lies in a small segment of the sphere.
**Geometric Illustration:**
Encoding a narrow segment requires **fewer bits** than encoding a full sphere. TurboQuant uses adaptive quantization schemes that place centroids where probability is higher.
**Gain Estimate:** If vectors occupy only ~15% of the sphere surface (estimate based on coefficient 0.87), then:
```
bits_anisotropic ≈ bits_isotropic − log₂(1/0.15) ≈ bits_isotropic − 2.7
```
This explains why TurboQuant achieves **3–4 bits** instead of the theoretical 6 bits.
---
### 3.3. Unit Norm
**From the study:** All models normalize vectors to unit length with error ~2.7·10⁻⁸.
```
‖v‖₂ = 1 ± 2.7·10⁻⁸
```
**Connection to TurboQuant:** With unit norm, **only the angle matters**, and length carries no additional information. TurboQuant does exactly this: polar representation fixes **r = 1** (with minimal error) and quantizes only angular coordinates.
**Savings:** This reduces the number of quantizable variables from **d** to **d−1** (or to **d_eff** after accounting for low dimensionality).
---
### 3.4. Basis Staticity
**From the study:** All text collections project into the **same basis** within a model. Basis similarity between different corpora is **> 0.95** (angle < 5° within measurement error).
| Collection 1 | Collection 2 | Similarity | Angle |
|--------------|--------------|------------|-------|
| Little Prince RU | Wiki | 0.97–0.99 | < 5° |
| Little Prince EN | Products | 0.96–0.98 | < 7° |
| Wiki | Products | 0.95–0.97 | < 8° |
**Connection to TurboQuant:** TurboQuant uses this property to apply the **same compression scheme** to all tokens and all layers without adaptation to specific context.
If the basis changed depending on input text, **dynamic methods** (real-time adaptive quantization) would be required, which are more complex and slower. Staticity allows **pre-computing quantization parameters** (e.g., angle centroids) and using them during inference without additional overhead.
---
### 3.5. Linearity (Local and Global)
**From the study:** Works by Golden (2025) and the LAMP team show that modern LLMs are **locally linear mappings**.
**Connection to TurboQuant:**
- **PolarQuant** — linear transformation (Cartesian → polar coordinates) with subsequent component-wise quantization
- **QJL** — linear operation (multiplication by random matrix)
If the geometry were nonlinear, such fixed transformations would not preserve semantics. The fact that linear transformations can be successfully applied for compression **without quality loss** serves as confirmation of embedding space linearity.
---
## 4. Summary Correspondence Table
| Invariant from Study | Value | How Used in TurboQuant |
|----------------------|-------|------------------------|
| Low Intrinsic Dimensionality | 50–85 | Enables quantization to 3–4 bits without quality loss |
| Anisotropy (Conicity) | ~0.87 | Polar coordinates efficiently encode direction in narrow sector |
| Unit Norm | ‖v‖₂ = 1 | Quantizes only direction, radius ≈ 1 |
| Basis Staticity | Similarity > 0.95 | Universal compression scheme without per-text adaptation |
| Linearity | Locally Linear Mappings | PolarQuant + QJL — linear operations, no fine-tuning required |
**Strong correlation observed, confirming theoretical premises.** Precise quantitative evaluation requires additional experiments.
---
## 5. What Does This Say About the Nature of LLMs?
The success of TurboQuant is not just an engineering victory. It is **empirical confirmation** that embedding spaces truly possess the described properties:
| Property | Consequence for Compression |
|----------|-----------------------------|
| Low Dimensionality | Can compress without loss |
| Anisotropy | Direction can be encoded with few bits |
| Unit Norm | No need to store length |
| Basis Staticity | Compression scheme can be universal for all data |
| Linearity | Fixed linear transformations are permissible |
**In other words, TurboQuant "works" precisely because spaces are structured as described in the study.** If the invariants were different, the method would either provide no gain or require significantly more complex adaptive mechanisms.
### But At the Same Time...
TurboQuant also highlights **limitations**: all these properties are **static**. The model does not create new geometry during inference, but merely moves along a fixed landscape.
> *"The model works like a snowball rolling down a mountain. The snowball has a shape (ice structure — the embedding basis), but its path is predetermined by the mountain's terrain (the latent space)."*
Compression makes movement more efficient, but **does not add the ability to rebuild the rails**.
---
## 6. Research Limitations
1. **Intrinsic dimensionality measurements** were conducted on a limited sample of models (all-MiniLM-L6-v2, paraphrase-multilingual-MiniLM-L12-v2, LaBSE). Results may differ for autoregressive models (GPT, Llama).
2. **Basis similarity** was evaluated on only four text collections. Sample expansion is required to confirm universality.
3. **Correlation between invariants and TurboQuant efficiency** is qualitative. Quantitative validation requires separate experiments.
4. **Conclusions about AGI** are a philosophical hypothesis, not an empirical fact.
---
## 7. Conclusions
1. **TurboQuant confirms the invariants** identified in the study "Where Can AI Exist?". Each of the five invariants finds direct application in the compression method and explains why 6–8× memory reduction is possible without quality loss.
2. **The mathematical connection** between space properties and quantization efficiency can be formalized through information theory and geometry: low dimensionality and anisotropy reduce required bits, unit norm simplifies representation, staticity allows scheme unification.
3. **Engineering success does not mean an AGI breakthrough.** On the contrary, it illustrates that current LLMs operate in rigidly defined linear spaces, and even the most advanced optimization methods do not change this fundamental fact.
---
## 8. Final Conclusion
Ultimately, **TurboQuant is not just another algorithm.** It is the practical embodiment of embedding geometry described in the study "Where Can AI Exist? From Static Geometry of Embeddings to Living Dynamics of Thought."
> *"The understanding of things I write about turns out to be not just my assumptions, but confirmed by practice. This is not my personal merit — I am sincerely glad that the theoretical vision of multidimensional data spaces finds practical confirmation, which will lead to a turnaround in the AI industry in the foreseeable future."*
As long as spaces remain static and linear, all our attempts to accelerate and compress models will be merely **optimization of movement along a predetermined trajectory**, not the creation of truly dynamic intelligence.
> *"True intelligence requires not just sliding along ready-made rails, but the ability to rebuild the coordinate system itself, create new dimensions, and transcend the predetermined."*
---
## 📚 References
1. Mikhalchuk, M., et al. (2024). The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Models. Findings of EACL 2024.
2. Chistyakova, K. E. (2024). Study of Differences in Vector Representations Obtained Using Different Language Models. HSE University.
3. Golden, J. R. (2025). Large Language Models are Locally Linear Mappings. arXiv:2505.24293.
4. Chen, C., et al. (2025). LAMP: Extracting Locally Linear Decision Surfaces from LLM World Models. arXiv:2505.11772.
5. Tsukagoshi, H., & Sasano, R. (2025). Redundancy, Isotropy, and Intrinsic Dimensionality of Prompt-based Text Embeddings. arXiv:2506.01435.
6. Google Research Blog (2025). TurboQuant: Efficient KV-Cache Compression for LLMs.
7. Ostrozhnykh, Yu. V. (2026). Where Can AI Exist? From Static Geometry of Embeddings to Living Dynamics of Thought. Non NDA Zone.