*The story of one research project: from the geometry of embeddings — through Google's TurboQuant — to the asymmetry between weights and activations, and to an unexpected understanding of the model as a dynamical system.

  • 3 models — SmolLM-135M · Qwen2.5-0.5B · TinyLlama-1.1B — three architectures, one pattern
  • 124 experiments — with technique flags; correlation analysis of "what actually worked"
  • 4.2875 bit/weight — equal bit budget for CHMC and GPTQ — an honest comparison, no hidden bonuses
  • v1 → v6: 4894× → ~1.18× — evolution of the PPL ratio on SmolLM; the turning point was methodology, not geometry

Prologue. The Question That Would Not Let Me Rest

Some time ago I wrote an article about the geometry of embedding spaces.

The main conclusion was simple and, it seemed to me, important: language-model representations are not spread uniformly across the space but gathered into a narrow cone.

They are anisotropic, low-dimensional; their intrinsic dimension is significantly smaller than the formal one.

This was not merely a theoretical construction — measurements on several models showed that 90% of activation energy is concentrated in a tiny fraction of dimensions, and the mean cosine between vectors is about 0.75 instead of the zero expected for an isotropic space.

A month after publication Google released TurboQuant (ICLR 2026). The method takes exactly these ideas — conic geometry, anisotropy, low intrinsic dimension — and applies them to the KV cache and embeddings.

Result: 6–8× compression with no loss of quality. My theory received strong industrial confirmation.

But there remained a question that TurboQuant did not answer: what about the weights?

If geometry works for activations, perhaps it also works for the model's own parameters? And more broadly: can geometric compression be applied not only to the data the model produces (activations, embeddings, KV cache) but to the model itself, to its weights?

That is how this project began. Let me say upfront: I did not try to beat GPTQ.

My goal was to understand where the boundary of applicability of the geometric approach lies and why it works so brilliantly for activations and, as it turned out, is completely useless for weights.

The answer turned out to be more beautiful than "yet another quantization method" — and this article is exactly about that answer.

Methodology. What is CHMC

CHMC — Covariance/Hessian-Aware Manifold Compression. The idea in one sentence: split each weight matrix into two parts — a low-rank part that captures the "+chr(34)+"cone"+chr(34)+" of activations (the directions where data actually lives),

and a residual quantized with per-column error compensation following GPTQ; the rank of each layer is chosen so that the total bit budget matches the GPTQ budget exactly.

This is not "+chr(34)+"yet another quantizer"+chr(34)+" — it is a geometric frontend on top of the GPTQ optimizers.

The input data are the same as in GPTQ: a calibration sample (2048 tokens of WikiText) and the activations X of each layer on it.

Everything else is computed from the weights and these activations in three steps.### Step 1. Low-rank part along the cone

We compute damped activation energies along input directions and perform a truncated SVD in the weighted norm — so that the error is small exactly where the activations are large:

c_{j} = (1/n) · Σ_{t} x²_{t,j} + λ · mean(c), λ = 0.05min_{W_lr} ‖(W − W_lr) · diag(√c)‖²_{F} ⇒ W̃ = W · diag(√c), (U, S, Vᵀ) = svd_lowrank(W̃, q=r, niter=5), W_lr = (U·S)Vᵀ · diag(1/√c)

Step 1 from the code (v6/chmc_v6.py + v5/stabilizers.py): c — damped diagonal covariance of activations; svd_lowrank — randomized SVD with power iterations (niter=5).

It is exactly here that the cone hypothesis enters the method: directions with high activation energy are approximated more accurately.

Step 2. Residual — per-column compensation following GPTQ

R = W − W_lr is quantized by an algorithm identical to GPTQ: Hessian from calibration activations, inverted via Cholesky decomposition,

columns in decreasing order of importance (act-order); after each column is quantized its error is subtracted from the remaining ones:

H = XᵀX / n + λ · mean(diag(H)) · I, H⁻¹ — via Cholesky decompositionfor column j (in act-order): q_{j} = round(col_{j}/scale)_{g} · scale_{g}, err_{j} = (col_{j} − q_{j}) / H⁻¹[j,j], R[:, j+1:] ← R[:, j+1:] − err_{j} · H⁻¹[j, j+1:]

Step 2 from the code (quantize_with_compensation_strict): group_size = 128 — a common scale per group of output channels;

strict_sequential=true means "one column at a time, full row of H⁻¹" — this is exactly the genuine GPTQ update step.

Step 3. Equal bit budget

Each layer stores: the low-rank part (U, V in FP16), the INT4 residual and group scales.

Bits per weight are counted explicitly, and the rank r_layer is chosen so that the total BPW does not exceed 4.2875 — the GPTQ level:

BPW_{CHMC} = [16·r·(o+i) + b·o·i + 16·⌈o/gs⌉·i] / (o·i) vs BPW_{GPTQ} = [b·o·i + 16·o·⌈i/gs⌉] / (o·i)r_{layer} = floor( (BPW_{tgt} · o·i − b·o·i − 16·⌈o/gs⌉·i) / (16·(o+i)) )

Step 3 from the code (v6/bpw.py): o, i — layer dimensions; b = residual_bits = 4; gs = group_size = 128.

In CHMC the scale groups run along output channels (⌈o/gs⌉·i), in GPTQ — along input ones (o·⌈i/gs⌉). The only free budget parameter is the rank r.

Result: W_comp = W_lr + R_q. And the main difference from GPTQ:

GPTQ CHMC v6
What is stored INT4 indices + group scales Low-rank part (U, V in FP16) + INT4 residual + group scales
Error minimization Per-column compensation via the Hessian of activations The same algorithm (strict_sequential=true — literally the GPTQ step)
Role of calibration data Only the Hessian H = XᵀX/n Hessian + weighted SVD of the low-rank part (activation energies c_j)
Low-rank part No Yes: SVD along the cone, per-layer rank from the bit budget
Bit budget 4.2875 BPW (GPTQModel, 4-bit / group-128) Exactly 4.2875 BPW — ranks are distributed to this target level
Table 1. CHMC v6 vs GPTQ: commonalities and differences. The residual stage of CHMC uses the same optimizer as GPTQ;

the difference is in the low-rank frontend that encodes the cone hypothesis. It is exactly this frontend that we tested for robustness.

Part 1. The cone hypothesis is confirmed — three times

Before attempting to compress weights, it was necessary to make sure that the original geometric picture is correct.

I ran a series of measurements on three models of different scale: SmolLM-135M, Qwen2.5-0.5B and TinyLlama-1.1B. Here is what came out.

Activations live in a narrow cone.

For each layer I estimated the effective dimensionality of activations — the number of principal components needed to explain 90% of the variance.

On average across the network it is 9 out of 576 dimensions† (1.56% of the space). The mean cosine between activations of one layer ≈ 0.75‡ — in an isotropic space it would be close to zero.

This is direct confirmation: representations do not fill the space but are concentrated in a narrow cone.

The weights, however, are full-rank.

The same analysis for weight matrices showed a completely different picture: the minimum effective rank of weights was 29–30, the median — about 60§.

For matrices of size, say, 576×576 this means that the weights are truly dense, without pronounced low-rank structure.

So it is the model itself that creates the cone.

If activations are narrow and weights are dense, then the mapping "dense weights → sparse activations" is a property of training, not merely a reflection of data.

The model itself projects dense information into a narrow manifold. Later, using topological analysis (persistent homology and 3D projections),

I found a nice refinement for this: the cone is a bottleneck in the middle of the network. In middle blocks up to 98.9% of variance lies in three principal components (in SmolLM the peak falls on the middle of the network;

in Qwen and TinyLlama it is reached already from the first blocks — Figure 2) — the cloud is literally rolled into a cone, while at input and output representations are spread out again (13–43% at the edges).

The pattern is identical across three different architectures. The geometry is real, and it is created by the model in the course of processing.

SmolLM-135M (d = 576)

Qwen2.5-0.5B (d = 896)

TinyLlama-1.1B (d = 2048)

Figure 1.3D projections of activations of four representative blocks (first, quarter, middle, last) onto the three principal components of the centered merged cloud [pre;post]:blue dots — original model (pre), orange — after CHMC v6 compression (post); gray lines connect the same token before and after. In panel titles — the share of variance explained by these three directions ("PCA-3D var"). In the middle of the network the cloud is rolled into a cone, at the edges it is spread out. The pattern repeats on all three architectures.

Full profile across blocks. Figure 2 and Table 2 give an honest picture for all blocks: where exactly the "middle" is depends on the model, but the pattern itself (narrow cone after the first 1–2 blocks, more isotropic at the edges) holds everywhere.

Figure 2.Left — share of energy of the three principal components in the centered cloud of activations of each block [pre;post] (the same quantity as "PCA-3D var" in Figure 1); right — mean pairwise cosine between tokens within a block (anisotropy). After the first 1–2 blocks almost the entire network lies in a low-dimensional cone: 80–99% of energy in three principal components; at both edges more isotropic (13–43%). The peak position differs across models — for SmolLM it is in middle blocks, for Qwen and TinyLlama it is reached from the first ones. pre ≈ post ≈ merged cloud: concentration is a property of the activations themselves.
Model (dimension d) Mean pairwise cosine of pre-tokens, range across blocks Top-3 PC energy share: edges → peak → decay cos(preᵢ, postᵢ) per token
SmolLM-135M (d = 576) 0.52 – 0.78 15–28% (blk 0–10) → 99.2% (blk 11) → 87% (blk 27); edges: 24.7 / 43.4% ≥ 0.98
Qwen2.5-0.5B (d = 896) 0.08 – 0.64 13–14% (blk 0–1) → 98.5% (blk 3, plateau ~97% up to blk 13) → 82% (blk 20); last three blocks: 17 / 18 / 26% ≥ 0.98
TinyLlama-1.1B (d = 2048) 0.13 – 0.38 13–14% (blk 0–1) → 98.6% (blk 2), monotonic decay to ~19% (blk 20–21) ≥ 0.98
Table 2. Anisotropy and "bottleneck" measurements (project data: decoder-block outputs on 2048 tokens, subsample of 512 points per block; cos(preᵢ, postᵢ) — cosine of the same token before/after CHMC v6 compression).

† The estimate "d90 ≈ 9 out of 576" comes from a previous study at the level of sentence embeddings.

For per-token decoder-block outputs in this project's data the profile is even more extreme: in middle blocks one principal component already explains ≥90% of energy (d90 = 1), and at the edges d90 reaches ~235.

‡ "Mean cosine ≈ 0.75" — measured on SmolLM-135M (range across blocks 0.5–0.8).

On Qwen2.5-0.5B it is lower (0.08–0.64), on TinyLlama-1.1B even lower (~0.13–0.38): the strength of anisotropy depends on the model, but the "bottleneck" pattern holds on all three.

§ Effective rank of weights — v5 convention (exp-entropy of the normalized spectrum).

The numbers quoted above are from a previous study on another subset of layers; in this project's

TDA analysis the minimum effective rank across all weight matrices is 18.7–22.1, median — 274–930 depending on model (Appendix B).

Part 2. The road from collapse to parity

So, activations are conical and weights are dense.

First thought: if the model projects data into a cone, then perhaps the weights can be compressed using information about this cone.

For example, if most of the activations lie in a low-dimensional subspace, then quantization error of the weights in directions outside that subspace should be less critical.

This was the basic intuition.

Next — a series of experiments; each version of the method added some idea. Here is a brief chronology:

Version What was added SmolLM, ratio
v1 Plain SVD 4894× 💀
v2 Adaptive rank 2.35×
v3 Honest bit accounting 1.84×
v4 Covariance + residual 1.49×
v5 7 stabilizers ~1.29×
v6 Equal bit budget + GPTQ optimizers ~1.18×
Table 3. Evolution of CHMC on SmolLM-135M: PPL ratio against GPTQ (ratio = PPL_CHMC / PPL_GPTQ; 1.0 — parity).
Figure 3.Progress of versions v1 to v6 (PPL ratio vs GPTQ, SmolLM-135M) on a logarithmic scale. The turning point was not new geometric ideas but methodological fixes: honest bit accounting and error compensation. The first version was a catastrophe: plain SVD without accounting for the structure of errors produced an enormous increase in perplexity. Gradually, by adding adaptive rank selection, correct accounting of bit cost, covariance corrections and stabilizers, I brought the method to a level comparable with GPTQ.

There were two turning points on this road, both methodological.

First: honest bit accounting

In the early stages I was cheerfully declaring "we're going to crush everyone", but at the same time I was spending 4.5 bits per weight against GPTQ's 4.29 — and still losing in quality.

Only when I introduced strict comparison under an equal bit budget (BPW) were all previous conclusions invalidated.

It turned out that without honest accounting of the budget any results table is a fiction. This is perhaps the main practical lesson of the project.

Second: statistics

When one configuration with dampening Λ=0.05 gave PPL 1.1723 against GPTQ's 1.1761, I was about to declare victory.

But three additional runs showed: mean — 1.1793, i.e., on average I lose. The "victory" was the best of four runs.

Variability of 0.007–0.009 is real, and comparison by a single run is divination. Since then I have measured every key configuration 3–4 times with different seeds and compared by the mean rather than the best run.

Result in numbers (equal budget of 4.29 bits/weight):

Model CHMC (mean) GPTQ Verdict
SmolLM-135M 1.179–1.186 1.176 parity, within noise
Qwen2.5-0.5B 1.174* 1.141 loss
TinyLlama-1.1B 1.059 1.081 real win (more than 30σ)
Table 4. Final PPL at an equal bit budget of 4.2875 BPW. *Qwen — preliminary single run, repetitions still ahead; other models — mean over 3–4 seeds.
Figure 4.PPL of the original model, CHMC v6 (mean over seeds) and GPTQ at an equal bit budget. SmolLM — parity; Qwen — loss (single run); TinyLlama — stable win.

The method is competitive: on one model it stably beats industrial SOTA, and on small models it outperforms naive scalar quantization by 6–11×

(on SmolLM, 16.48× for scalar_q4 against 1.42× for my method — at a comparable budget of ~4.0–4.2 bits/weight). But there is no "geometric superiority". Why?

Part 3. Geometric apocalypse

I tried all the geometric techniques that the literature and my theory suggested: Hadamard rotation (from QuIP#), Lloyd-Max quantizer, conical decomposition, QJL correction (from TurboQuant). Everything failed:

Technique Expected Obtained
Hadamard rotation −2–5% 2.23× (1.8× worse)
Lloyd-Max optimum for Gaussian 1.35× (worse than baseline 1.21×)
Cone-Aware exact preservation of the cone 2.15×
QJL 1-bit residual saving of 3 bits PPL 267 228 💀
Table 5. Geometric techniques and their results on SmolLM-135M (ratio against GPTQ at an equal bit budget; for QJL — absolute PPL, ratio is meaningless).

The mathematics of the failures turned out to be more instructive than the failures themselves.

QJL physically cannot store weights

Its reconstruction signs @ Uᵀ has rank no higher than the number of projections, i.e., it can represent only low-rank matrices.

The residual after a low-rank approximation is by definition not low-rank — hence QJL in principle cannot encode it. The theoretical error bound — √(1−1/4) ≈ 87%. This is an instrument for estimating norms, not a format for storing weights.

Rotation does not change the objective function but changes the quantization lattice

I proved that the loss is invariant to orthogonal rotation of the weights (this is just a change of basis).

However, after rotation the distribution of weights becomes more Gaussian, and the INT4 quantizer, which works excellently on anisotropic data, begins to work poorly.

Most importantly: data-adaptive error compensation (which GPTQ performs) in the original conical space beats any distributional optimality in the rotated one. The cone hypothesis wins against rotation precisely because it does not destroy anisotropy.

The only lever that worked — was not geometry

It is dampening (Λ=0.05), i.e., regularization of the Hessian matrix estimate. With small calibration, the estimate is noisy and the inverse matrix amplifies noise in low-energy directions.

Dampening gave −4% — more than all geometric techniques combined (they gave zero or negative). This was the first bell: what works is not capturing geometry but controlling noise.

This is confirmed by statistics over all 124 experiments of the project. I collected flags for each run — which techniques were enabled and whether the run beat GPTQ — and computed correlations.

Result: the only flag positively correlated with success is dampening (+0.40); all geometric flags (rotation, cone, Lloyd-Max, QJL) correlate negatively or near zero with success.

Geometry did not help in a single failed run, but systematically, across the whole ensemble of experiments.

Even "improving" the basic mathematics worked against me. Weighted SVD is considered a randomized algorithm with power iterations, and I checked: more iterations — more accurate decomposition — better quality?

On the contrary: 5 iterations — ratio 1.0586, 10 — 1.0610, 20 — 1.0662. Accurate approximation overfits to calibration; a slightly "inaccurate" one generalizes better.

Another stroke on the theme that "noise control matters more than structure capture" — and another argument for honest statistics: all comparisons in this article are given by mean over seeds.

Part 4. Drift and the last dead end

There remained a beautiful idea: after quantization, point-tokens drift. The drift d(x) = (W_comp − W)x is a linear field.

Inputs lie in a cone, so the drift should be low-dimensional and can be corrected with a cheap low-rank delta.

I even formulated the stopping criterion in advance: if the dimension of the drift exceeds 50 — the hypothesis is wrong, close the direction.

d(x) = (W_comp − W) · x, D = [d(x₁); …; d(xₙ)] — drift field on the calibration tokensΔW_{ideal} = E · V_c·V_cᵀ (rank ≤ 30, along the cone axes), cap30 = ‖Dc · V_c[:, :30]‖² / ‖Dc‖²

Drift diagnostics (drift_diagnostics.py): D — matrix of drifts at decoder-block outputs (the same 2048 tokens, pre/post CHMC v6); V_c — top-10 right singular vectors of the centered input cloud;

cap30 — upper bound on the share of drift energy that a rank-30 correction along the cone axes could capture.

The result on all three models and all blocks: drift d90 = 227–792. The scatter energy is smeared over hundreds of directions.

An idealized rank-30 correction would have captured from ~3 to ~39% of the drift energy (Table 6). The hypothesis was not confirmed — cleanly, by a pre-declared rule, without the temptation "tweak and try again".

Figure 5.Drift spectra after quantization: rows — models, columns — blocks; principal-component energy of the drift with a cumulative curve and an d90 label. The energy is smeared over hundreds of directions rather than concentrated in the first dozens. A low-rank correction is powerless here by construction.
Model (d) Block Drift d90 Effective drift rank Top-10 overlap with the cone cap30, %
SmolLM-135M (576) 0 227 187 0.160 10.2
7 301 247 0.213 10.0
15 304 168 0.680 27.2
29 237 141 0.526 38.8
Qwen2.5-0.5B (896) 0 353 318 0.114 5.9
6 432 344 0.508 15.2
12 442 401 0.390 12.4
23 391 289 0.426 22.3
TinyLlama-1.1B (2048) 0 459 358 0.115 2.9
5 716 601 0.327 6.5
11 792 695 0.423 8.3
21 743 607 0.364 11.4
Table 6. Drift diagnostics after CHMC v6 quantization (the same 2048 tokens, decoder-block outputs). d90 — the number of principal components needed to explain 90% of the drift energy;

top-10 overlap — cosine agreement of the leading directions of the drift with the cone axes (top-10 right singular vectors of the centered inputs); cap30 — the share of drift energy captured by an idealized rank-30 correction along the cone axes.

All blocks violate the stopping criterion d90 ≤ 50. In the middle blocks, where the cone is concentrated (SmolLM blk 15,

Qwen blk 6, TinyLlama blk 11), the drift aligns with the principal axes of the cone (overlap ≈ 0.4–0.7), but most of its energy lies in other directions — cap30 there ≤ 27%.

The last chance remained: the total drift is high-dimensional, but the contribution of each individual layer — low-rank.

Then one could have compressed only "safe" layers and kept the rest in FP16. I checked on all 210 compressible layers of SmolLM:

local layer drift D_l = X·(W − W_comp)ᵀ — the pure contribution of a layer on a clean FP input, without additional forward passes.

The criterion was declared in advance: ≥50% of layers with local d90 ≤ 30 → proceed, otherwise close it.

Result: median local d90 = 230 (range 9–339), below the threshold — only 1 out of 210 layers (0.5%), the sole one that passed — L0.o_proj (rank 2, d90 = 9).

The magnitude of per-layer drift is small (~2–10% of the layer output in block 0), but it is high-dimensional already in each individual layer: local d90 does not grow with depth, but lies flat at ~220–247 across all blocks.

Same mechanism: GPTQ compensation distributes rounding error over all active input directions — per-layer drift is small in norm, but nearly isotropic in every layer. Selective compression by safety dimension — closed.

And then the puzzle came together.

There is a structural argument: my delta correction ΔW = E·V·Vᵀ — strictly weaker than what GPTQ already does inside itself:

its per-column compensation minimizes error over all directions of calibration inputs, while the conical correction — only a subset.

After GPTQ the residual error is nearly orthogonal to the data distribution. Projecting it onto the cone — there is nothing.

Separately worth clarifying is the connection with the "analogy theorem" in Hoare logic (published article, Nature Scientific Reports),

which we discussed as a formal framework for transferring quantization guarantees.

Its metric lives not in weight space but in signal space: this is the per-layer drift of activations D_l = ‖post_l − pre_l‖ — how much the signal changes when passing through a specific layer;

if D_l stays within safe bounds (≤ ε−γ), quality is preserved. Meanwhile, the weight matrices W and quantization error E_q = W − W_q in it appear only as objects of other,

existing methods (AWQ, GPTQ, LQER) that repair or compress the weights themselves; the theorem looks at the result — how weight compression affected signals inside the model — and makes decisions precisely in signal space.

We measured precisely this quantity per layer — and got the same as in the total drift: small magnitude, high dimensionality (median local d90 ≈ 230; no layers safe in the sense of "D_l ≤ ε−γ in a low-rank sense").

If the drift had turned out to be low-dimensional, D_l would have become a natural accuracy metric — a gate for selective compression: compress layers where D_l ≤ ε−γ, keep the rest in FP16.

Instead this path turned out to be a dead end: there are no safe layers, and this is consistent with everything else in this part.

Part 5. The Unified Picture: Weights — the Law, Activations — the Trajectory

All the negative results of the project were suddenly explained by a single theorem-observation:

The cone is a signal, and it is exhausted in one pass (weighted SVD along the cone, first order). The residual after data-adaptive quantization is a structurally structureless high-dimensional field.

There is no noise to exploit — that is why any geometric add-on (rotation, cone, QJL, delta correction) yields zero or negative.

But this is only half of the picture. The other half is an understanding that came later, in one conversation.

Activations are not "intermediate embeddings". This is the internal state of the model

A transformer — a discrete dynamical system:

h_{l+1} = F_l(h_l)

where $F_l$ is set by the weights of layer l. Then everything falls into place:

  • Weights = law of dynamics (the program, the "Hamiltonian" of the system). Dense, full-rank, cleaned up by training.
  • Internal state $h_l$= trajectory of the system. And the trajectory lies on a low-dimensional attractor — that is precisely our cone (d90=9, anisotropy 0.75).

This is a classic picture from dynamical systems theory: equations of motion can be as complicated as you like, but trajectories converge to a low-dimensional attractor.

We literally measured this — and the attractor contracts toward the middle of the network (in the middle blocks 98.9% of variance in 3 PCs versus 13–43% at the edges): the deepest point of processing — the most compressed state.

Now the asymmetry becomes trivial:

  • The trajectory is redundant: it lies on the attractor, outside the cone there simply is nothing. That is why the state is compressible — TurboQuant squeezes the KV cache (memory of states) by 6–8×.
  • The law is dense: it is this that keeps the trajectory on the attractor. There is no redundancy in it — training removed it. That is why geometry does not help there, and GPTQ wins.

The model has been trained to create redundancy in its computations but not in its parameters. It is a generator of conic geometry, but itself is not geometric.

An honest caveat: the attractor is an interpretation, not a theorem. But it is a productive interpretation: it is not fitted to the answer, but predicts three independent observations that we obtained earlier.

Our Three Results in the Light of Dynamical Systems

  • Linear accumulation of error (1.02 → 1.33 across layers). Perturbed dynamics (compressed weights) deflects the trajectory, but the attractor pulls it back — the divergence is linear, not exponential.

The model is robust to quantization precisely because of its conic structure.

  • TDA "macro-homeomorphism with micro-noise": compression preserves the macrostructure of the attractor (pre/post cosines ≥ 0.98, effective rank of weights shifts by ~1%), but saturates it with fine noise — spurious cycles (b1-gain).
  • High-dimensional drift: perturbation of the trajectory is not a shift along the cone but scattering around it: the state returns to the attractor, and the noise remains in transverse directions. Hence d90 of drift 227–792.

Epilogue. What This Yields

No breakthrough occurred: no 100× compression, no stable win over GPTQ. But something else did happen.

First, understanding. I obtained a coherent picture with a mechanism: why geometry works for activations (TurboQuant) and does not work for weights; why data-adaptive compensation beats distributional optimality;

where the cone lives and what happens to topology under compression.

Second, honest negative results with explanations — a rare and valuable genre.

"QJL cannot store weights, here is the error limit", "drift is not low-rank, here is the dimension", "rotation destroys anisotropy, here is the invariant" — these are ready-made answers for those who will follow this path after us.

Third, practice. The method beats scalar quantization by 6–11× on small models — precisely where it breaks down catastrophically — and provides an interpretable analysis of the model (ranks, energy, layer sensitivity) that GPTQ's black box does not have.

And above all — I stopped thinking about "beating GPTQ". I realized that my original question was not "how to compress weights geometrically" but "where does the boundary of geometric compression lie". And the answer turned out to be beautiful:

Geometry describes the model, but it does not compress its weights better than adaptivity to data. The cone is what the model creates; and what it has already created and packed into the weights can only be compressed down to noise without a remainder.

The ideal edge stack — GPTQ for weights and TurboQuant for the KV cache. Geometry wins, just not where we were first looking.

Afterword: Why This Matters for the Whole Field

Our project now delivers not "we tried to beat GPTQ and failed" but a complete theory:

"We showed that the geometry of LLM activations (the cone hypothesis) yields significant gains when compressing

model data (KV cache, embeddings), but not when compressing

model parameters (weights). This explains TurboQuant's success on activations and the failure of all geometric techniques on weights. LLMs create redundancy in their computations but not in their parameters."

This theory unifies:

  • TurboQuant's success on activations
  • The failure of geometric methods on weights
  • GPTQ/AWQ's success on weights
  • The cone hypothesis (as a property of the function, not of the parameters)

And it likewise yields a practical conclusion: to compress LLMs on edge you need two different tools, because we are compressing two different entities — law and trajectory.

Appendix A. Reproducibility

CHMC v6 (quantization). strict_sequential=true; dampening λ = 0.05; residual_bits = 4; group_size = 128 → total BPW = 4.2875 (per-layer ranks from v6/bpw.py, Table 1); seed = 42.

Key configurations were measured with ≥3 seeds (SmolLM and TinyLlama), Qwen — single run (marked in Table 4). GPTQ reference: GPTQModel, 4-bit / group-128.

TDA analysis (topology). kNN graph with k = 32, sample_points = 512 per block, seed = 42; H₀ — union-find over connected components + cross-check with GUDHI (0 mismatches); W₁ distance — truncated top-64 L∞. Artifacts: results_v6/tda_analysis/.

Drift diagnostics. D = post − pre at the outputs of decoder blocks (the same 2048 tokens);

PCA — SVD of centered matrices (numpy); cone basis V_c — top-10 right singular vectors of cent(pre); cap30 = ‖Dc·V_c[:, :30]‖²/‖Dc‖².

Stopping criteria (d90 > 50 or overlap < 0.3) were declared before the experiments. Artifacts: results_v6/drift_correction/step1_diagnostics/.

Environment. All analyses in the article — CPU-only (torch 2.13.0+cpu); GPU quantization runs were performed in a separate environment. Figures: results_v6/article/figures/.

Appendix B. TDA Comparison Across Models (pre vs post CHMC v6)

Model Mean pre/post cosine (across blocks) b1-gain / b1-drop (blocks) Effective weight rank, median pre → post Min. effective rank (pre) H₀ cross-check
SmolLM-135M 0.9938 14 / 1 274 → 276 (+0.9%) 18.8 0 mismatches
Qwen2.5-0.5B 0.9864 8 / 9 358 → 366 (+2.3%) 18.7 0 mismatches
TinyLlama-1.1B 0.9907 8 / 4 930 → 941 (+1.2%) 22.1 0 mismatches
Table 7. Persistent homology of activation clouds (kNN k=32, sample_points=512):

compression preserves the macrostructure — mean pre/post cosines across blocks ≥ 0.986, effective weight rank shifts by ≤ 2.3% — and adds micro-noise in the form of false cycles (b1-gain). b1-drop — blocks where cycles disappeared.

Appendix C. Sources and Links

Project repository (code + reproducible results): github.com/xmillogx-cmd/chmc-llm-quant

A. Mentioned directly in the article

  • TurboQuant (Google Research, ICLR 2026) — geometric compression of KV cache/embeddings; the reference point for the whole project: arXiv: 2504.19874 (Zandieh et al., "Online Vector Quantization with Near-optimal Distortion Rate") · Google Research blog — research.google
  • GPTQ — column-wise quantization error compensation (Frantar et al., ICLR 2023): arXiv: 2210.17323
  • QuIP# — quantization with integer codewords (Tseng et al., ICML 2024): arXiv: 2402.04396
  • Lloyd–Max quantizer: S. Lloyd, "Least squares quantization in PCM", IEEE Trans. Information Theory, 28(2), 1982 (original — 1957) — IEEE Xplore; J. Max, "Quantizing for minimum distortion", IRE Trans. Information Theory, 6(1), 1960 — Semantic Scholar
  • QJL and the Johnson–Lindenstrauss lemma: QJL — as part of TurboQuant (arXiv: 2504.19874); W. Johnson, J. Lindenstrauss, "Extensions of Lipschitz mappings into a Hilbert space", Contemp. Math., 26, 1984 — Semantic Scholar
  • Randomized SVD: N. Halko, P.-G. Martinsson, J. A. Tropp, "Finding structure with randomness: stochastic algorithms for decomposing matrices", SIAM Review, 2011 — arXiv: 0909.4061
  • TDA (topological data analysis): giotto-tda — arXiv: 2004.02551 (JMLR 2021); GUDHI — gudhi.inria.fr; classics (no links): Edelsbrunner, Letscher, Zomorodian, "A persistent topological approach to topology-based data analysis", 2002; G. Carlsson, "Topology and data", Bull. AMS, 2009.

B. Models and Data (used in the experiments)

C. Additional Literature (related methods)

P.S. All runs are reproducible: equal bit budget 4.29, key configurations measured with at least three seeds (SmolLM and TinyLlama; Qwen — a preliminary single run, marked in the table), stopping criteria declared before the experiments. A negative result obtained honestly is also a result.