I've been building Livnium, an NLI classifier with no…
0 reactions
I've been building Livnium, an NLI classifier with no transformers, no attention, just iterative geometry-aware state updates converging to a label basin before the final readout.
Discrete-time pseudo-gradient flow with anchor-directed forces. Here's the exact math, the geometric inconsistency I found, and what the Lyapunov analysis shows.
I've been building **Livnium**, an NLI classifier where inference isn't a single forward pass — it's a sequence of geometry-aware state updates converging to a label basin before the final readout. I initially used quantum-inspired language to describe it. That was a mistake. Here's the actual math.
**The update rule**
At each collapse step `t = 0…L−1`, the hidden state evolves as:
h_{t+1} = h_t
+ δ_θ(h_t) ← learned residual (MLP)
- s_y · D(h_t, A_y) · n̂(h_t, A_y) ← anchor force toward correct basin
- β · B(h_t) · n̂(h_t, A_N) ← neutral boundary force
where:
D(h, A) = 0.38 − cos(h, A) ← divergence from equilibrium ring
n̂(h, A) = (h − A) / ‖h − A‖ ← Euclidean radial direction
B(h) = 1 − |cos(h,A_E) − cos(h,A_C)| ← proximity to E–C boundary
Three learned anchors A\_E, A\_C, A\_N define the label geometry. The attractor is a *ring* at cos(h, A\_y) = 0.38, not the anchor point itself. During training only the correct anchor pulls. At inference, all three compete — whichever basin has the strongest geometric pull wins.
**The geometric inconsistency I found**
Force magnitudes are cosine-based. Force directions are Euclidean radial. These are inconsistent — the true gradient of a cosine energy is tangential on the sphere, not radial. Measured directly (dim=256, n=1000):
mean angle between implemented force and true cosine gradient = 135.2° ± 2.5°
So this is not gradient descent on the written energy. Correct description: **discrete-time attractor dynamics with anchor-directed forces**. Energy-like, not exact gradient flow. The neutral boundary force is messier still — B(h) depends on h, so the full ∇E would include ∇B terms that aren't implemented.
**Lyapunov analysis**
Define V(h) = D(h, A\_y)² = (0.38 − cos(h, A\_y))². Empirical descent rates (n=5000):
|δ\_θ scale|V(h\_{t+1}) ≤ V(h\_t)|mean ΔV|
|:-|:-|:-|
|0.00|100.0%|−0.00131|
|0.01|99.3%|−0.00118|
|0.05|70.9%|−0.00047|
|0.10|61.3%|\+0.00009|
When δ\_θ = 0, V decreases at every step. The local descent is analytically provable:
∇_h cos · n̂ = −(β · sin²θ) / (α · ‖h − A‖) ← always ≤ 0
Livnium is a **provably locally-contracting pseudo-gradient flow**. Global convergence with finite step size + learned residual is still an open question.
**Results**
|Model|ms / batch (32)|Samples/sec|SNLI train time|
|:-|:-|:-|:-|
|Livnium|0.4|85,335|\~6 sec|
|BERT-base|171|187|\~49 min|
SNLI dev accuracy: **77.05%** (baseline 76.86%)
Per-class: E 87.5% / C 81.2% / N 62.8%. Neutral is the hard part — B(h) is doing most of the heavy lifting there.
**What's novel (maybe)**
Most classifiers: `h → linear layer → logits`
This: `h → L steps of geometry-aware state evolution → logits`
h\_L is dynamically shaped by iterative updates, not just a linear readout of h\_0. Whether that's worth the complexity over a standard residual block — I genuinely don't know yet. Closest prior work I'm aware of: attractor networks and energy-based models, neither of which uses this specific force geometry.
https://preview.redd.it/pfuoqyx86apg1.png?width=2326&format=png&auto=webp&s=87262251aedc3292b897de64e326f0d3ba18e478
GitHub: https://github.com/chetanxpatil/livnium
HuggingFace: https://huggingface.co/chetanxpatil/livnium-snli
#technology
Discrete-time pseudo-gradient flow with anchor-directed forces. Here's the exact math, the geometric inconsistency I found, and what the Lyapunov analysis shows.
I've been building **Livnium**, an NLI classifier where inference isn't a single forward pass — it's a sequence of geometry-aware state updates converging to a label basin before the final readout. I initially used quantum-inspired language to describe it. That was a mistake. Here's the actual math.
**The update rule**
At each collapse step `t = 0…L−1`, the hidden state evolves as:
h_{t+1} = h_t
+ δ_θ(h_t) ← learned residual (MLP)
- s_y · D(h_t, A_y) · n̂(h_t, A_y) ← anchor force toward correct basin
- β · B(h_t) · n̂(h_t, A_N) ← neutral boundary force
where:
D(h, A) = 0.38 − cos(h, A) ← divergence from equilibrium ring
n̂(h, A) = (h − A) / ‖h − A‖ ← Euclidean radial direction
B(h) = 1 − |cos(h,A_E) − cos(h,A_C)| ← proximity to E–C boundary
Three learned anchors A\_E, A\_C, A\_N define the label geometry. The attractor is a *ring* at cos(h, A\_y) = 0.38, not the anchor point itself. During training only the correct anchor pulls. At inference, all three compete — whichever basin has the strongest geometric pull wins.
**The geometric inconsistency I found**
Force magnitudes are cosine-based. Force directions are Euclidean radial. These are inconsistent — the true gradient of a cosine energy is tangential on the sphere, not radial. Measured directly (dim=256, n=1000):
mean angle between implemented force and true cosine gradient = 135.2° ± 2.5°
So this is not gradient descent on the written energy. Correct description: **discrete-time attractor dynamics with anchor-directed forces**. Energy-like, not exact gradient flow. The neutral boundary force is messier still — B(h) depends on h, so the full ∇E would include ∇B terms that aren't implemented.
**Lyapunov analysis**
Define V(h) = D(h, A\_y)² = (0.38 − cos(h, A\_y))². Empirical descent rates (n=5000):
|δ\_θ scale|V(h\_{t+1}) ≤ V(h\_t)|mean ΔV|
|:-|:-|:-|
|0.00|100.0%|−0.00131|
|0.01|99.3%|−0.00118|
|0.05|70.9%|−0.00047|
|0.10|61.3%|\+0.00009|
When δ\_θ = 0, V decreases at every step. The local descent is analytically provable:
∇_h cos · n̂ = −(β · sin²θ) / (α · ‖h − A‖) ← always ≤ 0
Livnium is a **provably locally-contracting pseudo-gradient flow**. Global convergence with finite step size + learned residual is still an open question.
**Results**
|Model|ms / batch (32)|Samples/sec|SNLI train time|
|:-|:-|:-|:-|
|Livnium|0.4|85,335|\~6 sec|
|BERT-base|171|187|\~49 min|
SNLI dev accuracy: **77.05%** (baseline 76.86%)
Per-class: E 87.5% / C 81.2% / N 62.8%. Neutral is the hard part — B(h) is doing most of the heavy lifting there.
**What's novel (maybe)**
Most classifiers: `h → linear layer → logits`
This: `h → L steps of geometry-aware state evolution → logits`
h\_L is dynamically shaped by iterative updates, not just a linear readout of h\_0. Whether that's worth the complexity over a standard residual block — I genuinely don't know yet. Closest prior work I'm aware of: attractor networks and energy-based models, neither of which uses this specific force geometry.
https://preview.redd.it/pfuoqyx86apg1.png?width=2326&format=png&auto=webp&s=87262251aedc3292b897de64e326f0d3ba18e478
GitHub: https://github.com/chetanxpatil/livnium
HuggingFace: https://huggingface.co/chetanxpatil/livnium-snli
#technology
| 2,000 mlx | |
| 2 | |
| 0 |