Card: correct parameter count, phase descriptions and neuron-cluster share (measured from grokking_demo.pth)
Browse files
README.md
CHANGED
|
@@ -43,9 +43,9 @@ The model learns a **Fourier-based algorithm** to perform modular addition:
|
|
| 43 |
| Vocabulary (input) | 114 (0-112 for numbers, 113 for `=`) |
|
| 44 |
| Vocabulary (output) | 113 |
|
| 45 |
| Context length | 3 tokens: `[a, b, =]` |
|
| 46 |
-
| Parameters |
|
| 47 |
|
| 48 |
-
Design choices (no LayerNorm, ReLU,
|
| 49 |
|
| 50 |
## Usage
|
| 51 |
|
|
@@ -105,7 +105,7 @@ pip install torch transformer-lens
|
|
| 105 |
| Batch size | Full batch |
|
| 106 |
| Checkpoints | Every 100 epochs (250 total) |
|
| 107 |
| Seed | 999 (model), 598 (data split) |
|
| 108 |
-
| Training time | ~2 minutes on GPU |
|
| 109 |
|
| 110 |
The high weight decay (1.0) is critical for grokking -- it gradually erodes memorization weights in favor of the compact generalizing Fourier circuit.
|
| 111 |
|
|
@@ -113,16 +113,18 @@ The high weight decay (1.0) is critical for grokking -- it gradually erodes memo
|
|
| 113 |
|
| 114 |
The training exhibits three distinct phases:
|
| 115 |
|
| 116 |
-
1. **Memorization** (
|
| 117 |
-
2. **Circuit Formation** (~
|
| 118 |
-
3. **Cleanup** (~
|
|
|
|
|
|
|
| 119 |
|
| 120 |
## Mechanistic Interpretability Findings
|
| 121 |
|
| 122 |
Analysis of the trained model reveals:
|
| 123 |
|
| 124 |
- **Fourier-sparse embeddings**: The model learns embeddings concentrated on key frequencies (k = 9, 33, 36, 38, 55)
|
| 125 |
-
- **Neuron clustering**:
|
| 126 |
- **Logit periodicity**: Output logits approximate `cos(freq * 2pi/p * (a + b - c))` for key frequencies
|
| 127 |
- **Progress measures**: Restricted loss and excluded loss track the formation and cleanup of circuits independently, revealing that grokking is not a sudden phase transition but the delayed visibility of a gradually forming algorithm
|
| 128 |
|
|
|
|
| 43 |
| Vocabulary (input) | 114 (0-112 for numbers, 113 for `=`) |
|
| 44 |
| Vocabulary (output) | 113 |
|
| 45 |
| Context length | 3 tokens: `[a, b, =]` |
|
| 46 |
+
| Parameters | 227,313 (226,048 weights + 1,265 bias entries, which stay at 0) |
|
| 47 |
|
| 48 |
+
Design choices (no LayerNorm, ReLU, biases frozen at zero) were made to simplify mechanistic interpretability analysis.
|
| 49 |
|
| 50 |
## Usage
|
| 51 |
|
|
|
|
| 105 |
| Batch size | Full batch |
|
| 106 |
| Checkpoints | Every 100 epochs (250 total) |
|
| 107 |
| Seed | 999 (model), 598 (data split) |
|
| 108 |
+
| Training time | ~2 minutes on GPU (as reported when trained; not in the file) |
|
| 109 |
|
| 110 |
The high weight decay (1.0) is critical for grokking -- it gradually erodes memorization weights in favor of the compact generalizing Fourier circuit.
|
| 111 |
|
|
|
|
| 113 |
|
| 114 |
The training exhibits three distinct phases:
|
| 115 |
|
| 116 |
+
1. **Memorization** (epochs 0 to ~1,600): Train loss falls to ~10⁻⁷ while test loss rises far above chance, to a peak of 31.7 at epoch 1,381 (uniform guessing would give ln 113 ≈ 4.73). The model memorizes all training examples.
|
| 117 |
+
2. **Circuit Formation** (~1,600 to ~9,000): The Fourier-based generalizing circuit gradually forms in the weights while memorization still dominates; test loss falls only slowly (≈ 20 at epoch 6,000).
|
| 118 |
+
3. **Cleanup** (~9,000 to ~12,800): Weight decay erodes memorization faster than the compact Fourier circuit. Test loss collapses (below 1 at epoch 9,546, below 0.01 at epoch 10,160) -- this is the grokking moment.
|
| 119 |
+
|
| 120 |
+
Phase boundaries come from the objective rules of [visualizing-grokking-modular-addition](https://github.com/BurnyCoder/visualizing-grokking-modular-addition/blob/main/docs/operationalizations.md) applied to this file's losses and weights: memorization ends when the train loss stops falling by 1 % per 100 epochs, circuit formation when the test loss reaches 20 % of its early plateau, and cleanup when the sum of squared weights has nearly settled.
|
| 121 |
|
| 122 |
## Mechanistic Interpretability Findings
|
| 123 |
|
| 124 |
Analysis of the trained model reveals:
|
| 125 |
|
| 126 |
- **Fourier-sparse embeddings**: The model learns embeddings concentrated on key frequencies (k = 9, 33, 36, 38, 55)
|
| 127 |
+
- **Neuron clustering**: 298 of 512 MLP neurons (58 %) have more than 85 % of their activation variance explained by a single Fourier frequency (clusters k = 9: 42, k = 33: 56, k = 36: 107, k = 38: 45, k = 55: 48 neurons)
|
| 128 |
- **Logit periodicity**: Output logits approximate `cos(freq * 2pi/p * (a + b - c))` for key frequencies
|
| 129 |
- **Progress measures**: Restricted loss and excluded loss track the formation and cleanup of circuits independently, revealing that grokking is not a sudden phase transition but the delayed visibility of a gradually forming algorithm
|
| 130 |
|