BurnyCoder commited on
Commit
33ead8d
·
verified ·
1 Parent(s): 59d8dd0

Card: correct parameter count, phase descriptions and neuron-cluster share (measured from grokking_demo.pth)

Browse files
Files changed (1) hide show
  1. README.md +9 -7
README.md CHANGED
@@ -43,9 +43,9 @@ The model learns a **Fourier-based algorithm** to perform modular addition:
43
  | Vocabulary (input) | 114 (0-112 for numbers, 113 for `=`) |
44
  | Vocabulary (output) | 113 |
45
  | Context length | 3 tokens: `[a, b, =]` |
46
- | Parameters | ~2.5M |
47
 
48
- Design choices (no LayerNorm, ReLU, no biases) were made to simplify mechanistic interpretability analysis.
49
 
50
  ## Usage
51
 
@@ -105,7 +105,7 @@ pip install torch transformer-lens
105
  | Batch size | Full batch |
106
  | Checkpoints | Every 100 epochs (250 total) |
107
  | Seed | 999 (model), 598 (data split) |
108
- | Training time | ~2 minutes on GPU |
109
 
110
  The high weight decay (1.0) is critical for grokking -- it gradually erodes memorization weights in favor of the compact generalizing Fourier circuit.
111
 
@@ -113,16 +113,18 @@ The high weight decay (1.0) is critical for grokking -- it gradually erodes memo
113
 
114
  The training exhibits three distinct phases:
115
 
116
- 1. **Memorization** (~epoch 0-1,500): Train loss drops to ~0, test loss stays at ~4.73 (random guessing over 113 classes). The model memorizes all training examples.
117
- 2. **Circuit Formation** (~epoch 1,500-13,300): The Fourier-based generalizing circuit gradually forms in the weights, but memorization still dominates. Test loss appears unchanged.
118
- 3. **Cleanup** (~epoch 13,300-16,600): Weight decay erodes memorization faster than the compact Fourier circuit. Test loss suddenly drops -- this is the grokking moment.
 
 
119
 
120
  ## Mechanistic Interpretability Findings
121
 
122
  Analysis of the trained model reveals:
123
 
124
  - **Fourier-sparse embeddings**: The model learns embeddings concentrated on key frequencies (k = 9, 33, 36, 38, 55)
125
- - **Neuron clustering**: ~85% of MLP neurons are well-explained by a single Fourier frequency
126
  - **Logit periodicity**: Output logits approximate `cos(freq * 2pi/p * (a + b - c))` for key frequencies
127
  - **Progress measures**: Restricted loss and excluded loss track the formation and cleanup of circuits independently, revealing that grokking is not a sudden phase transition but the delayed visibility of a gradually forming algorithm
128
 
 
43
  | Vocabulary (input) | 114 (0-112 for numbers, 113 for `=`) |
44
  | Vocabulary (output) | 113 |
45
  | Context length | 3 tokens: `[a, b, =]` |
46
+ | Parameters | 227,313 (226,048 weights + 1,265 bias entries, which stay at 0) |
47
 
48
+ Design choices (no LayerNorm, ReLU, biases frozen at zero) were made to simplify mechanistic interpretability analysis.
49
 
50
  ## Usage
51
 
 
105
  | Batch size | Full batch |
106
  | Checkpoints | Every 100 epochs (250 total) |
107
  | Seed | 999 (model), 598 (data split) |
108
+ | Training time | ~2 minutes on GPU (as reported when trained; not in the file) |
109
 
110
  The high weight decay (1.0) is critical for grokking -- it gradually erodes memorization weights in favor of the compact generalizing Fourier circuit.
111
 
 
113
 
114
  The training exhibits three distinct phases:
115
 
116
+ 1. **Memorization** (epochs 0 to ~1,600): Train loss falls to ~10⁻⁷ while test loss rises far above chance, to a peak of 31.7 at epoch 1,381 (uniform guessing would give ln 113 ≈ 4.73). The model memorizes all training examples.
117
+ 2. **Circuit Formation** (~1,600 to ~9,000): The Fourier-based generalizing circuit gradually forms in the weights while memorization still dominates; test loss falls only slowly (≈ 20 at epoch 6,000).
118
+ 3. **Cleanup** (~9,000 to ~12,800): Weight decay erodes memorization faster than the compact Fourier circuit. Test loss collapses (below 1 at epoch 9,546, below 0.01 at epoch 10,160) -- this is the grokking moment.
119
+
120
+ Phase boundaries come from the objective rules of [visualizing-grokking-modular-addition](https://github.com/BurnyCoder/visualizing-grokking-modular-addition/blob/main/docs/operationalizations.md) applied to this file's losses and weights: memorization ends when the train loss stops falling by 1 % per 100 epochs, circuit formation when the test loss reaches 20 % of its early plateau, and cleanup when the sum of squared weights has nearly settled.
121
 
122
  ## Mechanistic Interpretability Findings
123
 
124
  Analysis of the trained model reveals:
125
 
126
  - **Fourier-sparse embeddings**: The model learns embeddings concentrated on key frequencies (k = 9, 33, 36, 38, 55)
127
+ - **Neuron clustering**: 298 of 512 MLP neurons (58 %) have more than 85 % of their activation variance explained by a single Fourier frequency (clusters k = 9: 42, k = 33: 56, k = 36: 107, k = 38: 45, k = 55: 48 neurons)
128
  - **Logit periodicity**: Output logits approximate `cos(freq * 2pi/p * (a + b - c))` for key frequencies
129
  - **Progress measures**: Restricted loss and excluded loss track the formation and cleanup of circuits independently, revealing that grokking is not a sudden phase transition but the delayed visibility of a gradually forming algorithm
130