Update autointerp results (first 900 latents, tighter CIs)
Browse files
README.md
CHANGED
|
@@ -38,8 +38,8 @@ sae = Sae.load_from_hub("EleutherAI/sae-SmolLM2-1.7B-layer17-32x-embedskip")
|
|
| 38 |
|
| 39 |
Explanations were generated and scored with
|
| 40 |
[delphi](https://github.com/EleutherAI/delphi) automated interpretability over a
|
| 41 |
-
growing sample of latents (currently the first
|
| 42 |
-
**
|
| 43 |
|
| 44 |
**Protocol**
|
| 45 |
|
|
@@ -55,11 +55,11 @@ growing sample of latents (currently the first 800 latent indices, of which
|
|
| 55 |
|
| 56 |
| Metric | Detection | Fuzzing |
|
| 57 |
|---|---|---|
|
| 58 |
-
| Balanced accuracy | 0.
|
| 59 |
-
| F1 | 0.
|
| 60 |
-
| Frequency-weighted F1 | 0.
|
| 61 |
-
| Precision | 0.
|
| 62 |
-
| Recall | 0.
|
| 63 |
|
| 64 |
Scores are computed over a representative latent sample that includes rare
|
| 65 |
features. Balanced accuracy sits near chance for the rarer latents, which are
|
|
|
|
| 38 |
|
| 39 |
Explanations were generated and scored with
|
| 40 |
[delphi](https://github.com/EleutherAI/delphi) automated interpretability over a
|
| 41 |
+
growing sample of latents (currently the first 900 latent indices, of which
|
| 42 |
+
**899** cleared the ≥200 activating-example threshold and were scored).
|
| 43 |
|
| 44 |
**Protocol**
|
| 45 |
|
|
|
|
| 55 |
|
| 56 |
| Metric | Detection | Fuzzing |
|
| 57 |
|---|---|---|
|
| 58 |
+
| Balanced accuracy | 0.536 [0.530, 0.543] | 0.571 [0.564, 0.579] |
|
| 59 |
+
| F1 | 0.608 [0.603, 0.612] | 0.496 [0.485, 0.506] |
|
| 60 |
+
| Frequency-weighted F1 | 0.680 [0.650, 0.688] | 0.418 [0.357, 0.684] |
|
| 61 |
+
| Precision | 0.527 [0.522, 0.532] | 0.602 [0.592, 0.612] |
|
| 62 |
+
| Recall | 0.718 [0.709, 0.727] | 0.422 [0.411, 0.433] |
|
| 63 |
|
| 64 |
Scores are computed over a representative latent sample that includes rare
|
| 65 |
features. Balanced accuracy sits near chance for the rarer latents, which are
|