luciaquirke commited on
Commit
66f0b91
·
verified ·
1 Parent(s): 1f520ce

Update autointerp results (first 900 latents, tighter CIs)

Browse files
Files changed (1) hide show
  1. README.md +7 -7
README.md CHANGED
@@ -38,8 +38,8 @@ sae = Sae.load_from_hub("EleutherAI/sae-SmolLM2-1.7B-layer17-32x-embedskip")
38
 
39
  Explanations were generated and scored with
40
  [delphi](https://github.com/EleutherAI/delphi) automated interpretability over a
41
- growing sample of latents (currently the first 800 latent indices, of which
42
- **799** cleared the ≥200 activating-example threshold and were scored).
43
 
44
  **Protocol**
45
 
@@ -55,11 +55,11 @@ growing sample of latents (currently the first 800 latent indices, of which
55
 
56
  | Metric | Detection | Fuzzing |
57
  |---|---|---|
58
- | Balanced accuracy | 0.538 [0.531, 0.546] | 0.572 [0.564, 0.580] |
59
- | F1 | 0.609 [0.604, 0.615] | 0.497 [0.486, 0.508] |
60
- | Frequency-weighted F1 | 0.683 [0.660, 0.691] | 0.409 [0.349, 0.678] |
61
- | Precision | 0.528 [0.522, 0.534] | 0.603 [0.592, 0.614] |
62
- | Recall | 0.721 [0.710, 0.731] | 0.422 [0.411, 0.433] |
63
 
64
  Scores are computed over a representative latent sample that includes rare
65
  features. Balanced accuracy sits near chance for the rarer latents, which are
 
38
 
39
  Explanations were generated and scored with
40
  [delphi](https://github.com/EleutherAI/delphi) automated interpretability over a
41
+ growing sample of latents (currently the first 900 latent indices, of which
42
+ **899** cleared the ≥200 activating-example threshold and were scored).
43
 
44
  **Protocol**
45
 
 
55
 
56
  | Metric | Detection | Fuzzing |
57
  |---|---|---|
58
+ | Balanced accuracy | 0.536 [0.530, 0.543] | 0.571 [0.564, 0.579] |
59
+ | F1 | 0.608 [0.603, 0.612] | 0.496 [0.485, 0.506] |
60
+ | Frequency-weighted F1 | 0.680 [0.650, 0.688] | 0.418 [0.357, 0.684] |
61
+ | Precision | 0.527 [0.522, 0.532] | 0.602 [0.592, 0.612] |
62
+ | Recall | 0.718 [0.709, 0.727] | 0.422 [0.411, 0.433] |
63
 
64
  Scores are computed over a representative latent sample that includes rare
65
  features. Balanced accuracy sits near chance for the rarer latents, which are