Exp 6 — Listener-persona disagreement analysis
N panel runs analyzed: 17 Distinct personas seen: 4
Per-persona base rates (across all runs)
| persona | runs | mean notes/run | mean severity | top severity |
|---|---|---|---|---|
radio_4_devotee |
17 | 4.4 | 5.7 | 10 |
podcast_listener |
17 | 4.3 | 5.8 | 10 |
audio_engineer |
17 | 4.6 | 6.8 | 10 |
comedy_writer |
17 | 4.4 | 5.6 | 9 |
Pairwise anchor overlap (within-run, severity >= 5)
Jaccard of target_anchor sets per pair of personas. 1.0 = perfect agreement, 0.0 = disjoint complaints.
| persona A | persona B | mean Jaccard | n runs |
|---|---|---|---|
podcast_listener |
radio_4_devotee |
0.07 | 17 |
comedy_writer |
radio_4_devotee |
0.06 | 17 |
audio_engineer |
comedy_writer |
0.03 | 17 |
audio_engineer |
radio_4_devotee |
0.03 | 17 |
audio_engineer |
podcast_listener |
0.01 | 17 |
comedy_writer |
podcast_listener |
0.00 | 17 |
Overall mean pair-Jaccard: 0.03 (median 0.00, max 0.33)
Reading: if mean Jaccard < ~0.15, personas are largely independent — either complementary signal or noise. If mean Jaccard > ~0.40, personas converge on the same moments and the panel is acting as a 4-way confirmation.
Severity-score distribution per persona
| persona | sev≤3 | sev 4-6 | sev 7+ | total notes |
|---|---|---|---|---|
audio_engineer |
0 | 35 | 44 | 79 |
comedy_writer |
11 | 36 | 28 | 75 |
podcast_listener |
11 | 32 | 30 | 73 |
radio_4_devotee |
13 | 34 | 28 | 75 |