Where You Measure Decides What You Measure: Position Selection in Ablation-Based SAE Evaluation

Sparse autoencoders are meant to name the things a language model computes, and the usual way to check that a latent matters is to switch it off and see what changes. But a latent fires at many tokens, and the effect has to be measured at one of them. The convention is to measure where the latent fi...