A/B test the two models and compare?
Would be interesting to see if these activations only show up on larger models, or they're some relation to model size.
Hah. Yes. It looks like they only show up in models with 6.7B parameters or more.
The problem can start at 125M. Small enough to test on a whim.
So train a model that exhibits these behaviours, then try it out.
A/B test the two models and compare?
Would be interesting to see if these activations only show up on larger models, or they're some relation to model size.