We're talking about "the best predictor of IQ" which is not the same thing as having a large, negative effect in a small number of cases.