-march tells the compiler which instruction set extensions it is allowed to use. For e.g. the difference between arch=haswell and arch=sandybridge is AVX2, MOVBE, FMA, BMI, and BMI2. If you build with haswell, or any given CPU, the compiler might emit instructions that's won't run on older hardware. That's why most Linux distributions for PC are compiled for k8-generic, an x86-64 platform with few extensions (MMX, SSE, and SSE2, if I recall correctly).
In the particular case as I recall it was AVX2 that was counterproductive on haswell. Disabling it made the program faster, even though AVX2 was supposed to be a headline feature of that generation.
Are you thinking of -mtune? I'd never heard of people using -march for performance, I thought it was just specifying the ISA.