I had an old iPod that was unlistenable while it was charging, because noise from the charger leaked into the audio. I tried again recently with an Android phone and found the same problem to a lesser degree. Nice to hear that people are able to do this without trouble.

Sounds like you may need a ground loop isolator.

I've got portable speakers that sound absolutely terrible when actively charging. Do you reckon this would help?

Ground loops are very odd phenomena that can be hard to debug, but it will probably work.

(The stupid option is to lift the ground on your mains adapter, but you really shouldn't do that, somebody might get killed.)