I don't know the correct terms here, but there is a path from your throat to your ears that is normally closed. When you swallow or yawn the paths usually open, but some people can voluntarily open them.

When you do it, you can hear a low humming, but also sounds through your mouth. Maybe this is what you mean?

It's clearer if you're wearing for example hearing protection, and open your mouth and the paths.

The path from your throat to your ears is called the Eustachian tube.

Try clenching your teeth. I can hear a sound when I first clench my teeth and then it dies down.

Pretty sure it's normal. Feels like I'm pushing from behind my eyes.