In the USA, at least, this system is generally fair. I think most jurisdictions will allow you at least one switch of judge before the case begins if you don't like the one that got assigned - you can basically throw your lottery ticket back in the hat and get them to pull one more time.