Yes the size is much reduced, and you do have reduced quality as a result, but it isn't as bad as what you're implying. Just a few days ago Meta released q4 versions of their llama models. It's an active research topic.