My understanding (not an expert) is that there is a difference depending how much of an initial dose of the virus you get. If it is very small then your immune system has more time to respond before the virus replicates to dangerously large numbers, whereas if your initial dose is large then you are starting with a large number of virus particles that replicate too fast for your immune system to respond.
Yes, that is the discussion I have heard, that healthcare workers without proper PPE get larger doses than those with the right gear and are more likely to fall ill. Same effect applies to the average person in public when exposed to an infected person who may not be showing systems - even imperfect PPE reduces their initial dose and increases their odds of having a less severe infection.