The problem is that all these so-called "self-driving" systems expect you to be alert and ready because at any moment they might decide that the human needs to be in control again.
That's right now and that's why companies have to provide level 5 autonomy or there won't be any customers. Yet for days with good weather in urban environments level 5 might come within a decade; it's still better to be able to drive only half of the time instead of all the time. The long tail of problems that is supposedly plaguing self-driving systems exists with humans as well - how would you react if a bridge/building in front of you started collapsing or somebody started overtaking right in front of you?
But then again, I am actually able to discern what is a bridge and what isn't, and thus don't have the need to suddenly cover my eyes and scream, "I'm confused and I don't want to drive anymore!"
Does it matter though? If it drives like my grandma, I can still do other things while in a car and hate commuting/traffic jams a bit less.