SSH keys are a massive security liability. A private key, readable by any process you execute, without you noticing, that acts as a key to the kingdom? Maybe protected by a weak passphrase - since you probably have to enter that often - an attacker can brute force at their own discretion after copying it? A heap of public keys belonging to god-knows-which-machine, copied manually to every other server?
Designing a system that is secure, scalable, and works for billions of people worldwide in a highly adversarial environment is freaking hard.
> A private key, readable by any process you execute, without you noticing, that acts as a key to the kingdom?
You're lumping together the data and access that SSH keys protect (which might be actually nothing) with the key mechanism themselves. The private key itself can be armored or stored in a Yubikey itself, or you can even use more exotic ways of protecting it.
The public keys can be easily automated while the private keys stay safe somewhere. Systems like SSH Universal Key Manager or Userify are out there (both on-prem, and Userify also has saas) to make maintaining the public keys across huge swathes of servers relatively simple (or sometimes extremely simple).
And not just authentication, but authorization, too (usually through something like sudo or doas). Or you can just roll your own with Ansible or LDAP (not nearly as flexible when dealing with two axis of variations - users and servers, but still doable). SSH keys being easy to manage is extremely important, because when things are hard to manage, people open security holes, either through ignorance or to save time.
So, like all keys, yes, SSH keys can be a massive security liability if not properly secured, but they're not so (intrinsically), or even by default.
Designing a system that is secure, scalable, and works for billions of people worldwide in a highly adversarial environment is freaking hard.