Well, after changing the ssh port to something really big, OOM and heavy CPU usage stopped, as I was still using that public IP, so concluded it was not an inside job .
There were like thousands of requests in an hour, and that went on continuously, before I changed the port.
Yeah that sounds quite annoying, but has nothing to do with ssh log noise. Maybe investigate what's causing the OOM. I have multiple 1GB vps with ssh open to the world and they never OOM, and they're obviously not just running ssh. It sounds like you've been compromised.
The number of attempts were staggering though, i think there were requests every seconds non-stop.
Once I changed the ssh port to a large number, the OOM and heavy CPU usage stopped, and never came back. So, I think I'm safe, though I keep an eye on the logs, and for any unknown processes, but never seen anything out of ordinary.
The 64gb machine is my dev machine, as my IDE(intellij) runs on high memory config and I run some heavy process, it could've been combined with the ssh spam it went OOM. I still run all the things, without any issues now.
And one simple mistake, and we're screwed