I've been fascinated by this question since I learned the sieve of eratosthenes as a kid. The meta logic of it is so simple:

Primes are specifically the numbers that are left over after the structured numbers (composite) ones are removed.

Everything - [structured numbers] = [ chaos? the abyss? some meta structure? ]