The book that "clicked" for me is: Cryptography Engineering: Design Principles and Practical Applications, by Ferguson/Schneier/Kohno (2015) [1]
In plain language it walks through what I wanted to know, in a modern and paranoid perspective, as a readable narrative, from the point of view that we want to design each of the basic crypto primitives ourselves.
Unfortunately, Cryptography Engineering (nee Practical Cryptography) is very much showing its age. It's a much better book than Applied Cryptography, but it's still pre-modern --- it spends a lot of time on outmoded multiplicative group asymmetric encryption, essentially excludes AEAD cryptography (which are the most important constructions in modern cryptography), and has weird advice on random number generation (that probably made sense before the world standardized on OS-level CSPRNGs).
It's easy to forget how old Practical Cryptography is, but: it predates Vaudenay's padding oracle attack.
In plain language it walks through what I wanted to know, in a modern and paranoid perspective, as a readable narrative, from the point of view that we want to design each of the basic crypto primitives ourselves.
[1] https://onlinelibrary.wiley.com/doi/book/10.1002/97811187223...