If you have all your application code in headers you'll get the fastest build times (for full rebuilds at least) by including all headers into a single main.cpp file and build just that, since that way there is no redundant code for the compiler to build at all.
Of course the downside is that every tiny code change triggers a full rebuild then, but it's quite likely that the most time is spent in the linker anyway, so maybe worth a try.
I think I've heard this called a Unity build where there's a precompile step that just dumps everything into a single file and compiles that so it doesn't have to re-include everything at different compilation units (when I first heard the term I got confused because it was in a game dev context but had nothing to do with the Unity engine lol).
Except then there's no parallelism since you're only building one file. Ideally you'd split it into N files to take advantage of multiple cores, but then you have to decide how to split it...
Right. My rule of thumb would be "one implementation file per system", or what would be called a "module" in other languages. So that a moderately complex code base ends up with a about a couple dozen source files to build.
And each system should only have a single 'public interface header' to keep the number of cross-system include dependencies low.
Of course the downside is that every tiny code change triggers a full rebuild then, but it's quite likely that the most time is spent in the linker anyway, so maybe worth a try.