Is there a reason why a stack growing towards the beginning of memory is better than one growing towards the end of memory, or is that just an arbitrary choice the x86 designers made?
It's mostly because it allows you to put code at a fixed address at the beginning of the memory and the stack at the opposite end. That lets the OS place the stack at a constant address (for MS-DOS COM files the stack pointer starts at 0xFFFE, with a zero already pushed so that a RET instruction exits the program; a similar convention existed in CP/M).
It's got a lot of feng shui for simple computer architectures.
For one thing, you can have your code, static memory, and then heap at the beginning of the address space, and the stack at the end of the address space, and not have to worry about explicitly sizing the stack. On some architectures you could even just zero-initialize the stack pointer, and it will roll over to the end of memory on the first push.
You can also use the same machine code instructions that make array indexing efficient to access variables on the stack (load/store with literal offset,) without having to support signed offsets which would have effectively wasted a precious bit in a very common opcode.
When you overrun a buffer, it is more likely to cause your program to crash by corrupting the stack. This is bad in modern times for obvious security reasons, but pragmatically it would have been a useful "feature" in the early days of programming.
It pairs nicely with the semantics of a semaphore, where you decrement to acquire and increment to release.
The stack pointer conceptually points to valid memory, rather than pointing to invalid memory adjacent to valid memory. This likely has some favorable traits when it comes to how an architecture implements function calls. For example, RET could load the value at the stack pointer into the program counter, then increment the stack pointer. Loading from memory often takes multiple cycles compared to incrementing a register, and this starts the load a cycle earlier than otherwise for the same amount of combinatorial logic.
I don't know if this is the real reason, but it's more natural for stack-relative addressing to have the stack grow downwards, because you can write e.g. [rsp+10] instead of [rsp-10].
Apparently on PA-RISC (hppa) the stack grows the other way, so it is arbitrary.
