I've been looking into buy a mac for llm experimentation - 64, 96 or 128gb of ram? I'm trying to decide if 64gb is enough, or should I go to 96gb or even 128gb. But it's really expensive - even for an overpaid software engineer. Then there's the 1 or 2 tb storage question. Apple list price is another $400 for that second tb of storage.
For 64gb of ram, you can get an m2 pro, or get 96gb which requires the upgraded cpu on the pro. The studio does 64gb or 128gb. But the 128 requires you to spend 5k.
I can't decide between 64 or 96 on m2 pro, and 128 on the studio. Probably go for 96gb. Also what's the impact of the extra gpu cores on the various options? And there are still some "m1" 64gb pros & studios out there. What's the perf difference for m1 vs m2? This area needs serious perf benchmarking. If anyone wants to work with me, maybe I would try my hand. But I'm not spending 15k just to get 3 pieces of hardware.
I’m pretty sure you can get a purpose-built pc tower in that range. Why would you favor a Mac over that? A lot of this stuff only has limited support for MacOS.
Is there a big difference in principle between that and the "shared video memory" that has long existed on cheap x86 machines?
…or is it just that the latter had a way too weak iGPU and not enough RAM for AI purposes, whereas the bigger ARM MACs have more GPU power and enough RAM (more than most affordable discrete graphic cards) so that they are usable for some AI models?
For 64gb of ram, you can get an m2 pro, or get 96gb which requires the upgraded cpu on the pro. The studio does 64gb or 128gb. But the 128 requires you to spend 5k.
I can't decide between 64 or 96 on m2 pro, and 128 on the studio. Probably go for 96gb. Also what's the impact of the extra gpu cores on the various options? And there are still some "m1" 64gb pros & studios out there. What's the perf difference for m1 vs m2? This area needs serious perf benchmarking. If anyone wants to work with me, maybe I would try my hand. But I'm not spending 15k just to get 3 pieces of hardware.
List prices:
64gb/2tb m2 12cpu/30gpu 14" pro $3900
96gb/2tb m2 max 12/38 14" pro $4500
128gb/2tb m2 max 28/48 studio $5200