As usual, YeGoblynQueene never has a good thing to say about deep learning, and isn't nearly as expert as they think they are. Unlike you, OP has actually been paying attention, and he knows that the GPT-3 paper seriously understates the arithmetic performance of GPT-3 because they failed to deal with the BPE issue. If you use that by adding commas, you drastically improve the arithmetic. Matt Brockman has done some more systematic evaluation: http://gptprompts.wikidot.com/logic:math