As far as I have tried, at least chatgpt basically cannot do brainfuck. I think it might just be too far from natural language. It basically just gives you hello worlds with randomly injected characters.

I wonder why. GPT-3 could do brainfuck just fine:

https://papers.nips.cc/paper/2021/hash/0cd6a652ed1f7811192db...

(though worse, for complex cases, to one of the compared systems - see figure 2) (oops, fully disclosure: the system in question is my PhD work).

Doesn't ChatGPT excel at providing step-by-step instructions? That's kind of what Brainfuck is with only eight options for each step. I'd imagine that it could reach the end result using a combination of those eight options. Can it not?

Idk, I'd get nonfunctioning code. With explanations of what every piece of code does. Just that the code it is explaining, is actually gibbrish.

Has anyone tried it yet? Would be crazy if it worked.

I remember when Siri was first demoed, a dev I worked with was convinced it was just like having human personal assistant. I hadn't seen the demo, but I was confident he was wrong. Haven't tried this either, but I'm equally confident it wouldn't work .. or even kind of come close to working

Siri was objectively better in the past - I remember using it when I got my iPhone 4S and being impressed with how it returned very relevant data from WolframAlpha and other sources - then it all dried-up and all Siri is useful for now is voice-control for my bedroom lights and phone alarm...