I created "apfel" https://github.com/Arthur-Ficial/apfel a CLI for the apple on-device local foundation model (Apple intelligence) yeah its super limited with its 4k context window and super common false positives guardrails (just ask it to describe a color) ... bit still ... using it in bash scripts that just work without calling home / out or incurring extra costs feels super powerful.
Neat! I’ve actually been building with AFM, including training some LoRA adapters to help steer the model. With the right feedback mechanisms and guardrails, you can even use it for code generation! Hopefully I’ll have a few apps and tools out soon using AFM. I think embedded AI is the future, and in the next few years more platforms will come around to AI as a local API call, not an authorized HTTP request. That said, AFM is still incredibly premature and I’m experimenting with newer models that perform much better.
this is great! Incredibly fast and is working pretty well running loads on my m4 max studio.
Weirdly though I'm getting things like this: Apple FM is fast and free but has a hard limitation — it can't process prompts with
Spanish/non-English words, which is a dealbreaker for California and Southwest real estate where half the street names are Spanish.
Honestly I can't believe Apple put that foundation model product out the door. I was so excited about it, but when I tried it, it was such a disappointment. Glad to hear you calling that out so I know it wasn't just me.
Looks like they have pivoted completely over to Gemini, thank god.
This really makes me think of A Deepness in the Sky by Vernor Vinge. A loose prequel to A Fire Upon The Deep, and IMO actually the superior story. It plays in the far future of humanity.
In part of it, one group tries to take control of a huge ship from another group. They in part do this by trying to bypass all the cybersecurity. But in those far future days, you don't interface with all the aeons of layers of command protocols anymore, you just query an AI who does it for you. So, this group has a few tech guys that try the bypass by using the old command protocols directly (in a way the same thing like the iOS exploit that used a vulnerability in a PostScript font library from 90s).
Imagine being used to LLM prompting + responses, and suddenly you have to deal with something like
sed '/^```/d;/^#/d;s/^[[:space:]]\*//;/^$/d' | head -1); [[ $r ]]
and generally obtuse terminal output and man pages.
:)
(offtopic: name your variables, don't do local x c r a;. Readability is king, and a few hundred thousand years from now some poor Qeng Ho fellow might thank his lucky stars you did).
That would be pretty cool if the model was a little more useful, but it isn't very good. And the guardrails are hilariously bad.
cmd print first and last line from stdin
$ echo -n | tail -n 2
cmd print first and last line from stdin using sed
error: [guardrail] The request was blocked by Apple's safety guardrails. Try rephrasing.
no command generated
