I think you might be talking about GPTs with actions?
This implementation of function calling works differently from those.
OpenAI/Anthropic don't make any API calls themselves at all here. You call their chat APIs with a list of your own available functions, then they may reply to you saying "run function X yourself with these parameters, and once you've run that tell us what the result was".
> I also imagine many tools wouldn't run under the developers' control
How? There is no execution model. The LLM simply responds, in JSON format, with the name of a function and its corresponding arguments in alignment with the JSON Schema spec you provided beforehand. It is entirely on you to do something with that information.
At the end of the day it is really not all that different to asking an LLM to respond with JSON in the prompt, but offers greater stability in the response as it will strictly adhere to the spec you defined and not sometimes go completely off the rails with unparseable gobbledygook as LLMs are known to do.
Huh? You have to use your api key and pay for the service.
Requests you make to the service providers are made on your own buck, you are supposed to track user stuff on your end. It would make no difference chatgpt wise who the user is, that's not part of the abstraction provided.
Presumably oezi wants to do that complicated three-party OAuth stuff.
Like when you use an online PDF editor with Google Drive integration - paying for the storage etc is between Google and the user, the files belong to the user's Google Drive account, but the PDF editor gets read/write access to them.
I think the disconnect is that he's talking about building plugins/"gpts" inside of chat GPT while others are thinking about using the API to build something from scratch?
That's my read. And he's totally right! Plugins/GPTs aren't a good platform or product, partly for some of the technical reasons he mentioned, but really because they're basically a tech demo for the real product (the tool API).
Many interesting API usages must be bound to the user and must be payed based on usage so must be tied to the user. OpenAI doesn't provide ways to monetize GPTs so it is hard to justify spending on behalf of the user.
Bro you are given a state of the art multi million dollar compute for like a couple of cents and you complain about not having it spoonfed to you.
You have an http api, implement all of this yourself, the devs can't read your mind.
You should be able to issue a request and do stuff before reading the response, boom non-blocking. If you can't handle low level, just use threads plus your favourite abstraction?
User API auth. Never seen this by an api provider, you are in charge of user auth, what do you even expect here?
Do your job, openai isn't supposed to magically solve this for you, you are not a consumer of magical solutions, you are now a provider of them
OpenAI isn't offering a viable product as it currently stands. This is why we only saw toy usage with the Plugins API and now with tools as part of GPTs. Since OpenAI wants to own the front end of the GPTs there isn't any way to implement the parts which aren't there.
About non-blocking: I am asking for their tools API to not block the user from continuing the conversation while my tool works. You seem to be thinking about something else.
> About non-blocking: I am asking for their tools API to not block the user from continuing the conversation while my tool works. You seem to be thinking about something else.
To be fair, that was very ambiguous (talking about API’s and non-blocking IO) and their initial assumption was the same as mine (and quite reasonable).
I agree so much but the last line struck me as hilarious given that 90% of the hype around LLM-based AI is explicitly that people do believe it’s magical. People already believe this tech is on the verge of replacing doctors, programmers, writers, actors, accountants, and lawyers. Why shouldn’t they expect the boring stuff like auth pass-thru to be pre-solved? Surely the AI companies can just have their LLM generate the required code, right?
Auth-pass thru is impossible/impractical with OpenAI tool API, because there is no way to identify users. Thus even if users log into my website first and I get their OAuth there, I can't associate to their OpenAI session.
The most crucial things missing in OpenAI's implementation for me were:
- Authentication for the API by the user rather than the developer.
- Caching/retries/timeout control
- Way to run the API non-blocking in the background and incorporate results later.
- Dynamic API tools (use an API to provide the tools for a conversation) and API revisions (for instance by hosting the API spec under a URL/git).