Depends how you do this. If you allow users to configure their own key, they basically end up using https to communicate the API key directly to the party that issued it. Not much of a risk of leakage there and very common with e.g. browser and editor extensions written in javascript. In a browser, you need the server to be setting CORS headers for this to work.
Provisioning some key to your users so they can then pass it on via a client side API call would indeed be more risky. Don't do that. But if it's their own key it's all fine.
1. From a product perspective, this is like going to a restaurant to get dinner but having to bring your own kitchen utensils, food and cooking your dinner yourself.
2. Anything running in a browser is inherently insecure - what's the guarantee that the site where you're pasting your key doesn't have some incredibly stupid security flaw and your key gets leaked?
3. Even if there are no vulnerabilities, you're still pasting your code in a random form somewhere on the web. All it takes is an ajax call or a websocket and someone, somewhere has your key.
Sure, but this approach also has a lot of benefits, and there's a market segment that really appreciates those features, one that you likely aren't even serving right now, and which you could capture nearly for free by just adding a form field that stores an API key client-side and forwards it to requests to the API vendor. No operations costs for you at all.
This is true: you do have to trust the site author that you are pasting your key into not to steal it.
For my https://tools.simonwillison.net/haiku thing I deliberately kept the code as simple as possible: if you know basic JavaScript you can view source and confirm that your key is not being stolen.
The code is also open source, so you can run a copy on your own hosting if you want to.
If you don’t trust that then I guess you don’t get to use my tool to write haikus about your dog!
As for usability: obviously if you want your thing to be used by people who don’t know how to pay for their own API key you should use a different solution.
I mainly want to ship cool demos that are trivial to host and that other people can try out without bankrupting me, so I’m really excited about this.
I might be biased, but I think there's room for a service that can make BYOK frictionless. Best of both worlds, unless you're a money sucking corporation trying to turn a purchase once product into a subscription service because late stage capitalism.
Also from the product side, if you're making client side requests with users' own keys, doesn't that also mean that your LLM prompts are visible to the user if they just inspect their network requests?
If your app is largely just a wrapper around a neat prompt, it means I can just go and copy the prompt and use it myself and save the fees on your app.
Your app has to really be a valuable UX wrap over the calls in that case, or catering to a nontechnical audience.
> Your app has to really be a valuable UX wrap over the calls in that case, or catering to a nontechnical audience.
There is space on the market for such apps, too. Lots of space, in fact, as the idea of making software tools instead of toys seems to be forgotten. "Bicycle for the mind" got stolen some years ago, and it's time to get it back.
And frankly, an app that's "largely just a wrapper around a neat prompt", is something I consider to fall somewhere between Fischer-Price copycat toy and a direct scam. It's definitely not a tool empowering people, if it can be replaced with "paste this into ChatGPT config" (or "paste this into this more configurable bring-your-own-key frontend for ChatGPT").
Your analogy in 1. sounds off to me. It’s definitely like bringing your own food, but the already well equipped kitchen and chefs will prepare it for you.
From a security standpoint it's no different than a password form except you're sharing one company credential with another. The issue is not html forms though, it's that you might trust Anthropic more than you do the hot new website built yesterday.
Bit of an odd warning though, considering how half the internet works. I suppose the warning is missing some context.
It is a bad idea to give your API key directly to web services you don't know or trust. For those situations, OAuth with fine-grained scopes would be more suitable.
If it is just your own web app, and you have an input for a key, I don't really see the issue.
Hilarious that even the LLM warned against this