Claude's API now supports CORS requests, enabling client-side applications • Hacker News Board

Re: Claude's API now supports CORS requests, enabling client-side applications

#31

Post by simonw » Fri, Aug 23, 2024, 7:33 AM UTC

axegon_ wrote: post #30

jillesvangurp wrote: post #21

Earlier quoted context omitted.

Depends how you do this. If you allow users to configure their own key, they basically end up using https to communicate the API key directly to the party that issued it. Not much of a risk of leakage there and very common with e.g. browser and editor extensions written in javascript. In a browser, you need the server to be setting CORS headers for this to work. Provisioning some key to your users so they can then pa…

I see three problems with this: 1. From a product perspective, this is like going to a restaurant to get dinner but having to bring your own kitchen utensils, food and cooking your dinner yourself. 2. Anything running in a browser is inherently insecure - what's the guarantee that the site where you're pasting your key doesn't have some incredibly stupid security flaw and your key gets leaked? 3. Even if there are no…

This is true: you do have to trust the site author that you are pasting your key into not to steal it.

For my https://tools.simonwillison.net/haiku thing I deliberately kept the code as simple as possible: if you know basic JavaScript you can view source and confirm that your key is not being stolen.

The code is also open source, so you can run a copy on your own hosting if you want to.

If you don’t trust that then I guess you don’t get to use my tool to write haikus about your dog!

As for usability: obviously if you want your thing to be used by people who don’t know how to pay for their own API key you should use a different solution.

I mainly want to ship cool demos that are trivial to host and that other people can try out without bankrupting me, so I’m really excited about this.

Re: Claude's API now supports CORS requests, enabling client-side applications

#32

Post by dvdkon » Fri, Aug 23, 2024, 7:40 AM UTC

bboygravity wrote: post #28

afiodorov wrote: post #10

I love making web apps where users bring their own keys. This approach combines the best of both worlds: the convenience of distributing executable files and the benefits of open source. So far, I have developed two web apps: 1. A live transcription and translation app that uses microphone input. This is useful for watching proprietary content and facilitating communication. 2. An app that translates SRT subtitles in…

I do the same now for a firefox extension I wrote (automatic form-filler that works way way better than anything else out there). So it's also "bring your own keys" but then how do you monetize at all? I personally don't like "bring your own keys" at all from a user-friendlyness perspective. It means that you exclude the vast majority of potential users, because they don't know what that even means. Even "create an a…

"Bring your own keys" can mean a "log in with OpenAI" button. Having users navigate through a third party's arcane dev portal isn't a good experience, but that third party can make it painless for users if they so choose.

Re: Claude's API now supports CORS requests, enabling client-side applications

#33

Post by TeMPOraL » Fri, Aug 23, 2024, 7:49 AM UTC

bboygravity wrote: post #28

afiodorov wrote: post #10

I love making web apps where users bring their own keys. This approach combines the best of both worlds: the convenience of distributing executable files and the benefits of open source. So far, I have developed two web apps: 1. A live transcription and translation app that uses microphone input. This is useful for watching proprietary content and facilitating communication. 2. An app that translates SRT subtitles in…

I do the same now for a firefox extension I wrote (automatic form-filler that works way way better than anything else out there). So it's also "bring your own keys" but then how do you monetize at all? I personally don't like "bring your own keys" at all from a user-friendlyness perspective. It means that you exclude the vast majority of potential users, because they don't know what that even means. Even "create an a…

> So it's also "bring your own keys" but then how do you monetize at all?

TypingMind.com is a "bring you own API key" (obviously, being a LLM frontend), that's also successfully monetizing users. The secret is that it's actually a very good product; until recently, it was far ahead of the official tools (I mean, they had plugins for like half a year before OpenAI started talking about "GPTs"), so paying for the license feels worth it (definitely was, when TypingMind was strictly better than ChatGPT Plus subscription).

It's also not a subscription - another reason "bring your own keys" apps are interesting, because you likely already have a paid subscription with the API vendor; adding another one on top of that needs some good justification.

Anyway, "bring your own key" users are a different market from general audience, and unlike the latter, it isn't already saturated with fly-by-night garbage and scam extensions, so you can both charge more for that feature, and have smaller costs marketing it.

Re: Claude's API now supports CORS requests, enabling client-side applications

#34

Post by TeMPOraL » Fri, Aug 23, 2024, 7:52 AM UTC

dvdkon wrote: post #32

bboygravity wrote: post #28

Earlier quoted context omitted.

I do the same now for a firefox extension I wrote (automatic form-filler that works way way better than anything else out there). So it's also "bring your own keys" but then how do you monetize at all? I personally don't like "bring your own keys" at all from a user-friendlyness perspective. It means that you exclude the vast majority of potential users, because they don't know what that even means. Even "create an a…

"Bring your own keys" can mean a "log in with OpenAI" button. Having users navigate through a third party's arcane dev portal isn't a good experience, but that third party can make it painless for users if they so choose.

They don't; OpenAI maintains a separation between "general population" ChatGPT Frontend and the LLM API on purpose. It's arguably a good purpose.

And the issue should really be inverted: it's not about excluding less technically savvy users - it's about recognizing a market niche of more sophisticated users, that really want that feature, can likely pay more for it, give you word-of-mouth marketing for free if you execute well. It's a niche so underserved that you don't even have to compete with scammers and shovelware all that much.

Re: Claude's API now supports CORS requests, enabling client-side applications

#35

Post by TheCapeGreek » Fri, Aug 23, 2024, 7:53 AM UTC

axegon_ wrote: post #30

jillesvangurp wrote: post #21

Earlier quoted context omitted.

Depends how you do this. If you allow users to configure their own key, they basically end up using https to communicate the API key directly to the party that issued it. Not much of a risk of leakage there and very common with e.g. browser and editor extensions written in javascript. In a browser, you need the server to be setting CORS headers for this to work. Provisioning some key to your users so they can then pa…

I see three problems with this: 1. From a product perspective, this is like going to a restaurant to get dinner but having to bring your own kitchen utensils, food and cooking your dinner yourself. 2. Anything running in a browser is inherently insecure - what's the guarantee that the site where you're pasting your key doesn't have some incredibly stupid security flaw and your key gets leaked? 3. Even if there are no…

Also from the product side, if you're making client side requests with users' own keys, doesn't that also mean that your LLM prompts are visible to the user if they just inspect their network requests?

If your app is largely just a wrapper around a neat prompt, it means I can just go and copy the prompt and use it myself and save the fees on your app.

Your app has to really be a valuable UX wrap over the calls in that case, or catering to a nontechnical audience.

Re: Claude's API now supports CORS requests, enabling client-side applications

#36

Post by TeMPOraL » Fri, Aug 23, 2024, 7:56 AM UTC

axegon_ wrote: post #30

jillesvangurp wrote: post #21

Earlier quoted context omitted.

Depends how you do this. If you allow users to configure their own key, they basically end up using https to communicate the API key directly to the party that issued it. Not much of a risk of leakage there and very common with e.g. browser and editor extensions written in javascript. In a browser, you need the server to be setting CORS headers for this to work. Provisioning some key to your users so they can then pa…

I see three problems with this: 1. From a product perspective, this is like going to a restaurant to get dinner but having to bring your own kitchen utensils, food and cooking your dinner yourself. 2. Anything running in a browser is inherently insecure - what's the guarantee that the site where you're pasting your key doesn't have some incredibly stupid security flaw and your key gets leaked? 3. Even if there are no…

Sure, but this approach also has a lot of benefits, and there's a market segment that really appreciates those features, one that you likely aren't even serving right now, and which you could capture nearly for free by just adding a form field that stores an API key client-side and forwards it to requests to the API vendor. No operations costs for you at all.

Re: Claude's API now supports CORS requests, enabling client-side applications

#37

Post by TeMPOraL » Fri, Aug 23, 2024, 8:00 AM UTC

TheCapeGreek wrote: post #35

axegon_ wrote: post #30

Earlier quoted context omitted.

I see three problems with this: 1. From a product perspective, this is like going to a restaurant to get dinner but having to bring your own kitchen utensils, food and cooking your dinner yourself. 2. Anything running in a browser is inherently insecure - what's the guarantee that the site where you're pasting your key doesn't have some incredibly stupid security flaw and your key gets leaked? 3. Even if there are no…

Also from the product side, if you're making client side requests with users' own keys, doesn't that also mean that your LLM prompts are visible to the user if they just inspect their network requests? If your app is largely just a wrapper around a neat prompt, it means I can just go and copy the prompt and use it myself and save the fees on your app. Your app has to really be a valuable UX wrap over the calls in tha…

> Your app has to really be a valuable UX wrap over the calls in that case, or catering to a nontechnical audience.

There is space on the market for such apps, too. Lots of space, in fact, as the idea of making software tools instead of toys seems to be forgotten. "Bicycle for the mind" got stolen some years ago, and it's time to get it back.

And frankly, an app that's "largely just a wrapper around a neat prompt", is something I consider to fall somewhere between Fischer-Price copycat toy and a direct scam. It's definitely not a tool empowering people, if it can be replaced with "paste this into ChatGPT config" (or "paste this into this more configurable bring-your-own-key frontend for ChatGPT").

Re: Claude's API now supports CORS requests, enabling client-side applications

#38

Post by TeMPOraL » Fri, Aug 23, 2024, 8:02 AM UTC

skybrian wrote: post #12

ozfive wrote: post #8

Earlier quoted context omitted.

Even using the client's keys it would be a good idea to give a disclaimer that their key may be stored (If that is the case) and can be accessible through nefarious means. People are very susceptible to phishing attempts etc. and this sort of business model (where you have the client supply the key and store it through the browser is a slippery slope.

Yes, it’s like storing passwords.

Yes. It's as bad as using password managers.

Wait.

Re: Claude's API now supports CORS requests, enabling client-side applications

#39

Post by TeMPOraL » Fri, Aug 23, 2024, 8:05 AM UTC

creesch wrote: post #19

unglaublich wrote: post #6

Earlier quoted context omitted.

Unless it's the client's keys. Or keys obtained on the client's behalf.

Generally speaking that is what OAuth should be used for with clearly defined scopes and insights in what apps are making use of the API through your account. Not an API key with full access and no limitations. With openAI and other providers, I know you can limit the budget for a key, but that is still a pretty broad scope you are left with.

OAuth is nice when you're making an interactive in-browser SaaS. Sucks for just about any other application, in particular anything that may run headless.

Thankfully, OpenAI and the like offer actual APIs I can use for software and automation I write. And it is my right, both as a user and a developer, to let someone else write the software I'll use with my keys. It's up to me to decide if I trust that software, and suffer the consequences of a mistake. It's like the most basic way of using software, and I appreciate when I can use it like that, without having anyone insert themselves in the middle to help me stay "more secure".

Re: Claude's API now supports CORS requests, enabling client-side applications

#40

Post by InsideOutSanta » Fri, Aug 23, 2024, 8:05 AM UTC

KeplerBoy wrote: post #17

afiodorov wrote: post #10

I love making web apps where users bring their own keys. This approach combines the best of both worlds: the convenience of distributing executable files and the benefits of open source. So far, I have developed two web apps: 1. A live transcription and translation app that uses microphone input. This is useful for watching proprietary content and facilitating communication. 2. An app that translates SRT subtitles in…

Could ads even support such AI heavy use cases? I'm kind of out of touch with current AI API pricing and ad revenues, so i'm curious how the economics work out.

Yes, we're now at an inflection point where this is starting to become possible. gpt-4o mini costs $0.15 per 1 million input tokens, and $0.60 per 1 million output. This is cheap enough that it can, at least in some cases, be funded by ad impressions.

Of course, the implications here are mixed. If you want to build an ad-supported tool that actually helps people, that's great. But it also means it now makes clear financial sense to fill the web with AI-generated garbage with the assumption that that ad impressions will pay for it.