Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: My GPT project has become a local hit but I can't afford the bill

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 141 posts

Re: Ask HN: My GPT project has become a local hit but I can't afford the bill

#51

Post by ibizabear »

ugh123 wrote: post #13
f0e4c2f7 wrote: post #11

You could charge for heavy use. At that many users and only $50/day you might be able to cover costs with premium users. Alternatively you could try swapping out an open source model like llama on the back end (possible licensing issues here). Another option would be to seek out ads for local companies directly since it is an explicitly local product. Cold calling would probably be the best place to start. You could…

To add to that, OP should really be tracking usage a bit to see if there are abusers taking up a large amount of queries

Good shout, thank you

Re: Ask HN: My GPT project has become a local hit but I can't afford the bill

#52

Post by ibizabear »

stndef wrote: post #26

Is it GlasgowGPT by any chance? Either way, there's definitely an opportunity for funding. In the case of Glasgow, there's a huge populous out there that are descendants of folk from the city/country taking pride in it (arguably more than a bunch of us that live here!) and like to get in on the banter. That's certainly an avenue worth exploring if there's a similar relationship with folks or families of folks that em…

Haha yes it is! And appreciate the suggestion, that's a good idea that I'll look into. I've plastered a big banner up and will see if anyone goes for it, from locally or elsewhere. Cheers

Re: Ask HN: My GPT project has become a local hit but I can't afford the bill

#53

Post by Alexanfa »

Hey,

What you can do is to use caching and embeddings. Embeddings are like X,Y coordinates (except with OpenAI it's 1500 dimensions not just two). You use embeddings on any type of text. Here you would embed user prompts and store answers. And what this means is, you would be able to get X,Y for "best restaurants close to the museum", "best places to eat" etc.. If a new user comes along and mentions food and museum, it's likely to rank highest for "best restaurants close to the museum". I haven't done this and idk if possible, but you could try to then check if this is "close enough". If it is, then you can probably return the same generated answer for this user. You can also embed answers and see if that has better result.

Process: 1. When user asks a question, you embed it and search with it. 2. If similar prompts exist; return the answer from the most "relevant" prompt. 3. If no similar prompt exists; generate a new answer, and store it to be associated with the prompt.

This might decrease quality, but maybe in "enough" cases, it does just fine.

I'm developing tzap.io that uses some of these techniques, join our discord if you want to get in touch: https://discord.gg/88xDVYbPVB

Re: Ask HN: My GPT project has become a local hit but I can't afford the bill

#55

Post by rjtavares »

serial_dev wrote: post #32
colinnordin wrote: post #18

If you haven’t already: Start to store question and answer pairs and reuse the answer if the same question is asked multiple times. You could also compute embeddings for the questions (don’t have to be OpenAI embeddings), and reuse the answer if the question is sufficiently similar to a prevously asked question.

I'm not sure it's practical and if it will result in any savings. Wouldn't it be almost impossible to hit a duplicate when the users each form their own question? Another issue I see is that these chat AIs usually have "history", so the question might be the same, but the context is different: the app might have received "when was he born", but in one context, the user talks about Obama and in another, she talks abou…

With embeddings you can compute distance. The questions don't have to be the same, they just have to be sufficiently close.

Regarding context, that should be a part of the input for the embeddings.

Re: Ask HN: My GPT project has become a local hit but I can't afford the bill

#56

Post by throwuwu »

Premium subscribers. Offer a token benefit to people who pay, can even be something worthless like flair on a username or listing their name in the “credits” or “hall of fame”. Then advertise it on your site and make it easy for people to pay.

Re: Ask HN: My GPT project has become a local hit but I can't afford the bill

#57

Post by scrapcode »

Midjourney has a model that worked for me ... you can queue into the "free lane" which is slower and limited by total requests for the day, and I assume is adjusted for how successful they get and are willing to spend. Or, you can subscribe for a low fee and get instant access.

Re: Ask HN: My GPT project has become a local hit but I can't afford the bill

#58

Post by scosman »

Traffic is most likely a spike. In a month it will probably be down to a few percent of that. I wouldn’t shutdown or optimize too much unless you see it persist for a few more weeks at this level.

If your lucky and it’s still going strong after that time you’ve got something worth more than your hosting bills.

Re: Ask HN: My GPT project has become a local hit but I can't afford the bill

#59

Post by ke88y »

You could try to apply for grants. 20K+ chats sounds like a large number -- you must live in a 1M+ person metro area, right? It would probably be fairly easy to get a four figure grant from one agency or another. You could frame it as "localization of generative ai to the regional culture of X", or as a tourism thing, etc.

You could also sell advertising the old-fashioned way -- ask local businesses or perhaps a local university if they'd like the model to mention them in certain prompts.

But your best bet is probably to treat this as a loss leader and put up a small banner offering "In-Person Generative AI Consulting" in your metro area.

Re: Ask HN: My GPT project has become a local hit but I can't afford the bill

#60

Post by lawik »

austhrow743 wrote: post #21

There will be local businesses with fairly aggressive marketing. Tons of billboards, on buses, local newspaper, etc. Should be easy to find because they're the ones advertising everywhere. Call them up one by one and say "hey I run [project] that everyone has been using lately, would you be interested in having your business name listed as a sponsor? It'll be shown on 1/4 page loads which is currently 5000 times a da…

This seems like the best play since it is so locally relevant.
Post reply on HN