31–40 of 141 posts
Also, maybe you can use sometime kind of caching combined with some mbeddongd search to serve the previous response, if the input is similar above a certain threshold.
If you haven’t already: Start to store question and answer pairs and reuse the answer if the same question is asked multiple times. You could also compute embeddings for the questions (don’t have to be OpenAI embeddings), and reuse the answer if the question is sufficiently similar to a prevously asked question.
Wouldn't it be almost impossible to hit a duplicate when the users each form their own question?
Another issue I see is that these chat AIs usually have "history", so the question might be the same, but the context is different: the app might have received "when was he born", but in one context, the user talks about Obama and in another, she talks about Tom Brady.
If there are ways around these issues, I'd love to hear it, but it sounds like this will just increase costs via cache hardware costs and any dedup logic instead of saving money.
I've been wondering how to create a ChatGPT-powered app and not have to spend a fortune on operating it. Use a cheaper model? Limit the input/output length? Train a custom model?
Hugging face and others should help verify if anything can, or how close it is.
Switch out the backend, there are open source alternatives, and the results could be "good enough".
Get a list of local sponsors, offer them 1k ad placements for X dollars/euros/etc... , and inject them into your prompts.
As it is a joke site, I don't believe people will mind if the chat has ads, especially if you mention somewhere that it's financially hard to keep the site up.
There will be local businesses with fairly aggressive marketing. Tons of billboards, on buses, local newspaper, etc. Should be easy to find because they're the ones advertising everywhere. Call them up one by one and say "hey I run [project] that everyone has been using lately, would you be interested in having your business name listed as a sponsor? It'll be shown on 1/4 page loads which is currently 5000 times a da…
If you haven’t already: Start to store question and answer pairs and reuse the answer if the same question is asked multiple times. You could also compute embeddings for the questions (don’t have to be OpenAI embeddings), and reuse the answer if the question is sufficiently similar to a prevously asked question.
I'm not sure it's practical and if it will result in any savings. Wouldn't it be almost impossible to hit a duplicate when the users each form their own question? Another issue I see is that these chat AIs usually have "history", so the question might be the same, but the context is different: the app might have received "when was he born", but in one context, the user talks about Obama and in another, she talks abou…
The embeddings approach would increase the likelyhood of finding the same question, even if phrased slightly differently.
There will be local businesses with fairly aggressive marketing. Tons of billboards, on buses, local newspaper, etc. Should be easy to find because they're the ones advertising everywhere. Call them up one by one and say "hey I run [project] that everyone has been using lately, would you be interested in having your business name listed as a sponsor? It'll be shown on 1/4 page loads which is currently 5000 times a da…
Or charge even more for product placement in your project's prompts.