Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I made the slowest, most expensive GPT

ithy.com

Post reply on HN
Print view View original

51–60 of 61 posts

Re: Show HN: I made the slowest, most expensive GPT

#51

Post by ricktdotorg »

wluk wrote: post #45
sergiotapia wrote: post #9

`how to build a birdhouse to attract a bluebird` https://ithy.com/article/4b116d2032e54c03862db84e71bcfc8f https://big-agi.com/ has this "BEAM" concept as well where you can put your message through as many models as you have configured then run fuse/guided/compare/custom to merge them all together into one comprehensive. more expensive response. https://files.catbox.moe/tr82vs.png https://files.catbox.moe/beuyfx.png…

Thanks! Yeah that's an excellent idea - this is my response from another thread: I have a feeling that Perplexity and ChatGPT are doing something similar [caching], since common questions I'd ask like "top movies this year" will be answered nearly-instantaneously, way faster than GPT-4o could have done on its own. The only explanation for this is that so many users ask certain questions, they cache the response and r…

i started looking into using Cloudflare AI gateways for this exact [caching] reason a few months ago but got distracted with GPU Cloud Run so i never did get decent load/numbers on the AI gateway cache to see if it was worth bothering about

Re: Show HN: I made the slowest, most expensive GPT

#52

Post by ricktdotorg »

wluk wrote: post #16

Update 3:00 PM ET: I've finished scaling up from 2 VPCs to 5 VPCs. Limits have been increased back up to 3 anonymous / 10 signed-in. Update 2:30 PM ET: Back up (for now). Still waiting for Anthropic and Gemini quota increase requests, so those have been migrated to GPT-4o for now. Running on 2 VPCs, in the process of launching 2 more. Confident that I can increase the daily limits by EOD once everything's more stable…

to be fair to you tho, it's a lot easier to be almost-infinite-scale-ultimate-uptime SRE when you have an almost-infinite-scale-company CC/bank draft supporting you instead of your personal dalla is staked on a really cool PoC.

thank you. just seeing this and playing with this has expanded how i think about these type of systems

Re: Show HN: I made the slowest, most expensive GPT

#53

Post by ricktdotorg »

wluk wrote: post #41
LorenDB wrote: post #23

You could at least cheapen some of your queries by moving to something like Groq.

Good idea, maybe I'll add Groq as another option, since I don't have an internal Llama 3.1 flow yet. But I'll still need to keep the others to maintain the diversity of responses.

possible to allow users of ithy the option to add their own API keys for N of your used services to offset your own $?

Re: Show HN: I made the slowest, most expensive GPT

#54

Post by ricktdotorg »

wluk wrote: post #35
zamadatix wrote: post #28

Interesting idea, cool concept. I tried asking "What is the best SNES game most people haven't played". The top answer (Terranigma) was unfortunately the same as I got just asking any of Claude/ChatGPT/Llama/Qwen (maybe too easy a question) but the rest of the list did seem a bit more balanced. Thanks for the free try without a login! Thought: there is a marquee of example queries but it doesn't seem like there is a…

That's a good idea! I have a feeling that Perplexity and ChatGPT are doing something similar, since common questions I'd ask like "top movies this year" will be answered nearly-instantaneously, way faster than GPT-4o could have done on its own. The only explanation for this is that so many users ask certain questions, they cache the response and return the cached answer. I'd love to do this for Ithy, but it'll be a w…

already mentioned in another comment by me but this would/could be a primo use-case of Cloudflare AI gateway. i'm not related/affiliated with them at all, just a user/dev with very little traffic thinking it might help someone with a lot of it

Re: Show HN: I made the slowest, most expensive GPT

#55

Post by wluk »

ricktdotorg wrote: post #52
wluk wrote: post #16

Update 3:00 PM ET: I've finished scaling up from 2 VPCs to 5 VPCs. Limits have been increased back up to 3 anonymous / 10 signed-in. Update 2:30 PM ET: Back up (for now). Still waiting for Anthropic and Gemini quota increase requests, so those have been migrated to GPT-4o for now. Running on 2 VPCs, in the process of launching 2 more. Confident that I can increase the daily limits by EOD once everything's more stable…

to be fair to you tho, it's a lot easier to be almost-infinite-scale-ultimate-uptime SRE when you have an almost-infinite-scale-company CC/bank draft supporting you instead of your personal dalla is staked on a really cool PoC. thank you. just seeing this and playing with this has expanded how i think about these type of systems

haha true, when something goes wrong in big tech, your first thought is "what if we just double the instances and see if it goes away?"

Re: Show HN: I made the slowest, most expensive GPT

#56

Post by wluk »

ProfessorZoom wrote: post #49

so now you summarize everyone’s hallucinations

well that's one possibility. The key (unproven) idea here is that if you use Anthropic to edit o1's responses, it's less likely to hallucinate than if you use o1 to edit o1's responses (which is what o1 actually does).

Re: Show HN: I made the slowest, most expensive GPT

#59

Post by vinni2 »

cootsnuck wrote: post #33
vinni2 wrote: post #20

Earlier quoted context omitted.

this is not ideal. If someone wants to fact check controversial claim filtering it only makes it worse.

This is just a dude's hobby project, chill.

The issue is not about this project. This is a fundamental problem with LLMs. Leaving the decision of what is malicious to LLMs is not ideal.
Post reply on HN