Earlier quoted context omitted.
> 13 000 BTU/hr In sane units: 3.8 kW
> In sane units: 3.8 kW 5.1 Horsepower
> > In sane units: 3.8 kW
> 5.1 Horsepower
0-60 in 1.8 seconds
Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?
201–210 of 379 posts
Re: Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?
#202Earlier quoted context omitted.
I'm in the market for an oven right now and 230V/16A is the voltage/current the one I'll probably be getting operates under. At 90°C you can do sous vide, so basically use that waste heat entirely. For such temperatures you'd need a CO2 heat pump, which is still expensive. I don't know about gas, as I don't even have a line to my place.
How can you bear to eat sous vide though? I've tried it for months and years, and I still find it troublesome. So mushy, nothing enjoy.
Did you skip searing it after sous vide? Did you sous vide it to the "instantly kill all bacteria" temperature (145°F for steak) thereby overcooking & destroying it, or did you sous vide to a lower temperature (at most 125°F) so that it'd reach a medium-rare 130°F-140°F after searing & carryover cooking during resting? It should have a nice seared crust, and the inside absolutely shouldn't be mushy.
Re: Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?
#203Earlier quoted context omitted.
They probably are right, but a counter argument could be how people thought going to the moon was pointless and insanely expensive, but the technology to put stuff in space and have GPS and comms satellites probably paid that back 100x
I don’t mean to invalidate your point (about genuine value arising from innovations originating from the Apollo program), but GPS and comms satellites (and heck, the Internet) are all products of nuclear weapons programs rather than civilian space exploration programs (ditto the Space Shuttle, and I could go on…).
Yes, and no. The people working on GPS paid very close attention to the papers from JPL researchers describing their timing and ranging techniques for both Apollo and deep-space probes. There was more cross-pollination than meets the eye.
Re: Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?
#204Earlier quoted context omitted.
An AI lab with no R&D. Truly a hacker news moment
The unspoken context there is that the inference isn't the thing causing the losses.
Inference contributes to their losses. In January 2025, Altman admitted they are losing money on Pro subscriptions, because people are using it more than they expected (sending more inference requests per month than would be offset by the monthly revenue).
Re: Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?
#205I dunno I ran `ollama run gpt-oss:20b` locally and it only used 16GB locally and I had decent enough inference on my Macbook.
Now do the 120b model.
Re: Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?
#206Earlier quoted context omitted.
> People are starving to death and the world's brightest engineers are ... This is a political will, empathy, and leadership problem. Not an engineering problem.
Those problems might be more tractable if all of our best and brightest were working on them.
>>> People are starving to death and the world's brightest engineers are ...
>> This is a political will, empathy, and leadership problem. Not an engineering problem.
> Those problems might be more tractable if all of our best and brightest were working on them.
The ability to produce enough food for those in need already exists, so that problem is theoretically solved. Granted, logistics engineering[0] is a real thing and would benefit from "our best and brightest."
What is lacking most recently, based on empirical observation, is a commitment to benefiting those in need without expectation of remuneration. Or, in other words, empathetic acts of kindness.
Which is a "people problem" (a.k.a. the trio I previously identified).
Re: Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?
#207Earlier quoted context omitted.
Even is the AI bubble does not pops, your prediction about those servers being available on ebay in 10 years will likely be true, because some datacenters will simply upgrade their hardware and resell their old ones to third parties.
Someone's take on AI was that we're collectively investing billions in data centers that will be utterly worthless in 10 years. Unlike the investments in railways or telephone cables or roads or any other sort of architecture, this investment has a very short lifespan. Their point was that whatever your take on AI, the present investment in data centres is a ridiculous waste and will always end up as a huge net loss…
This isn’t my original take but if it results in more power buildout, especially restarting nuclear in the US, that’s an investment that would have staying power.
Re: Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?
#208Earlier quoted context omitted.
> So I simultaneously can tell you that its smart people really thinking about every facet of the problem, and I can't tell you much more than that. "we do 1970s mainframe style timesharing" there, that was easy
For real. Say it takes 1 machine 5 seconds to reply, and that a machine can only possibly form 1 reply at a time (which I doubt, but for argument). If the requests were regularly spaced, and they certainly won’t be, but for the sake of argument, then 1 machine could serve 17,000 requests per day, or 120,000 per week. At that rate, you’d need about 5,600 machines to serve 700M requests. That’s a lot to me, but not to…
Yes. And batched inference is a thing, where intelligent grouping/bin packing and routing of requests happens. I expect a good amount of "secret sauce" is at this layer.
Here's an entry-level link I found quickly on Google, OP: https://medium.com/@wearegap/a-brief-introduction-to-optimiz...
Re: Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?
#209Earlier quoted context omitted.
It would take talent for them to mess up hosting businesses who want to use their TPUs on GCP. But then again even there, their reputation for abandoning products, lack of customer service, condescension when it came to large enterprises’ “legacy tech” lets Microsoft who is king of hand holding big enterprise and even AWS run rough shod over them. When I was at AWS ProServe, we didn’t even bother coming up with talki…
Google employees collectively have a lot of talent.
A truly astonishing amount of talent applied to… hosting emails very well, and losing the search battle against SEO spammers.
Re: Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?
#210Earlier quoted context omitted.
Yes. Google is probably gonna win the LLM game tbh. They had a massive head start with TPUs which are very energy efficient compared to Nvidia Cards.
The only one who can stop Google is Google. They’ll definitely have the best model, but there is a chance they will f*up the product / integration into their products.
There is plenty of time left to fumble the ball.