Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?

news.ycombinator.com

Post reply on HN
Print view View original

161–170 of 379 posts

Re: Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?

#161

Post by abletonlive »

YossarianFrPrez wrote: post #154
catigula wrote: post #148

Earlier quoted context omitted.

A lot of really smart people working on problems that don't even really need to be solved is an interesting aspect of market allocation.

Can you explain what you mean about 'not needing to be solved'? There are versions of that kind of critique that would seem, at least on the surface, to better apply to finance or flash trading. I ask because scaling an system that a substantially chunk of the population finds incredibly useful, including for the more efficient production of public goods (scientific research, for example) does seem like a problem tha…

They won’t be honest and explain it to you but I will. Takes like the one you’re responding to are from loathsome pessimistic anti-llm people that are so far detached from reality they can just confidently assert things that have no bearing on truth or evidence. It’s a coping mechanism and it’s basically a prolific mental illness at this point

Re: Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?

#162

Post by kazinator »

One trick is that multiple prompts are combined into a batch that is subject to the token prediction simultaneously.

I've had Google Gemini start talking to be in the context of someone else's chat, unrelated to mine.

ME: Does POSIX have any API for creating a filesystem object (such as a file) with a predetermined inode number supplied by the caller?

AI: 

  \documentclass{article}
  \usepackage{amsmath}
  \usepackage{amsfonts}
  \usepackage{amssymb}
  \begin{document}
  \section*{Does POSIX have any API for creating a filesystem object with a predetermined inode number supplied by the caller?}
  \textbf{No, POSIX does not provide a standard API for creating a filesystem object (like a file or directory) with a caller-specified inode number.}

  \subsection*{Inode Allocation in POSIX Filesystems}
  [... SNIP]
ME: Why are you spewing LaTeX syntax at me?

AI: You are absolutely right! My apologies. That was a mistake on my part. I got carried away with the instruction to use LaTeX for mathematical and scientific notations and incorrectly applied it to the entire response. [... SNIP]

There was no such instruction. I've never chatted with any AI about LaTeX. it leaked from the tokens of someone else's chat.

Re: Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?

#163

Post by virgil_disgr4ce »

catigula wrote: post #148
canyon289 wrote: post #6

I work at Google on these systems everyday (caveat this is my own words not my employers)). So I simultaneously can tell you that its smart people really thinking about every facet of the problem, and I can't tell you much more than that. However I can share this written by my colleagues! You'll find great explanations about accelerator architectures and the considerations made to make things fast. https://jax-ml.git…

A lot of really smart people working on problems that don't even really need to be solved is an interesting aspect of market allocation.

> working on problems that don't even really need to be solved

Very, very few problems _need_ to be solved. Feeding yourself is a problem that needs to be solved in order for you to continue living. People solve problems for different reasons. If you don't think LLMs are valuable, you can just say that.

Re: Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?

#164

Post by hattmall »

catigula wrote: post #157
YossarianFrPrez wrote: post #154

Earlier quoted context omitted.

Can you explain what you mean about 'not needing to be solved'? There are versions of that kind of critique that would seem, at least on the surface, to better apply to finance or flash trading. I ask because scaling an system that a substantially chunk of the population finds incredibly useful, including for the more efficient production of public goods (scientific research, for example) does seem like a problem tha…

[flagged]

The only solution to those people starving to death is to kill the people that benefit from them starving to death. It's a solved problem, the solution isn't palatable. No one is starving to death because of a lack of engineering prowess.

Re: Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?

#166

Post by catigula »

hattmall wrote: post #164
catigula wrote: post #157

Earlier quoted context omitted.

[flagged]

The only solution to those people starving to death is to kill the people that benefit from them starving to death. It's a solved problem, the solution isn't palatable. No one is starving to death because of a lack of engineering prowess.

Figuring out how to align misaligned incentives is an engineering problem. Obviously I disavow what you said, I reject all forms of advocacy of violence.

Re: Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?

#167

Post by catigula »

virgil_disgr4ce wrote: post #163
catigula wrote: post #148

Earlier quoted context omitted.

A lot of really smart people working on problems that don't even really need to be solved is an interesting aspect of market allocation.

> working on problems that don't even really need to be solved Very, very few problems _need_ to be solved. Feeding yourself is a problem that needs to be solved in order for you to continue living. People solve problems for different reasons. If you don't think LLMs are valuable, you can just say that.

The notion that simply pretending to not understand that I was making a value judgment about worth is an argument is tiring.

Re: Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?

#169

Post by handfuloflight »

minimaxir wrote: post #2

> Sure, they have huge GPU clusters That's a really, really big "sure." Almost every trick to run a LLM at OpenAI's scale is a trade secret and may not be easily understood by mere mortals anyways (e.g. bare-metal CUDA optimizations)

Trade secrets also exist to hide faults and blemishes.

Re: Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?

#170

Post by _zoltan_ »

Tade0 wrote: post #159
hdgvhicv wrote: post #116

Earlier quoted context omitted.

Opt if you ignore that both gas furnaces and heat pumps are more efficient than resistive loads.

I'm in the market for an oven right now and 230V/16A is the voltage/current the one I'll probably be getting operates under. At 90°C you can do sous vide, so basically use that waste heat entirely. For such temperatures you'd need a CO2 heat pump, which is still expensive. I don't know about gas, as I don't even have a line to my place.

90C for sous vide??? You're going to kill any meal at 90.
Post reply on HN