Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What if A.I. doesn't get better than this?

newyorker.com

Post reply on HN
Print view View original

61–70 of 119 posts

Re: What if A.I. doesn't get better than this?

#61

Post by noboostforyou »

supriyo-biswas wrote: post #6
qcnguy wrote: post #5

> You didn’t need a bar chart to recognize that GPT-4 had leaped ahead of anything that had come before. You did though. I remember when GPT-4 was announced, OpenAI downplayed it and Altman said the difference was subtle and wouldn't be immediately apparent. For a lot of the stuff ChatGPT was being used for the gap between 3 and 4 wasn't going to really leap out at you. https://fortune.com/2023/03/14/openai-releases-…

> Back then people were happy if they got models to write a small simple function and it worked. Now they expect models to manipulate large production codebases and get it right first time. This push is mostly coming from the C-level and the hustler types, both of which need this to work out in order for their employeeless corporation fantasy to work out.

> This push is mostly coming from the C-level and the hustler types, both of which need this to work out in order for their employeeless corporation fantasy to work out.

The irony, at least in my mind, is that C-level hustler types are exactly the perfect role to be replaced by "AI" for big cost-savings. For obvious reasons, it won't happen.

Re: What if A.I. doesn't get better than this?

#62

Post by tempodox »

9rx wrote: post #55
rossdavidh wrote: post #15

What happens is they go out of business: "these firms spent five hundred and sixty billion dollars on A.I.-related capital expenditures in the past eighteen months, while their A.I. revenues were only about thirty-five billion." DeepSeek (and the like) will prevent the kind of price increases necessary for them to pay back hundreds of billions of dollars already spent, much less pay for more. If they don't find a way…

> If they don't find a way to make LLMs do significantly more than they do thus far... They only need two things, really: A large user base and a way to include advertising in the responses. The market willing to pay hundreds of billions of dollars will soon follow. The businesses are currently in the user base building stage. Hemorrhaging money to get them is simply the cost of doing business. Once they feel that is…

That's nothing new. The question is, will those users be willing and able to pay ten times as much or more for those same services they get for a significant discount now.

Re: What if A.I. doesn't get better than this?

#63

Post by Yossarrian22 »

marviio wrote: post #24
piskov wrote: post #7

Earlier quoted context omitted.

Because every s-curve looks like an exponent for those in the start. I mean look at the first plane, then first air-jets: it’s understandable to assume we would travel the galaxy in something like 2050. Meanwhile planes are basically the same last 60 years. LLMs are great but I firmly believe that in 2100 all is basically the same as in 2020: no free energy (fusion), no AGI.

OTOH: First flight: 1903. Moon landing: 1969. Humanity went from ”Look, we’re 3 meters off the ground!” to “We just parked on the Moon” in barely a lifetime. 66 years.

And has gone no further in almost 60 years

Re: What if A.I. doesn't get better than this?

#64

Post by olddustytrail »

bee_rider wrote: post #54
stephc_int13 wrote: post #20

My current intuition on this topic is that they are right about scaling but they are training on the wrong data. LLMs were not intended to be the core foundation of artificial intelligence but an experiment around deep learning and language. Its success was an almost accidental byproduct of the availability of large amount of structured data to train from and the natural human bias to be tricked by language (Eliza ef…

Definitely smarter people than me have thought about this already, but I’ve been trying to think about human language and how thoughts form in my head lately. How does thinking feel to you? I feel like thoughts appear in my head conceptually mostly formed, but then I start sequentially coming up with sentences to express them, almost as if I’m writing them down for somebody else. In that process, I edit a bunch, so t…

LLMs aren't monolingual so you might want to expand that beyond English. Consider how multilingual people think.

Also, apparently it's pretty common for people to think in words and have an internal monologue. I hadn't realized this was a thing until recently but it seems many people don't think abstractly as you've described.

Re: What if A.I. doesn't get better than this?

#65

Post by 9rx »

tempodox wrote: post #62
9rx wrote: post #55

Earlier quoted context omitted.

> If they don't find a way to make LLMs do significantly more than they do thus far... They only need two things, really: A large user base and a way to include advertising in the responses. The market willing to pay hundreds of billions of dollars will soon follow. The businesses are currently in the user base building stage. Hemorrhaging money to get them is simply the cost of doing business. Once they feel that is…

That's nothing new. The question is, will those users be willing and able to pay ten times as much or more for those same services they get for a significant discount now.

Will they pay ten times more for a Big Mac? Probably not, but why would they need to? Hundreds of billions is Facebook's revenue. The businesses in this space are there if they can take those customers alone, never mind all the other places where advertising takes place. The market exists, is sufficiently large, and willing to spend. All these "AI" businesses need to do is show that the users are spending time on their services instead, which is exactly what they are working on right now.

The question is really only: Will users actually want to continue to use these services once the novelty wears off? The assumption is that they are useful enough to become an integral part of our lives, but time will tell...

Re: What if A.I. doesn't get better than this?

#66

Post by microtonal »

eddythompson80 wrote: post #59
reactordev wrote: post #50

Earlier quoted context omitted.

Not all AI is LLMs. That's just what's most prevalent right now. There's still great work being done by models that don't "speak" but "perform". The issue is they need to be trained to perform like you said. The more tools like Claude Code are used, the more training they receive as well. I do think we'll see a plateau (if we haven't reached it already) of diminishing returns and we'll seek out new algorithms to impr…

> The more tools like Claude Code are used, the more training they receive as well. What do you mean? A model doesn't improve because it's being used more. Are you saying Anthropic invests more into Claude Code the more people use it? Or are you saying they collect its output and train it on it?

I assume they mean that they can gather users inputs (e.g. the user correcting the model, suggesting improvements, etc.).

Re: What if A.I. doesn't get better than this?

#67

Post by disgruntledphd2 »

izzydata wrote: post #30
lenerdenator wrote: post #17

Earlier quoted context omitted.

Nah, I'm not afraid of working here the rest of my days. Consistent paycheck, benefits, challenging-but-rewarding work. If you provide people with that they typically shut up and stay out of the way. Everyone should be more afraid of the former than the latter.

There is no plan in place at all for the outcome of most work becoming redundant. At least in the US I highly doubt we will be capable of implementing some system such as UBI for the benefit of all citizens so everyone can take advantage of most work being automated. Everyone will be left to pick up scraps and barely survive. But I am extremely skeptical that current "AI" will be capable of eliminating so much of the…

> There is no plan in place at all for the outcome of most work becoming redundant. At least in the US I highly doubt we will be capable of implementing some system such as UBI for the benefit of all citizens so everyone can take advantage of most work being automated. Everyone will be left to pick up scraps and barely survive.

If 80% of US citizens lose their jobs, I assure you that there will be a political response. It might not be one you (or I) like, but it will happen and it will be a big deal.

Re: What if A.I. doesn't get better than this?

#68

Post by AtlasBarfed »

energy123 wrote: post #25

How can you say progress has stalled two weeks after LLMs won gold medals at IOI and IMO? How can you say progress has stalled without having visibility on the compute costs of gpt-5 relative to o3? How can you say progress has stalled by referring to changes in benchmarks at the frontier over just 3.5 months?

My personal test question keeps bombing, and I think it's something they should be capable of doing?

Are those math contests? Are their questions and answers in the training set?

Let's say that these things really won a math Olympiad by thinking. Ok, I would like it to to write parsers based on a well defined expression or language spec. Not as bad as near unparseable C++ or JavaScript.

The AIs refuse, despite the prompt, to write a complete parser, hallucinate tests, do things like just call the already working compiler on the CLI, force repetitive reprompts that still won't complete the task.

To me, this is a good example of a task I would give AI as a service to see if it will reliably do something that's well specified, moderately annoying, and is most definitely in the training set if they are pulling data from "the internet".

Re: What if A.I. doesn't get better than this?

#69

Post by benterix »

9rx wrote: post #65
tempodox wrote: post #62

Earlier quoted context omitted.

That's nothing new. The question is, will those users be willing and able to pay ten times as much or more for those same services they get for a significant discount now.

Will they pay ten times more for a Big Mac? Probably not, but why would they need to? Hundreds of billions is Facebook's revenue. The businesses in this space are there if they can take those customers alone, never mind all the other places where advertising takes place. The market exists, is sufficiently large, and willing to spend. All these "AI" businesses need to do is show that the users are spending time on the…

> The question is really only: Will users actually want to continue to use these services once the novelty wears off? The assumption is that they are useful enough to become an integral part of our lives, but time will tell...

But LLMs do have some niche, stable applications already. For example, they replaced Stack Overflow to a large extent, because you can get the answer you need faster, and it's often better adapted to your situation. So you could argue the novelty of SO wore off a long time ago but people were still using it when LLMs appeared. ChatGPT is no more en vogue, people are ashamed to (and shamed for) using it, but it still has some uses, helping people in their jobs and lives in general.

Re: What if A.I. doesn't get better than this?

#70

Post by ben_w »

mgfist wrote: post #26
rossdavidh wrote: post #15

What happens is they go out of business: "these firms spent five hundred and sixty billion dollars on A.I.-related capital expenditures in the past eighteen months, while their A.I. revenues were only about thirty-five billion." DeepSeek (and the like) will prevent the kind of price increases necessary for them to pay back hundreds of billions of dollars already spent, much less pay for more. If they don't find a way…

DeepSeek is also undercutting itself. No one is making a profit here, everyone is trying to gobble market share. Even if you have the best model and don't care to make a dime, inference is very expensive.

> inference is very expensive

I am surprised that this claim keeps getting made, given the observed prices.

Even if one thinks that the losses of big model providers are due to selling below operating costs (rather than below that plus training costs plus the cost of growth), then even big open-weights models that need beefy machines, look like they eventually* amortise the cost so low that electricity is what matters; so when (and *only* when) the quality is good enough, inference is cheaper than the food needed to have a human work for peanuts — and I mean literally peanuts, not metaphorical peanuts, as in the calories and protein content of bags of peanuts sufficient to not die.

* this would not happen if computers were still following the improvements trends of the 90s, because then we'd be replacing them every few years; a £10k machine that you replace every 3 years cost you £9.13/day even if it did nothing.

https://www.tesco.com/groceries/en-GB/products/300283810 -> £0.59 per bag * (2500 per day/645 per bag) = £2.29/day; then combine your pick about which model, which model of home server, electricity costs etc. with your estimate of how many useful tokens a human does in 8,760 hours per calendar year given your assumptions about hours per working week and days of holiday or sick leave.

I know that even just order-of 100k useful tokens is implausible for any human because that would be like writing a novel a day, every day; and this article (https://aichatonline.org/blog-lets-run-openai-gptoss-officia...) claims a Mac Studio can output 65.9/second = 65.9 * 3600 * 24 = 5,693,760 / day or ~= 2e9/year, compare to a deliberate over-estimate of human output (100k/day * 5 days a week * 47 weeks a year = 2.35e7/year)

The top-end Mac Studio has a maximum power draw of 270 W: https://support.apple.com/en-us/102027

270 W for *at least (2e9/year / 2.35e7/year) 85 times* the quantity (this only matters when the quality is sufficient, and as we all know AI often isn't that good yet) of output that a human can do with 100 W, is a bit over 31 times the raw energy efficiency, and electricity is much cheaper than calories — cheaper food than peanuts could get the cost of the human down to perhaps about £1/day, but even £1/day is equivalent to electricity costing £1/(24 hours * 100 W) = £0.416666… / kWh

Post reply on HN