Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s CEO says the age of giant AI models is already over

wired.com

Post reply on HN
Print view View original

101–110 of 525 posts

Re: OpenAI’s CEO says the age of giant AI models is already over

#101

Post by thewataccount »

sitkack wrote: post #88
thewataccount wrote: post #73

Earlier quoted context omitted.

I work for a company with the web development team. We have ~6 software developers. I'd love to be able to just have people submit their resume's and extract the data from there, but instead I'm going to build a form and make applicants fill it out because chatGPT is going to be at least $0.05USD depending on the length of the resume. I'd also love to have mini summeries of order returns summerized in human form, but…

5 cents. Per resume. $500 per 10k. 1-3 hours of a fully loaded engineers salary per year. You are being criminally cheap.

The problem is that it would take us the same amount of time to just add a form with django. Plus you have to handle failure cases, etc.

And yeah I agree this would be a great use-case, and isn't that expensive.

I'd like to do this in lots of places, and the problem is I have to convince my boss to pay for something that otherwise would have been free.

The conversation would be "We have to add these fields to our model, and we either tell django to add a form for them, which will have 0 ongoing cost and no reliance on a third party,

or we send the resume to openai, pay for them to process it, make some mechanism to sanity check what GPT is responding with, alert us if there's issues, and then put it into that model, and pay 5 cents per resume."

> 1-3 hours of a fully loaded engineers salary per year.

That's assuming 0 time to implement, and because of our framework it would take more hours to implement the openai solution (that's also more like 12 hours where we are).

> $500 per 10k.

I can't stress this enough - the alternative is 0$ per 10k. My boss wants to know why we would pay any money for a less reliable solution (GPT serialization is not nearly as reliable as a standard django form).

I think within the next few years we'll be able to run the model locally and throw dozens of tasks just like this at the LLM, just not yet.

Re: OpenAI’s CEO says the age of giant AI models is already over

#102

Post by sangnoir »

ftxbro wrote: post #18
labrador wrote: post #2

https://archive.is/s4V9e He did not say what kind of research strategies or techniques might take its place. In the paper describing GPT-4, OpenAI says its estimates suggest diminishing returns on scaling up model size. Altman said there are also physical limits to how many data centers the company can build and how quickly it can build them.

> In the paper describing GPT-4, OpenAI says its estimates suggest diminishing returns on scaling up model size. I read the two papers (gpt 4 tech report, and sparks of agi) and in my opinion they don't support this conclusion. They don't even say how big GPT-4 is, because "Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about…

OpenAI has likely run into a wall (or is about to) for model size given it's funding amount/structure[1] - unlike its competition who actually own data centers and have lower marginsl costs. It's just like when peak-iPad Apple claimed that a "post-PC" age was upon us.

1. What terms could Microsoft wring out of OpenAI for another funding round?

Re: OpenAI’s CEO says the age of giant AI models is already over

#103

Post by throwaway22032 »

I don't understand why size is an issue in the way that is being claimed here.

Intelligence isn't like processor speed. If I have a model that has (excuse the attempt at a comparison) 200 IQ, why would it matter that it runs more slowly than a human?

I don't think that, for example, Feynman at half speed would have had substantially fewer insights.

Re: OpenAI’s CEO says the age of giant AI models is already over

#104

Post by solveit »

boringuser2 wrote: post #78

Eh. Altman has a financial incentive to lie and obfuscate about what it takes to train a model like GPT-4 and beyond, so his word is basically worthless.

Anyone with the expertise to have insightful takes in AI also has a financial incentive to steer the conversation in particular directions. This is also the case for many, many other fields! You do not become an expert by quarantining your livelihood away from your expertise!

The correct response is not to dismiss every statement from someone with a conflict of interest as "basically worthless", but to talk to lots of people and to be reasonably skeptical.

Re: OpenAI’s CEO says the age of giant AI models is already over

#105

Post by whywhywhywhy »

xt00 wrote: post #7

Saying “hey don’t go down the path we are on, where we are making money and considered the best in the world.. it’s a dead end” rings pretty hollow.. like “don’t take our lunch please?” Might be a similar statement it feels..

Everyone hoping to compete with OpenAI should have an "Always do the opposite of what Sam says" sign on the wall.

Re: OpenAI’s CEO says the age of giant AI models is already over

#106

Post by LoganWhitwer »

Art9681 wrote: post #98
hbn wrote: post #85

Earlier quoted context omitted.

It could also be argued that there's financial incentive to just saying "giving us more money to train bigger models = better AI" forever

I don't think these comments are driven from financial incentives. It's a distraction and only a fool would believe Altman here. What this likely means is they are prioritizing adding more features to their current models while they train the next version. Their competitors scramble to build an LLM with some sort of intelligence parity, when that happens no one will care because ChatGPT has the ecosystem and plugins…

[dead]

Re: OpenAI’s CEO says the age of giant AI models is already over

#107

Post by whywhywhywhy »

thewataccount wrote: post #48
xt00 wrote: post #7

Saying “hey don’t go down the path we are on, where we are making money and considered the best in the world.. it’s a dead end” rings pretty hollow.. like “don’t take our lunch please?” Might be a similar statement it feels..

Nah - GPT-4 is crazy expensive, paying 20$/mo only get's you 25messages/3hours and it's crazy slow. The api is rather expensive too. I'm pretty sure that GPT-4 is ~1T-2T parameters, and they're struggling to run it(at reasonable performance and profit). So far their strategy has been to 10x the parameter count every GPT generation, and the problem is that there's diminishing returns everytime they do that. AFAIK they…

It's never been in OpenAIs interest to make their model affordable or fast, they're actually incentivized to do the opposite as an excuse to keep the tech locked up.

This is why Dall-e 2 ran in a data centre and Stable Diffusion runs on a gamer GPU

Re: OpenAI’s CEO says the age of giant AI models is already over

#108

Post by jutrewag »

brookst wrote: post #93
boringuser2 wrote: post #78

Eh. Altman has a financial incentive to lie and obfuscate about what it takes to train a model like GPT-4 and beyond, so his word is basically worthless.

In this case I think it's Wired that's lying. Altman didn't say large models have no value, or that there will be no more large models, or that people shouldn't invest in large models. He said that we are at the end of the era where capability improvements come primarily from making models bigger. Which stands to reason... I don't think anyone expect us to hit 100T parameters or anything.

What about 1T though, seems silly to stop here.

Re: OpenAI’s CEO says the age of giant AI models is already over

#109

Post by uoaei »

no_wizard wrote: post #61
tfehring wrote: post #20

Related reading: https://dynomight.net/scaling/ In short it seems like virtually all of the improvement in future AI models will come from better algorithms, with bigger and better data a distant second, and more parameters a distant third. Of course, this claim is itself internally inconsistent in that it assumes that new algorithms won't alter the returns to scale from more data or parameters. Maybe a more precise…

All the LC grinding may come in handy after all! /s What algorithms specifically show the most results upon improvement? Going into this I thought the jump of improvements were really related more advanced automated tuning and result correction, in which it could be done at scale as it were allowing a small team of data scientists to tweak the models until desired results were being achieved. Are you saying instead,…

Traditional CS may have something to do with slightly improving the performance by allowing more training for the same compute, but it won't be an order of magnitude or more. The improvements to be gained will be found more in statistics than CS per se.
Post reply on HN