Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s CEO says the age of giant AI models is already over

wired.com

Post reply on HN
Print view View original

421–430 of 525 posts

Re: OpenAI’s CEO says the age of giant AI models is already over

#421

Post by anthlax »

imposter wrote: post #397
hackernewds wrote: post #377

Earlier quoted context omitted.

The scale of the damage Open AI has done to the trust ecosystem with soliciting not just the work, but also massive fundraising and then privatizing the profits is almost unprecedented and permanent.

> trust ecosystem What trust ecosystem are you talking about ? It was a lack of foresight by google on their own discovery of transformers, and it would probably have been sitting in dust or been killed off by the time it would have taken them to reach GPT-2 level of progress.

The trust ecosystem of the AI community - most every breakthrough was published publicly until OpenAI decided to take advantage of that.

Besides that, this comment contained a ton of statements on what “would” have happened had Google not published. Interesting but worthless way to defend openAI’s actions.

Re: OpenAI’s CEO says the age of giant AI models is already over

#422

Post by anthlax »

bla3 wrote: post #297

In other words, "this approach is getting to expensive for us and we need to control the narrative for our model sizes going down". Most increases in Imagenet etc scores came from bigger models. "The Unreasonable Effectiveness of Data" has aged very well. It seems very convenient for OpenAI that this trend should be over a few months after their "eye-wateringly expensive" comment.

SamA strikes me as overly disingenuous - as if most of his interactions are marketing pieces / warfare. The entire lex + sama podcast seemed this way to me.

I’m incredibly biased though and feel slighted (as a part of humanity) by openAI’s actions, so perhaps I’m looking at his actions through a hateful lens.

Re: OpenAI’s CEO says the age of giant AI models is already over

#423

Post by Ambix »

suby wrote: post #333
bob1029 wrote: post #278

Earlier quoted context omitted.

> There could be exponential or quadratic scaling laws with any of these black boxes that makes one approach suddenly extremely viable or even dominant. The reason I like the CPU approach is the memory scaling is bonkers compared to GPU. You can buy a server that has 12TB of DRAM (in stock right now) for the cost of 1 of those H100 GPU systems. This is enough memory to hold over 3 trillion parameters with full 32-bit…

Is your current prototype open source?

Not the TS, but that's actually the same goal I have in mind with [0] project.

Right now I'm building my homelab server which aimed to fit 1 TB RAM and 2 CPUs with ~100 cores total.

It will cost like 0.1% of what I need to pay for GPU cluster with the same memory size :)

[0] https://github.com/gotzmann/llama.go/

Re: OpenAI’s CEO says the age of giant AI models is already over

#424

Post by famouswaffles »

dataangel wrote: post #336
famouswaffles wrote: post #182

Earlier quoted context omitted.

>How are they at human performance? Because it performs at least average human level (mostly well above average) on basically every task it's given. "Invest something new" is a nonsensical benchmark for human level intelligence. The vast majority of people have never and will never invent anything new. If your general intelligence test can't be passed by a good chunk of humanity then it's not a general intelligence t…

> Because it performs at least average human level (mostly well above average) on basically every task it's given. No, GPT4 fails at some very basic tasks. It can't count letters passed 15.

It doesn't see words.

Re: OpenAI’s CEO says the age of giant AI models is already over

#425

Post by fauxpause_ »

mensetmanusman wrote: post #235
fauxpause_ wrote: post #141

Earlier quoted context omitted.

Seems like a wild claim to make without any examples of gpt models which are bigger and no demonstrably better.

Better on which axis? Do you want an AI that takes one hour to respond to? Some would for certain fields, but getting something fast and cheap is going to be hard now that Moore’s law is over.

Idk. I’m not the one making the claim we’ve already nailed the optimal point though.

Re: OpenAI’s CEO says the age of giant AI models is already over

#426

Post by dragonwriter »

DesiLurker wrote: post #392
dragonwriter wrote: post #388

Earlier quoted context omitted.

> it likely means we might be headed for an AI improvement pause for couple of years after GPT5. I suspect that a pause in base LLM performance won’t be an AI improvement pause; there’s a whole lot of space to improve the parts of AI systems around the core “brain in a jar” model.

I agree, there will be other things to be improved in AI system, but IMHO (tea leaves reading really) it would only lead to incremental improvements in overall systems. Also there is a lot of 'interfacing' work that needs to happen & i suspect that would end up filling the pause, which really is LLM productization loosely speaking. far as AGI is concerned I dont believe LLMs are really the right architecture for it,…

> but IMHO (tea leaves reading really) it would only lead to incremental improvements in overall systems.

It will reach a point where that is the case, sure; it is not there now, and if we are within one model generation of exhausting (for now) major core model improvements, I don’t think we’ll have reached the point of gradual incremental improvement from rest-of-system improvements yet.

Re: OpenAI’s CEO says the age of giant AI models is already over

#427

Post by swexbe »

The_Colonel wrote: post #387
ericabiz wrote: post #215

Earlier quoted context omitted.

Many years ago, there was an image that floated around with Craigslist and all the websites that replaced small parts of it—personals, for sale ads, etc. It turned out the way to beat Craigslist wasn’t to build Yet Another Monolithic Craigslist, but to chunk it off in pieces and be the best at that piece. This is analogous to what’s happening with AI models. Sam Altman is saying we have reached the point where spendi…

> Sam Altman is saying we have reached the point where spending $100M+ trying to “beat” GPT-4 at everything isn’t the future. I don't disagree, but it does align pretty well with the OpenAI business model, no? "No need to develop your own base model, just buy our own"

Isn't this really bad for OpenAI? Essentially, this means Meta/Google/others will catch up to them pretty soon and there is nothing OpenAI can do in the near future to get further ahead.

Re: OpenAI’s CEO says the age of giant AI models is already over

#428

Post by AbrahamParangi »

bob1029 wrote: post #151

I strongly believe the next generation of models will be based upon spiking neural concepts wherein action potentials are lazily-evaluated throughout the network (i.e. event-driven). There are a few neuron models that can be modified (at some expense to fidelity) in order to tolerate arbitrary delays between simulation ticks. Using actual latency between neurons as a means of encoding information seems absolutely ess…

I don’t know how much the biological inspiration is a useful abstraction anymore. I’ve gone from thinking of matrix operations as an efficient way to implement neural networks to thinking of neural networks as a complicated sort of allegory for chained matrix ops and it feels much more natural to me.

Re: OpenAI’s CEO says the age of giant AI models is already over

#429

Post by nerdix »

okwhateverdude wrote: post #415
hn_go_brrrrr wrote: post #394

Earlier quoted context omitted.

I have never been so pleased to not get any of these references.

NewPipe/Freetube/Invidious instances + SponsorBlock API support is really great at filtering all of these useless advertising memes. Somehow ($$$) the behavior of TV adverts culture seeped into YouTube. I've not watched broadcast/cable TV in years, but when I interact with people that do, inevitably, they make similar references. It's super weird.

Wait...

You can auto skip the in-video sponsor ads?

Re: OpenAI’s CEO says the age of giant AI models is already over

#430

Post by mclightning »

anthlax wrote: post #422
bla3 wrote: post #297

In other words, "this approach is getting to expensive for us and we need to control the narrative for our model sizes going down". Most increases in Imagenet etc scores came from bigger models. "The Unreasonable Effectiveness of Data" has aged very well. It seems very convenient for OpenAI that this trend should be over a few months after their "eye-wateringly expensive" comment.

SamA strikes me as overly disingenuous - as if most of his interactions are marketing pieces / warfare. The entire lex + sama podcast seemed this way to me. I’m incredibly biased though and feel slighted (as a part of humanity) by openAI’s actions, so perhaps I’m looking at his actions through a hateful lens.

I feel the same. His narrative seem very calculated. It doesn't help that OpenAI is very much the most closed Saas thing to be released in recent years. I am a paying customer for sometime, have ChatGPT Plus as well. I am still in waiting list for GPT-4 API.

Not only that, but their business model is completely unclear, which is the scariest part for me, as an individual developer.

Let's go one step forward, they use our texts online including this very comment perhaps. They have a software that can analyse all our texts at scale. But we don't get even the API access to this thing.

It is the most self-serving software ever released, consuming the human creativity both the content and the future relevance of it.

Post reply on HN