Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s CEO says the age of giant AI models is already over

wired.com

Post reply on HN
Print view View original

111–120 of 525 posts

Re: OpenAI’s CEO says the age of giant AI models is already over

#111

Post by babyshake »

joebiden2 wrote: post #92

We will need a combination of technologies we have in order to really achieve emergent intelligence. Humans are comprised of various "subnets" modelling aspects which, in unison, produce self-conciousness and real intelligence. What is missing in the current line of approaches is that we only rely on auto-alignment of subnetworks by machine learning, which scales only up to a point. If we would produce a model which…

The other elements that may be required could be some version of the continuous sensory input that to us creates the sensation of "living" and, this one is a bit more philosophical, the sensation of suffering and a baseline establishment that the goal of the entity is to take actions that help it avoid suffering.

Re: OpenAI’s CEO says the age of giant AI models is already over

#112

Post by jutrewag »

spaceman_2020 wrote: post #90
thunderbird120 wrote: post #27

>"the company’s CEO, Sam Altman, says further progress will not come from making models bigger. “I think we're at the end of the era where it's going to be these, like, giant, giant models,” he told an audience at an event held at MIT late last week. “We'll make them better in other ways.” So to reiterate, he is not saying that the age of giant AI models is over. Current top-of-the-line AI models are giant and likely…

Is cost really that much of a burden? Intelligence is the single most expensive resource on the planet. Hundreds of individuals have to be born, nurtured, and educated before you might get an exceptional 135+ IQ individual. Every intelligent person is produced at a great societal cost. If you can reduce the cost of replicating a 135 IQ, or heck, even a 115 IQ person to a few thousand dollars, you're beating biology b…

115 IQ isn’t all that high- that’s basically every Indian American or a healthy percentage of the Chinese population.

Edit: I don’t understand the downvotes. I don’t mean this in any disparaging way, just that an AGI is probably going to be a lot higher than that.

Re: OpenAI’s CEO says the age of giant AI models is already over

#113

Post by yunwal »

sitkack wrote: post #88
thewataccount wrote: post #73

Earlier quoted context omitted.

I work for a company with the web development team. We have ~6 software developers. I'd love to be able to just have people submit their resume's and extract the data from there, but instead I'm going to build a form and make applicants fill it out because chatGPT is going to be at least $0.05USD depending on the length of the resume. I'd also love to have mini summeries of order returns summerized in human form, but…

5 cents. Per resume. $500 per 10k. 1-3 hours of a fully loaded engineers salary per year. You are being criminally cheap.

Also you could likely use GPT3.5 for this and still get near perfect results.

Re: OpenAI’s CEO says the age of giant AI models is already over

#114

Post by uoaei »

junipertea wrote: post #64
no_wizard wrote: post #61

Earlier quoted context omitted.

All the LC grinding may come in handy after all! /s What algorithms specifically show the most results upon improvement? Going into this I thought the jump of improvements were really related more advanced automated tuning and result correction, in which it could be done at scale as it were allowing a small team of data scientists to tweak the models until desired results were being achieved. Are you saying instead,…

We need more data efficient neural network architectures. Transformers work exceptionally well because they allow us to just dump more data into it, but ultimately we want to learn advanced behavior without having to feed it Shakespeare

Inductive Bias Is All You Need

Re: OpenAI’s CEO says the age of giant AI models is already over

#115

Post by transcriptase »

cowpig wrote: post #82
boringuser2 wrote: post #78

Eh. Altman has a financial incentive to lie and obfuscate about what it takes to train a model like GPT-4 and beyond, so his word is basically worthless.

OpenAI has gone from open-sourcing its work, to publishing papers only, to publishing papers that omit important information, to GPT-4 being straight-up closed. And Sam Altman doesn't exactly have a track record of being overly concerned about the truth of his statements.

I had a fun conversation (more like argument) with ChatGPT about the hypocrisy of OpenAI. It would explicitly contradict itself and then began starting every reply with “I can see why someone might think…” and then just regurgitating fluff about democratizing AI. I finally was able to have it define democratization of technology and then recognize the absurdity of using that label to describe a pivot to gating models and being for-profit. Then it basically told me “well it’s for safety and protecting society”.

An AI, when presented with facts counter to what it thought it should say, agreed and basically went: “Won’t someone PLEASE think of the children!”

Love it.

Re: OpenAI’s CEO says the age of giant AI models is already over

#116

Post by yunwal »

throwaway22032 wrote: post #103

I don't understand why size is an issue in the way that is being claimed here. Intelligence isn't like processor speed. If I have a model that has (excuse the attempt at a comparison) 200 IQ, why would it matter that it runs more slowly than a human? I don't think that, for example, Feynman at half speed would have had substantially fewer insights.

We're not going to get a 200 IQ model by simply scaling up the current model, even with all the datacenters in the world running 24/7

Re: OpenAI’s CEO says the age of giant AI models is already over

#117

Post by thewataccount »

whywhywhywhy wrote: post #107
thewataccount wrote: post #48

Earlier quoted context omitted.

Nah - GPT-4 is crazy expensive, paying 20$/mo only get's you 25messages/3hours and it's crazy slow. The api is rather expensive too. I'm pretty sure that GPT-4 is ~1T-2T parameters, and they're struggling to run it(at reasonable performance and profit). So far their strategy has been to 10x the parameter count every GPT generation, and the problem is that there's diminishing returns everytime they do that. AFAIK they…

It's never been in OpenAIs interest to make their model affordable or fast, they're actually incentivized to do the opposite as an excuse to keep the tech locked up. This is why Dall-e 2 ran in a data centre and Stable Diffusion runs on a gamer GPU

I think you're mixing the two. They do have an incentive to make it affordable and fast because that increases the use cases for it, and the faster it is the cheaper it is for them, because the expense is compute time (half the time ~= half the cost).

> This is why Dall-e 2 ran in a data centre and Stable Diffusion runs on a gamer GPU

This is absolutely why they're keeping it locked up. By simply not releasing the weights, you can't run Dalle2 locally, and yeah they don't want to do this because they want you to be locked to their platform, not running it for free locally.

Re: OpenAI’s CEO says the age of giant AI models is already over

#119

Post by treis »

spaceman_2020 wrote: post #94
mrtksn wrote: post #81

Earlier quoted context omitted.

You can transcribe all spoken words everywhere and keep the model up to date? Keep indexing new data from chat messages, news articles, new academic work etc. The data is not finite.

What about all the siloed content kept inside corporate servers? You won't get normal GPT to train on it, of course, but IBM could build a "IBM-bot" that has all the GPT-4 dataset + all of IBM's internal data. That model might be very well tuned to solve IBM's internal problems.

I don't think you can just feed it data. You've got to curate it, feed it to the LLM, and then manually check/further train the output.

I also question that most companies have the volume and quality of data worth training on. It's littered with cancelled projects, old products, and otherwise obsolete data. That's going to make your LLM hallucinate/give wrong answers. Especially for regulated and otherwise legally encumbered industries. Like can you deploy a chat bot that's wrong 1% or 0.1% of the time?

Re: OpenAI’s CEO says the age of giant AI models is already over

#120

Post by throwaway5959 »

neel8986 wrote: post #71
machdiamonds wrote: post #41

Earlier quoted context omitted.

Ilya Sutskever (OpenAI Chief Scientist): "Yeah, I would say the data situation is still quite good. There's still lots to go" - https://youtu.be/Yf1o0TQzry8?t=685 There was a rumor that they were going to use Whisper to transcribe YouTube videos and use that for training. Since it's multimodal, incorporating video frames alongside the transcriptions could significantly enhance its performance.

And why will google allow them to do that at scale?

Why would they ask Google for permission?
Post reply on HN