[dead]
OpenAI’s CEO says the age of giant AI models is already over
511–520 of 525 posts
Re: OpenAI’s CEO says the age of giant AI models is already over
#512Earlier quoted context omitted.
GPT4: In the sentence "What is the third letter in the third word in this sentence?", the third word is "third." The third letter in "third" is 'i'. Given that we have only provided a text prompt and no context or body language hints, GPT seems to interpret this literally. The answer is correct. The third letter of the word "third" is "i".
>GPT seems to interpret this literally. No it doesn't interpret anything. (which is why I've gotten three completely different answers here on HN within 10 minutes). It cannot count at all or even understand the question which becomes obvious when you ask it for word counts or letter counts that don't match anything in the sentence, at which point it makes up nonsense. what is the tenth word in the following sentence…
GPT-4:
what is the tenth word in the following sentence: "Life is great!"
There are only three words in the given sentence: "Life is great!" There is no tenth word in this sentence.
Re: OpenAI’s CEO says the age of giant AI models is already over
#513Earlier quoted context omitted.
Ah yes, it fails to count letters but successfully explains quantum mechanics. It's useless.
Its not a matter of it being useless. Information compression and lookup is a fundamentally different task than reasoning, which GPT models can't do.
Sorry, I should have added /s in the end there
Re: OpenAI’s CEO says the age of giant AI models is already over
#514Earlier quoted context omitted.
He’s not presenting false evidence here, he’s presenting a hunch. It’s a guess. No one is going to gain anything from this one way or another.
> No one is going to gain anything from this one way or another. Then why did Altman even bother to make this very public statement?
He was a guest on an AI-related discussion panel [1]. I did not attend, but I would guess someone asked him “do you think models will have to continuing growing to even larger sizes,” or something like that, and this was his answer.
Honestly, that anyone thinks he has “ulterior motives” here is hilarious to me. Every day I think more and more that we can no longer think critically on the internet.
Re: OpenAI’s CEO says the age of giant AI models is already over
#515Earlier quoted context omitted.
It’s trivial to download some YouTube videos. But I am quite sure that if you start doing it at scale, google will notice. You could be sneaky, but people in this business talk (since they know another good paying job is just around the corner) so It would likely come out.
OpenAI can afford to build a scraping network, built with thousands of proxies.
They can afford to. But will Microsoft risk doing that and massive possible lawsuit from Google.
Google is also owns a lot of it's own backbone, so It would be a lot easier for them to play network games.
And they could even try to be sneaky and try poisoning the data if it comes to that.
And since OpenAI konw that, since they probably have people that used to work at google at some point, they are unlikely to try.
Even less likely if Microsoft would know. MS is probably the only company that has even more layers than Oracle and they would not approve.
Re: OpenAI’s CEO says the age of giant AI models is already over
#516Ilya Sutskever from OpenAI saying that the data situation is good and there's more data to train on - https://youtu.be/Yf1o0TQzry8?t=657
Maybe more data to train on while keeping model size the same
Re: OpenAI’s CEO says the age of giant AI models is already over
#517Earlier quoted context omitted.
> model complexity necessary for spam filtering went down as the size of the data set went up My naive conclusion in that this means there are still massive gains to be had, since, for example, something like ChatGPT is just text, and the phrase "a picture is worth a thousand words" seems incredibly accurate, from my perspective. There's an incredible amount of non-text data out there still. Especially technical data…
Excellent points in your post. You wrote: There's an incredible amount of non-text data out there still. Especially technical data. "Especially technical data." What does this part mean? Initially, I thought you meant things like images and video, but now I am confused.
Schematics (of any sort), block diagrams, general spatial awareness (including anything related to puzzle pieces/packing, like circuit layout), most physics problems involving force diagrams, anything mechanical, etc. The text representation of any of these is ludicrously more complex than simple images.
If you sit someone down, that works in one of these fields, you'll quickly see the limitations. It'll try to represent the concepts as text, with ascii art or some "attempt" at an ascii file format that can be used to draw, and its "reasoning" about these things is much more limited.
I think most people interacting with GPT are in a text-only (and especially programming) bubble.
Re: OpenAI’s CEO says the age of giant AI models is already over
#518Earlier quoted context omitted.
> trust ecosystem What trust ecosystem are you talking about ? It was a lack of foresight by google on their own discovery of transformers, and it would probably have been sitting in dust or been killed off by the time it would have taken them to reach GPT-2 level of progress.
The trust ecosystem of the AI community - most every breakthrough was published publicly until OpenAI decided to take advantage of that. Besides that, this comment contained a ton of statements on what “would” have happened had Google not published. Interesting but worthless way to defend openAI’s actions.
Seems to me that it's very hard to have a moat in LLMs without proprietary code, given that most of their training data is freely available. This is very different form the 2010's era of AI, where they were trained on large amounts of proprietary data that was specific to a given service and could not be replicated.
It's a lot easier to pat yourself on the back for releasing a paper about your techniques when your competitors can't replicate your service with it. I think that as generative AI models move past the hype phase into the competitive phase, they will be keeping a lot of innovation proprietary for at least a few years to maintain an edge over their competitors.
Let's just hope they don't move to patenting everything.
Re: OpenAI’s CEO says the age of giant AI models is already over
#519Earlier quoted context omitted.
The trust ecosystem of the AI community - most every breakthrough was published publicly until OpenAI decided to take advantage of that. Besides that, this comment contained a ton of statements on what “would” have happened had Google not published. Interesting but worthless way to defend openAI’s actions.
Seems to me that it's very hard to have a moat in LLMs without proprietary code, given that most of their training data is freely available. This is very different form the 2010's era of AI, where they were trained on large amounts of proprietary data that was specific to a given service and could not be replicated. It's a lot easier to pat yourself on the back for releasing a paper about your techniques when your co…
The point of OpenAI was that no single company would have a moat around LLM or foundation models in general. It was set as a non profit with this goal in mind and got money for it.
Whether Google pat itself on the back or not for releasing the paper no one could replicate, is not important, because an open research had never been their company’s goal. What happened is a for-profit company released a paper, that allowed a huge advantage to a company, whose mission was to ensure that no one has a huge advantage in the field. OpenAI was converted to for profit and established exclusive relationship with Microsoft.
Google fucked up and missed the train, but they can catch up. Much harder for smaller companies if as a result of this Fb, Google, etc AI research dept lock down their papers to tools to internal use only.