341–350 of 525 posts
I'd bet that what he, and the competition, is realizing is that the bigger models are too expensive to run. Pretty sure Microsoft swapped out Bing for something a lot smaller in the last couple of weeks; Google hasn't even tried to implement a publicly available large model. And OpenAI still has usage caps on their GPT-4. I'd bet that they can still see improvement in performance with GPT-5, but that when they look a…
dude someone lied their way into being president of the united states all while people fact checked him basically immediately after each lie. lying doesnt make a difference.
[flagged]
This isn't about some dumb grudge.
The point is that now we're at the point of diminishing return for increasing model size, unless we find a better modeling architecture than Transformer. I think this is likely true; while all the other companies underestimated the capability of transformer (including Google itself!), OpenAI made a fairly accurate bet on the transformer based on the scaling law, put all the efforts to squeeze it until the last drop a…
Earlier quoted context omitted.
I suspect you are right. We may be stuck at the gpt4 sizes for a bit just because of hardware costs though. As they get bigger it costs too much to run them until our hardware becomes more optimal for these large models at 4 bits or so. I think the YouTube videos is going to be the next big training set. A transformer trained on all text and all of YouTube will be killer amazing at so much. I bet it can understand lo…
> A transformer trained on all text and all of YouTube will be killer amazing at so much. ah yes, a bot where the answer to everything is to buy ridge wallets and play raid shadow legends
Earlier quoted context omitted.
Seems like a wild claim to make without any examples of gpt models which are bigger and no demonstrably better.
https://twitter.com/SmokeAwayyy/status/1646670920214536193 Sam explicitly said that there won't be GPT-5 in the near future, which is pretty clear evidence unless he's blatantly lying in public speaking.
Earlier quoted context omitted.
Does he even have any background in machine learning? I always found it bizarre that he was chosen to be OpenAI's CEO...
On the Lex Fridman podcast, he pretty much admitted he's not an AI (per se) and isn't the most excited about the tech (as he could be).
Earlier quoted context omitted.
> A transformer trained on all text and all of YouTube will be killer amazing at so much. ah yes, a bot where the answer to everything is to buy ridge wallets and play raid shadow legends
They'll give it the premium tier at least
"The great supercomputer Deep Thought is the most powerful computer ever built, with one exception. It was designed by hyper-intelligent pan-dimensional beings, who wanted to know the answer to Life, the Universe, and Everything. Its creation annoyed a fair few philosophers, who felt that it was taking over their turf. After seven and a half million years of serious cogitation, Deep Thought spoke the answer. However,…