Ah! Is Q* a play on the A* pathfinding algorithm, but with a Q presumably for the Query vector of attention mechanisms? Clever, and actually does reveal something about the project.
Please ignore the deluge of complete nonsense about Q*
101–110 of 208 posts
Re: Please ignore the deluge of complete nonsense about Q*
#102The leak about Q* feels like an olive branch to let the former OpenAI board and Ilya save a bit of face, probably part of the terms for Sam coming back plus it distracts from all the drama and puts a positive spin on things. >"The board didn't handle things well, but they were right to be concerned because OpenAI did have some sort of research breakthrough" Not coincidence that this leaks after Sam comes back, rather…
People always, always want random human bullshit to be intentional, rational, or preplanned by people smarter than the pliable masses. So they claim there's a secret grand motive behind every piece of information that gets leaked.
It's to save face.
It's all a cover to hide a big R&D discovery they made but don't want to reveal, so they made it about personal dispute with Sam.
It's a marketing ploy to hype people up about a new OpenAI product.
etc etc
Re: Please ignore the deluge of complete nonsense about Q*
#103One thing that makes no sense about Sam Altman hiding the Q* progress from the board is that Ilya Sutskever was part of the board too. As he is the technical lead and Altman isn't technical at all, how would it be a secret from Sutskever?
NB: I know nothing about these people and don't track it, but my consistency check failed on this claim based on the flood of news over the past week.
Re: Please ignore the deluge of complete nonsense about Q*
#104Earlier quoted context omitted.
> The default in science is that the side arguing a point has the burden of proving it correct Right, and Yann is arguing the point that AI and LLMs are not or will not be dangerous. Where's his proof? As the parent posters have said, he has none.
Show me LLM that has reached a level of automony and self sufficiency of a cat (Yann's example) or even of a simple beattle. LLMs are tools in amplifying individual human intelligence, 100% of automony and will come from their user (human). Also, even as tools they have fundamental limitations which stem from their autoregressive nature
/even of a simple beattle./
So, more Ringo than Lenon?
Re: Please ignore the deluge of complete nonsense about Q*
#105One thing that makes no sense about Sam Altman hiding the Q* progress from the board is that Ilya Sutskever was part of the board too. As he is the technical lead and Altman isn't technical at all, how would it be a secret from Sutskever? NB: I know nothing about these people and don't track it, but my consistency check failed on this claim based on the flood of news over the past week.
Excellent point. I hadn't thought of that.
Re: Please ignore the deluge of complete nonsense about Q*
#106Earlier quoted context omitted.
If you do search for Reinforcement Learning based decoding in language models (or neural machine translation) you'll find hundreds of papers. It's a super common research topic that has been around forever, way before LLMs were a "thing". Just follow the reference from there. Nothing new here, really. But doing it on the scale of something like GPT-4 is only something OAI can do, and I don't doubt that there's some b…
> But doing it on the scale of something like GPT-4 is only something OAI can do It's shocking to me that people think Google and Meta don't have the scale to operate at this level.
They have the scale in terms of hardware, but they likely don't have the infrastructure to experiment efficiently on this scale on LLMs specifically. OpenAI has been building LLM-specific infrastructure for years. Why do you think Google Bard sucks so much? It's not because there's some special magic secret that only OAI knows. It's because you need a huge amount of purpose-built infrastructure and trial-and-error iteration from experiments each of which take months. You can't easily catch up to that in a few months. Even if you throw 10x more staff at it, you are not going to make progress faster.
The only way you could catch up faster is by having orders of magnitude more hardware. But due the global shortage on hardware right now, nobody has that, not even Google.
Re: Please ignore the deluge of complete nonsense about Q*
#107Ah! Is Q* a play on the A* pathfinding algorithm, but with a Q presumably for the Query vector of attention mechanisms? Clever, and actually does reveal something about the project.
This question has been discussed to death in multiple other threads, just search Q* on hn.
Re: Please ignore the deluge of complete nonsense about Q*
#108The leak about Q* feels like an olive branch to let the former OpenAI board and Ilya save a bit of face, probably part of the terms for Sam coming back plus it distracts from all the drama and puts a positive spin on things. >"The board didn't handle things well, but they were right to be concerned because OpenAI did have some sort of research breakthrough" Not coincidence that this leaks after Sam comes back, rather…
To be fair, Altman separately alluded to a recent big discovery/breakthrough in a talk shortly before all the drama went down.
Could it have been due to him breaking some kind of NDA? After all that would make sense then that they're so mysterious about it, after all they don't want to release more information.
Re: Please ignore the deluge of complete nonsense about Q*
#109One thing that makes no sense about Sam Altman hiding the Q* progress from the board is that Ilya Sutskever was part of the board too. As he is the technical lead and Altman isn't technical at all, how would it be a secret from Sutskever? NB: I know nothing about these people and don't track it, but my consistency check failed on this claim based on the flood of news over the past week.
Maybe it wasn't secret, and Ilya wanted to bring it to the board but Sam fought him.
Re: Please ignore the deluge of complete nonsense about Q*
#110I dunno all this talk about OpenAI lately really feels more and more like hype to me.
What they've accomplished so far is really impressive. But until I see AGI for real I won't hold my breath.
Just feels to me like some expert/wizard level PR is going on right now.