Do anyone know what Q* is?
Q* is the optimal (ie, correct for the decision problem) function computing the total expected reward of taking an action from a given state in reinforcement learning.
That is just Q. The asterisk is new
OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
631–640 of 1001 posts
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#632Earlier quoted context omitted.
If you ask Ilya Sutskever he will say your kids head is full of neurons, so is LLMs. LLMs comsume training data and can then be asked questions. How different is that to your son watching YouTube and then answering questions? It's not 1:1 the same,yet, but it's in the neighborhood.
Well, my son is a meat robot who's constantly ingesting information from a variety of sources including but not limited to youtube. His firmware includes a sophisticated realtime operating system that models reality in a way that allows interaction with the world symbolically. I don't think his solving the |i+1| question was founded in linguistic similarity but instead in a physical model / visualization similarity.…
Maybe Altman should just go have some kids and RLHF them instead.
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#633I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…
> The reason LLMs fail at solving mathematical problems is because That's exactly what Go/Baduk/Weiqi players think some years ago. And superalignment is defintely OpenAI's major research objective: > https://openai.com/blog/our-approach-to-alignment-research > our AI systems are proposing very creative solutions (like AlphaGo’s move 37) When will mathematicians face the move 37 moment?
Probably in <3 years
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#634This matches far better with the board's letter re: firing Sam than a simple power struggle or disagreement on commercialisation. Seeing a huge breakthrough and then not reporting it to the board, who then find out via staff letter certainly counts as a "lack of candour".... As an aside, assuming a doomsday scenario, how long can secrets like this stay outside of the hands of bad actors? On a scale of 1 to enriched u…
[deleted]
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#635Let's repeat this: current LLMs != AGI. It will never be. It could be used for knowledge store, retrieval, synthesis, and low lever reasoning. There is at least a few decades worth of work remaining for it to reach the level of AGI. The proponents in this AI gold rush are casually throwing around the term without any real thought.
Wishful thinkin buddy
let's see buddy
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#636Earlier quoted context omitted.
What I wonder, as a computer scientist: If you want to solve grade school math problems, why not use an 'add' instruction? It's been around since the 50s, runs a billion times faster than an LLM, every assembly-language programmer knows how to use it, every high-level language has a one-token equivalent, and doesn't hallucinate answers (other than integer overflow). We also know how to solve complex reasoning chains…
You're missing the point: who's using the 'add' instruction ? You. We want 'something' to think about using the 'add' instruction to solve a problem. We want to remove the human from the solution design. It would help us tremendously tbh, just like I don't know, Google map helped me never to have to look for direction ever again ?
When the solution requires arithmetic, one trick is to simply ask GPT to write a Python program to compute the answer.
There's your 'add'.
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#637Earlier quoted context omitted.
But he also has the incentive to exaggerate the AI's ability. The whole idea of double-blind test (and really, the whole scientific methodology) is based on one simple thing: even the most experienced and informed professionals can be comfortably wrong. We'll only know when we see it. Or at least when several independent research groups see it.
I thought (and could be wrong) that all of these concerns are based on a very low probability of a very bad outcome. So: we might be close to a breakthrough, that breakthrough could get out of hand, then it could kill a billion+ people.
> I thought (and could be wrong) that all of these concerns are based on a very low probability of a very bad outcome.
Among knowledgeable people who have concerns in the first place, I'd say giving the probability of a very bad outcome of cumulative advances as "very low" is a fringe position. It seems to vary more between "significant" and "close to unity".
There are some knowledgeable people like Yann LeCun who have no concerns whatsoever but they seem singularly bad at communicating why this would be a rational position to take.
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#638Earlier quoted context omitted.
People have done experiments trying to get GPT-4 to come up with viable conjectures. So far it does such a woefully bad job that it isn't worth even trying. Unfortunately there are rather a lot of issues which are difficult to describe concisely, so here is probably not the best place. Primary amongst them is the fact that an LLM would be a horribly inefficient way to do this. There are much, much better ways, which…
After a year the entire argument you make boils down to “so far”.
Whereas your post sounds like "Just give the approach more time, it shall continue to incrementally improve until it finally works someday, cuz reasons."
Early attempts at human flight approached it by strapping wings to people's arms and flapping: Do you think that would have eventually worked too, if only we had just given it a bit more time and faith?
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#639Earlier quoted context omitted.
To quote Reddit user jstadig, > The thing that most worries me about technology is not the technology itself but the greed of those who run it. Someone slimy with limitless ambition like Altman seems to be the worst person to be in charge of things like this.
Why do you perceive Altman as "slimy with limitless ambition"? I've always perceived him as being quite humble from his interviews and podcast appearances.
Actions speak behalf of speech.
You see it from the all commercial deals, protecting company image more that eliminating threats, or even from this post, if it is true.
Not telling about breaktrough in research if it would end the deal with Microsoft.
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#640So can people stop their cyberbullying campaign against the previous batch of board members yet? The conspiratorial hysteria against them, especially Helen Toner and Tasha McCauley, over the weekend was off the charts and straight up vile at times. Tech bros and tech bro wannabes continue to prove the necessity of shoving DEI into STEM disciplines even when it results in nonsensical wasted efforts, because they are i…
The "tech bros" were right: The board absolutely were conspirators wrecking the company based on dogmatic ideology. "Tech bros" had clear evidence of Sutskever, Toner and McCauley's deep and well-known ties to the Effective Altruist (EA) movement and doomerist views. Nobody can doubt Sutskever's technical credentials, whatever his strange beliefs, but Toner and McCauley had no such technical background or even busine…
"valid" criticism? K, so you are one of those wannabes then.