Remember, about a month ago Sam posted a comment along the lines of "AI will be capable of superhuman persuasion well before it is superhuman at general intelligence, which may lead to very strange outcomes". The board was likely spooked by the recent breakthroughs (which were most likely achieved by combining transformers with another approach), and hit the panic button. Anything capable of "superhuman persuasion",…
OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
821–830 of 1001 posts
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#822Earlier quoted context omitted.
Well, my son is a meat robot who's constantly ingesting information from a variety of sources including but not limited to youtube. His firmware includes a sophisticated realtime operating system that models reality in a way that allows interaction with the world symbolically. I don't think his solving the |i+1| question was founded in linguistic similarity but instead in a physical model / visualization similarity.…
Maybe Altman should just go have some kids and RLHF them instead.
Too many years to max compute. All models limited lifespan inherent.
Avg $200k+ training cost over 18 year in house data center costs. More for reinforcement.
He's still 38. Gates took much longer. To stop working 24/7.
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#823Remember, about a month ago Sam posted a comment along the lines of "AI will be capable of superhuman persuasion well before it is superhuman at general intelligence, which may lead to very strange outcomes". The board was likely spooked by the recent breakthroughs (which were most likely achieved by combining transformers with another approach), and hit the panic button. Anything capable of "superhuman persuasion",…
I agree with this conclusion and it's also why I'm not that afraid of the AGI threat to the human race. AGI won't end the human race if "superhuman persuation" or "deception-as-a-service" does it first.
Superhuman pursuation to do good stuff.
That'll be a weird convo what is 'good'.
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#824Earlier quoted context omitted.
There is a general result in machine learning known as "the bitter lesson"[1], which is that methods which come from specialist knowledge tend to be beaten by methods which rely on brute force computation in the long run because of Moore's law and the ability to scale things by distributed computing. So the reason people don't use the "add instruction"[2] for example is that over the last 70 years of attempting to bu…
As someone with a CS background myself, I don't think this is what GP was talking about. Let's forget for a moment that stuff has to run on an actual machine. If you had to represent a quadratic equation, would you rather write: (a) x^2 + 5x + 4 = 0 (b) the square of the variable plus five times the variable plus four equals zero When you are trying to solve problems with a level of sophistication beyond the toy stuf…
Then as you collect more evidence you find that the age listed on the first one in the census was wildly off due to a transcription error and it's actually her.
You'd think some sort of rule-based system and database might help with these sorts of things. But the historical experience of expert system is that you then often automate the easy bits at the cost of demanding even more tedious data-entry. And you can't divorce data entry and deduction from each other either, because without context, good luck reading out a rare last name in the faded ink of some priest's messy gothic handwriting.
It feels like language models should be able to help. But they can't, yet. And it fundamentally isn't because they suck at grade school math.
Even linguistics, not something I know much about but another discipline where you try to make deductions from tons and tons of soft and vague evidence - you'd think language models, able to produce fluent text in more languages than any human, might be of use there. But no, it's the same thing: it can't actually combine common sense soft reasoning and formal rule-oriented reasoning very well.
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#825Earlier quoted context omitted.
Do you think any rhetoric could ever persuade you to you adopt the opposite general worldview of what you currently have? I'm positive that it could not for me. The reason for this is not because I'm obstinate, but because my worldview is not formed on persuasion, but on lived experience. And I think this is true for the overwhelming majority of people. It's why our views tend to change as we age, and experience more…
Yes. Do not forget that we literally live in the Matrix, getting all the information of import through tiny screens, the sources and validity of which we can only speculate on. All of the validity of the info we have is verified by heuristics we have, like groupthink, listening to 'experts' and trying to match up the info with our internal knowledge and worldview. I feel like our current system of information allows…
I do notice many don't seem to revisit their predictions for reflection though. Perhaps this happens more subconsciously.
I'm wrong regularly ofc.
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#826Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#827I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…
I don't know for Q* of course, but all the tests I made with GPT4, and all what I've read and seen about it, show that it is unable to reason. It was trained with an unfathomable amount of data, so it can simulate reasoning very well, but it is unable to reason
That simulated reasoning is enough to annihilate any human player they're faced with.
As Dijkstra famously said, "whether Machines Can Think... is about as relevant as the question of whether Submarines Can Swim".
Submarines don't swim, cars don't walk or gallop, cameras don't paint or draw... So what?
Once AI can simulate reasoning better than we can do the genuine thing, the question really becomes utterly irrelevant to the likely outcome.
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#828Earlier quoted context omitted.
And what if there are no alignment methods.
Yudkowsky’s doomsday cult almost blew OpenAI to pieces and sent everyone who knows the details in the wind like dandelion seeds. What’s next? A datacenter bombing or killing key researchers? We should be happy that this particular attempt failed, because this cult is only capable of strategic actions that make things far more dangerous. This will be solved like all other engineering and science: with experiments and…
Also non violent.
I think if we have a major AI induced calamity... Then I worry much more. Although... Enough scary capability in a short enough period of time... I could see violence being on the table for the more radical amongst the group.
Your concern is very interesting though, and I think important to consider. I wonder if the FBI agrees.
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#829I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…
Just today I asked GPT4 a simple task. Having mouse position in zoomed and scrolled image find it's position in the original image. GPT4 happily wrote the code, but it was completely wrong. I had to fix it manually.
However, the performance can be increased if there are several threads working on solution. Some suggesting and others analyzing the solution(s). This will increase the size of 'active' memory, at least. And decrease the load on threads, making them more specialized and deeper. This requires more resources, of course. And good management with task split. May be a dedicated thread for that.
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#830I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…
> 4) they (current LLMs) cannot backtrack when they find that what they already wrote turned out not to lead to a solution, and it is too expensive to give them the thousands of restarts they'd require to randomly guess their way through the problem if you did give them that facility This sounds like a reward function? If correctly implemented couldn't it enable an LLM to self-learn?