I want to _read_ science fiction, not live it.
[deleted]
OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
621–630 of 1001 posts
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#622Earlier quoted context omitted.
Exactly. The rational fear is that they will automate many lower middle class jobs and cause unemployment, not that Terminator was a documentary.
By this logic we should just forbid the wheel. Imagine how many untrained people could work in transport and there would always be demand. So why did the wheel not result in mass unemployment? And factories neither? Certainly it should have happened already but somehow it never did...
I think the argument here is that we are losing the _good_ jobs. It's like we're automating painting, arts and poetry instead of inventing the wheel. I don't fully agree with this premise (lots of intelectual work is rubbish) but it does sound much more fair when put this way.
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#623Earlier quoted context omitted.
But they didn’t hit the panic button. They said Sam lied to them about something and fired him.
According to this article Sam has been telling the board that this new advance is not AGI and not anything to worry about (so they can keep selling it to MSFT), then the researchers involved went behind Sam's back and reported to the board directly, claiming that they'd created something that could-maybe-be AGI and it needs to be locked down. That's the claim at least.
If that research team is unwanted at OpenAI, I know places they can go with coworkers writing to their boss’s boss.
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#624Earlier quoted context omitted.
Friend, the creator of this new progress is a machine learning PhD with a decade of experience in pushing machine learning forward. He knows a lot of math too. Maybe there is a chance that he too can tell the difference between a meaningless advance and an important one?
But he also has the incentive to exaggerate the AI's ability. The whole idea of double-blind test (and really, the whole scientific methodology) is based on one simple thing: even the most experienced and informed professionals can be comfortably wrong. We'll only know when we see it. Or at least when several independent research groups see it.
> even the most experienced and informed professionals can be comfortably wrong
That's the human hallucination problem. In science it's a very difficult issue to deal with, only in hindsight you can tell which papers from a given period were the good ones. It takes a whole scientific community to come up with the truth, and sometimes we fail.
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#625Earlier quoted context omitted.
But, isn't AlphaGo a solution to kind of specific mathematical problem? And that it has passed with flying colors? What I mean is, yes, neural networks are stochastic and that seems to be why they're bad at logic; on the other hand it' not exactly hallucinating a game of Go, and that seems different to how neural networks are prone to hallucination and confabulation on natural language or X-ray imaging.
Sure, but people have already applied deep learning techniques to theorem proving. There are some impressive results (which the press doesn't seem at all interested in because it doesn't have ChatGPT in the title). It's really harder than one might imagine to develop a system which is good at higher order logic, premise selection, backtracking, algebraic manipulation, arithmetic, conjecturing, pattern recognition, vi…
You seem knowledgeable. Can you share a couple of interesting papers for theorem proving that came out in the last year? I read a few of them as they came out, and it seemed neural nets can advance the field by mixing "soft" language with "hard" symbolic systems.
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#626Earlier quoted context omitted.
Even if it were true that human persuasion is of limited use in shifting opinions, the parent posted is talking about superhuman persuasion. I don't think we should just assume those are equally effective.
Do you think any rhetoric could ever persuade you to you adopt the opposite general worldview of what you currently have? I'm positive that it could not for me. The reason for this is not because I'm obstinate, but because my worldview is not formed on persuasion, but on lived experience. And I think this is true for the overwhelming majority of people. It's why our views tend to change as we age, and experience more…
While I agree that human persuasion would probably not change a worldview built on lived experience, you can't know in advance what might be possible with superhuman persuasion. You might be led to believe that your experience was interpreted incorrectly, that things are different now or that you live in an illusion and don't even know who you are. There is no way to tell what the limits of psychological manipulation are for reprogramming your beliefs unless you are totally above any human doubt about everything, which is in itself a sad state to be in.
I hope that persuaded you :)
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#627Insufficient information. Did OpenAI have a breakthrough, or not?
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#628I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…
Let's say a model runs through a few iterations and finds a small, meaningful piece of information via "self-play" (iterating with itself without further prompting from a human.) If the model then distills that information down to a new feature, and re-examines the original prompt with the new feature embedded in an extra input tensor, then repeats this process ad-infinitum, will the language model's "prime directive…
Yes, it seems like this is a direction to replace RLHF so another way to scale without baremetal and if not this then still just a matter of time before some model optimization outperforms the raw epoch/parameters/token approach.
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#629Earlier quoted context omitted.
Friend, the creator of this new progress is a machine learning PhD with a decade of experience in pushing machine learning forward. He knows a lot of math too. Maybe there is a chance that he too can tell the difference between a meaningless advance and an important one?
But he also has the incentive to exaggerate the AI's ability. The whole idea of double-blind test (and really, the whole scientific methodology) is based on one simple thing: even the most experienced and informed professionals can be comfortably wrong. We'll only know when we see it. Or at least when several independent research groups see it.
Unlikely. We'll know when OpenAI has declared itself ruler of the new world, imposes martial law, and takes over.
Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster
#630I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…
Back-tracking is a very nearly solved problem in the context of Prolog-like languages or mathematical theorem provers (as you probably well know). There are many ways you could integrate an LLM-like system into a tactic-based theorem prover without having to restart from the beginning for each alternative. Simply checkpointing and backtracking to a checkpoint would naively improve upon your described Monte Carlo algo…
You can also consider the chatGPT app as a RL environment. The environment is made of the agent (AI), a second agent (human), and some tools (web search, code, APIs, vision). This grounds the AI into human and tool responses. They can generate feedback that can be incorporated into the model by RL methods.
Basically every reply from a human can be interpreted as a reward signal. If the human restates the question, it means a negative reward, the AI didn't get it. If the human corrects the AI, another negative reward, but if they continue the thread then it is positive. You can judge turn-by-turn and end-to-end all chat logs with GPT4 to annotate.
The great thing about chat based feedback is that it is scalable. OpenAI has 100M users, they generate these chat sessions by the millions every day. Then they just need to do a second pass (expensive, yes) to annotate the chat logs with RL reward signals and retrain. But they get the human-in-the-loop for free, and that is the best source of feedback.
AI-human chat data is in-domain for both the AI and human, something we can't say about other training data. It will contain the kind of mistakes AI does, and the kind of demands humans want to solve with AI. My bet is that OpenAI have realized this and created GPTs in order to enrich and empower the AI to create the best training data for GPT-5.
The secret sauce of OpenAI is not their people, or Sam, or the computers, but the training set, especially the augmented and synthetic parts.