Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

reuters.com

Post reply on HN
Print view View original

851–860 of 1001 posts

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#851

Post by lagrange77 »

walthamstow wrote: post #847
Sol- wrote: post #832

Earlier quoted context omitted.

Thanks for the links, very interesting. Wonder how a "self-play" equivalent would look like for LLMs, since they have no easy criterion to evaluate how well they are doing like in Go (as mentioned in the videos).

ChatGPT does have some feedback that can be used to evaluate, in the form of thumbs up/down buttons, which probably nobody uses, and positive/negative responses to its messages. People often say "thanks" or "perfect!" in responses, including very smart people who frequent here.

ChatGPT was trained (in an additional step to supervised learning of the base LLM) with reinforcement learning from human feedback (RLHF) where some contractors were presented with two LLM output to the same prompt and they had to decide, which one is better. This was a core ingredient to the performance of the system.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#852

Post by gumballindie »

xanderlewis wrote: post #19
gumballindie wrote: post #14

> powerful artificial intelligence discovery that they said could threaten humanity Yup. These people are not well.

What do you mean by that?

Altman and those at openai appear to have an obsession with creating fear and panic, almost as if they were projecting schizophrenia on others.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#853

Post by pxeger1 »

sfjailbird wrote: post #816

I have a strong suspicion that this is a purposeful leak to hype up OpenAI's product. It wouldnt't be out of character for some of the people involved, and it's the kind of thing that rampant commercial (and valuation) focus can bring with it.

“Reuters was unable to review a copy of the letter” rings alarm bells for me

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#854

Post by omnicognate »

seanhunter wrote: post #735
nostrademons wrote: post #593

Earlier quoted context omitted.

What I wonder, as a computer scientist: If you want to solve grade school math problems, why not use an 'add' instruction? It's been around since the 50s, runs a billion times faster than an LLM, every assembly-language programmer knows how to use it, every high-level language has a one-token equivalent, and doesn't hallucinate answers (other than integer overflow). We also know how to solve complex reasoning chains…

There is a general result in machine learning known as "the bitter lesson"[1], which is that methods which come from specialist knowledge tend to be beaten by methods which rely on brute force computation in the long run because of Moore's law and the ability to scale things by distributed computing. So the reason people don't use the "add instruction"[2] for example is that over the last 70 years of attempting to bu…

The bitter lesson isn't a "general result". It's an empirical observation (and extrapolation therefrom) akin to Moore's law itself. As with Moore's law there are potential limiting factors: physical limits for Moore's law and availability and cost of quality training data for the bitter lesson.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#855

Post by _heimdall »

nexuist wrote: post #557
_heimdall wrote: post #356

> OpenAI defines AGI as autonomous systems that surpass humans in most economically valuable tasks. Are we really defining intelligence as economic value at this point? This is completely ridiculous. We have yet to decide exactly what human intelligence, how it manifests in the body, or how to reliably measure it. I get that people want to justify developing artificial intelligence before understanding intelligence i…

If it makes money who cares if it counts as real intelligence? If we can automate ourselves into a post scarcity society then everyone and everything is free to be as intelligent and as stupid as it desires.

It'd be one thing if their goal was to build ML tools that they want to market as AI, but they explicitly aim to develop an artificial general intelligence. There's a mountain of risks and ethical questions that we should be answering before even attempting such a goal, and for them to waive it off under the banner of "intelligence is just economic value" is dangerous and horribly irresponsible. If we can automate ourselves into a post scarcity society then everyone and everything is free to be as intelligent and as stupid as it desires.

Even an AGI can't create natural resources out of thin, post scarcity is a pipe dream that at best means we kicked the can down the road.

Based on our actions, society doesn't actually even want to be automated away for a simple life of freedoms where all the basics of life are taken care of. We could have done that a long time ago, instead we just raise the bar and continue to make life more complicated and grow the list of "necessities". People seem to need a sense of purpose in life, taking that away with an AGI-powered system that gives them everything the AGI deems necessary wouldn't end well.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#856

Post by BenoitP »

wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

What do you think of integrating propositional logic, first order logic and sat solvers in LLM output? ie forcing each symbol an LLM outputs to have its place in a formal proposition. And getting a prompt from the user to force that some parts be satisfiable.

I know this is not how us humans craft our thoughts, but maybe an AI can optimize to death the conjunction of these tools. The LLM just being an universal API to the core of formal logic.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#857

Post by _heimdall »

gryfft wrote: post #362
_heimdall wrote: post #356

> OpenAI defines AGI as autonomous systems that surpass humans in most economically valuable tasks. Are we really defining intelligence as economic value at this point? This is completely ridiculous. We have yet to decide exactly what human intelligence, how it manifests in the body, or how to reliably measure it. I get that people want to justify developing artificial intelligence before understanding intelligence i…

I'm as left leaning as HN commenters get, I think, and in terms of quacking like a duck, "surpassing humans in most economically valuable tasks" is 100% the meaningful Turing test in a capitalist society.

Take capitalism out of it, do we really want to boil down intelligence to a calculation of expected economic value?

Why force the term intelligence into it at all if what were talking about is simply automation? We don't have to bastardize the term intelligence along the way, especially when we have spent centuries considering what human intelligence is and how it separates us from other species on the planet.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#858

Post by vidarh »

waveBidder wrote: post #763
gmt2027 wrote: post #749

Earlier quoted context omitted.

We have an algorithm and computational hardware that will tune a universal function approximator to fit any dataset with emergent intelligence as it discovers abstractions, patterns, features and hierarchies. So far, we have not yet found hard limits that cannot be overcome by scaling the number of model parameters, increasing the size and quality of training data or, very infrequently, adopting a new architecture. T…

intelligence isn't a function unless you're talking about over every possible state of the universe.

Intelligence must inherently be a function unless there is a third form of cause-effect transition that can't be modelled as a function of determinism and randomness.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#859

Post by konart »

nopromisessir wrote: post #814
cduzz wrote: post #585

Earlier quoted context omitted.

Maybe; he still needs to finish his damned homework and remember to turn it in. And eat some vegetables.

All those things sound very boring to me. I can offer no concrete solutions. However, I have a friend who graduated from high school #1 of a big class and 2 years early. His mom explained that if he made at least a 1400(of 1600) on his SAT, she would buy him a new gaming computer. He then proceeded to make exactly a 1400. No more. No less. I recommend if you haven't tried already, an iteration to this approach using…

>reward system

Is a good idea if you need a (relatively) constant result because your brain adapts to the idea of getting a satisfying reward.

https://www.youtube.com/watch?v=s5geuTf8nqo (+ some other of his videos about rewards)

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#860

Post by passwordoops »

sfjailbird wrote: post #816

I have a strong suspicion that this is a purposeful leak to hype up OpenAI's product. It wouldnt't be out of character for some of the people involved, and it's the kind of thing that rampant commercial (and valuation) focus can bring with it.

These guys smell so much like Tesla it's not even funny. Very impressive core tech, and genuinely advancing knowledge. But the hype train is just so powerful that the insane (to put it mildly) claims are picked up without any sense of critical thinking by seemingly intelligent people. They're both essentially cults at this point
Post reply on HN