Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

reuters.com

Post reply on HN
Print view View original

671–680 of 1001 posts

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#671

Post by marvin »

Atheros wrote: post #603
dougmwne wrote: post #330

Earlier quoted context omitted.

Not long at all. Presumably you could write the method on the back of a napkin to lead another top AI researcher to the same result. That’s why trying to sit on breakthroughs is the worst option and making sure they are widely distributed along with alignment methods is the best option.

And what if there are no alignment methods.

Yudkowsky’s doomsday cult almost blew OpenAI to pieces and sent everyone who knows the details in the wind like dandelion seeds. What’s next? A datacenter bombing or killing key researchers? We should be happy that this particular attempt failed, because this cult is only capable of strategic actions that make things far more dangerous.

This will be solved like all other engineering and science: with experiments and iteration, in a controlled setting where potential accidents will have small consequences.

An unaligned system isn’t even useful, let alone safe. If it turns out that unaligned AGI is very hard, we will obviously not deploy it into the world at scale. It’s bad for the bottom line to be dead.

But there’s truly no way out but forward; game theory constrains paranoid actors more than the reckless. A good balance must be found, and we’re pretty close to it.

None of the «lesswrong» doomsday hypotheses have much evidence for them, if that changes then we will reassess.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#672

Post by kenjackson »

wbhart wrote: post #550
Davidzheng wrote: post #511

Earlier quoted context omitted.

I agree that in and of itself it's not enough to be alarmed. Also i have to say i don't really know what grade school mathematics means here(multiplication? Proving triangles are congruent?). But I think the question is, whether the breakthrough is an algorithmic change in reasoning. If it is, then it could challenge all 4 of your limitations. Again this article is low on details so really we are arguing over our bes…

I think maybe I didn't make myself quite clear here. There are already algorithms which can solve advanced mathematical problems 100% reliably (prove theorems). There are even algorithms which can prove any correct theorem that can be stated in a certain logical language, given enough time. There are even systems in which these algorithms have actually been implemented. My point is that no technology which can solve…

“Given enough time” makes that a useless statement. Every kid in college learns this.

The ability to eventually solve a given theorem isn’t interesting — especially if the time is longer than the time left in the universe.

It’s far more interesting to see if an AI can, given an arbitrarily stated problem make clear progress quickly.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#673

Post by danbmil »

zx8080 wrote: post #331
zxcvgrsd2 wrote: post #9

Two Sam Altman comments that seem to be referring to this same Q* discovery. November 18 comment at APEC (just before the current drama) [1]: > On a personal note, like four times now in the history of OpenAI, the most recent time was just in the last couple of weeks , I’ve gotten to be in the room when we pushed the veil of ignorance back and the frontier of discovery forward and a September 22 Tweet [2] > sure 10x…

> > sure 10x engineers are cool but damn those 10,000x engineer/researchers... What was he referring to?

I think he is referring to an AGI that is 10000x more productive than a median programmer

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#674

Post by torginus »

somenameforme wrote: post #590
naasking wrote: post #549

Earlier quoted context omitted.

Even if it were true that human persuasion is of limited use in shifting opinions, the parent posted is talking about superhuman persuasion. I don't think we should just assume those are equally effective.

Do you think any rhetoric could ever persuade you to you adopt the opposite general worldview of what you currently have? I'm positive that it could not for me. The reason for this is not because I'm obstinate, but because my worldview is not formed on persuasion, but on lived experience. And I think this is true for the overwhelming majority of people. It's why our views tend to change as we age, and experience more…

Yes. Do not forget that we literally live in the Matrix, getting all the information of import through tiny screens, the sources and validity of which we can only speculate on.

All of the validity of the info we have is verified by heuristics we have, like groupthink, listening to 'experts' and trying to match up the info with our internal knowledge and worldview.

I feel like our current system of information allows us to develop models that are quite distant from base reality, evidenced by the multitudes of realities existing in people's heads, leading some to question if 'truth' is a thing that can be discovered.

I think as people become more and more Internet-addicted, an increasing amount of our worldviews come through that little screen, instead of real-life experiences.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#675

Post by lovecg »

markisus wrote: post #506
lovecg wrote: post #389

Earlier quoted context omitted.

It does just fine for me (I’m impressed): You: Prove that if cos(x) is an integer then so is sin(x) ChatGPT: To prove this statement, let's first establish some basic facts about the sine and cosine functions: 1. *Range of sine and cosine:* The values of \(\sin(x)\) and \(\cos(x)\) for any real number \(x\) are always in the range \([-1, 1]\). 2. *Pythagorean Identity:* For any angle \(x\), \(\sin^2(x) + \cos^2(x) =…

Interesting. Which version did you use? My experiment failed (GPT 3.5)

GPT4, I find the difference consistently significant

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#676

Post by somenameforme »

placebo wrote: post #626
somenameforme wrote: post #590

Earlier quoted context omitted.

Do you think any rhetoric could ever persuade you to you adopt the opposite general worldview of what you currently have? I'm positive that it could not for me. The reason for this is not because I'm obstinate, but because my worldview is not formed on persuasion, but on lived experience. And I think this is true for the overwhelming majority of people. It's why our views tend to change as we age, and experience more…

While I agree that human persuasion would probably not change a worldview built on lived experience, you can't know in advance what might be possible with superhuman persuasion. You might be led to believe that your experience was interpreted incorrectly, that things are different now or that you live in an illusion and don't even know who you are. There is no way to tell what the limits of psychological manipulation…

Well, but I'm sure you'd accept that there are limits. Where we may differ is where those limits begin and where they end. In the end LLMs are not magical. All it's going to be able to do is present words to you. And how we respond to words is something that we can control. It's not like some series of words is just going to be able to completely reprogram you.

Like here I expect there is 0% chance, even if I had a superhuman LLM writing words for me, that I could ever convince you that LLMs will not be able to convince you to hold any arbitrary position. It's because you've formed your opinion, it's not falsifiable, and so there's not a whole heck of a lot else to be done except have some fun debates like this where, if anything, we tend to work to strengthen our own opinions by finding and repairing any holes in them.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#677

Post by comex »

Closi wrote: post #645
nostrademons wrote: post #593

Earlier quoted context omitted.

What I wonder, as a computer scientist: If you want to solve grade school math problems, why not use an 'add' instruction? It's been around since the 50s, runs a billion times faster than an LLM, every assembly-language programmer knows how to use it, every high-level language has a one-token equivalent, and doesn't hallucinate answers (other than integer overflow). We also know how to solve complex reasoning chains…

Why would we teach kids maths then, when they can use a calculator? It's much easier and faster for them. I believe it's because having a foundational understanding of maths and logic is important when solving other problems, and if you are looking to create an AI that can generally solve all problems it should probably have some intuitive understanding of maths too. i.e. if we want an LLM to be able to solve unsolve…

Judging by some YouTube videos I’ve seen, ChatGPT with GPT-4 can get pretty far through a game of chess. (Certainly much farther than GPT-3.5.) For that duration it makes reasonably strategic moves, though eventually it seems to inevitably lose track of the board state and start making illegal moves. I don’t know if that counts as being able to “actually play a game”, but it does have some ability, and that may have already influenced its answers about the other topics you mentioned.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#678

Post by Davidzheng »

wbhart wrote: post #526
nijave wrote: post #508

Earlier quoted context omitted.

ChatGPT (3.5) seems to do some rudimentary backtracking when told it's wrong enough times. However, it does seem to do very poorly in the logic department. LLMs can't seem to pick out nuance and separate similar ideas that are technically/logically different. They're good at putting things together commonly found together but not so good at separating concepts back out into more detailed sub pieces.

I've tested GPT-4 on this and it can be induced to give up on certain lines of argument after recognising they aren't leading anywhere and to try something else. But it would require thousands (I'm really under exaggerating here) of restarts to get through even fairly simple problems that professional mathematicians solve routinely. Currently the context length isn't even long enough for it to remember what problem i…

Weird time estimate given that a little more than a year ago, the leading use of LLMs was generating short coherent paragraphs (3-4 sentences)

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#679

Post by oezi »

poulpy123 wrote: post #667
wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

I don't know for Q* of course, but all the tests I made with GPT4, and all what I've read and seen about it, show that it is unable to reason. It was trained with an unfathomable amount of data, so it can simulate reasoning very well, but it is unable to reason

What is the difference between simulating reasoning very well and "actual" reasoning?

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#680

Post by nabakin »

wegfawefgawefg wrote: post #343

If I had to guess, the name Q* is pronounced Q Star, and probably the Q refers to Q values or estimated rewards from reinforcement learning, and the star refers to a search and prune algorithm, like A* (A star). Possibly they combined deep reinforcement learning with self training and search and got a bot that could learn without needing to ingest the whole internet. Usually DRL agents are good at playing games, but…

https://youtu.be/PtAIh9KSnjo?t=3754
Post reply on HN