Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mira Murati leaves OpenAI

twitter.com

Post reply on HN
Print view View original

571–580 of 644 posts

Re: Mira Murati leaves OpenAI

#571

Post by trashtester »

gnulinux wrote: post #564
trashtester wrote: post #552

Earlier quoted context omitted.

Nobody really knows what Earth will look like once AGI arrives. It could be anything from extinction, through some Cyberpunk corporate dystopia (like you seem to think) to some kind of Techno-Socialist utopia. One thing it's not likely to be, is a neo-classical capitalist system based on the value of human labor.

> One thing it's not likely to be, is a neo-classical capitalist system based on the value of human labor. I'm finding it difficult to believe this. For me, your comment is accurate (and very insightful) except even a mostly vanilla continuation of the neoliberal capitalist system seems possible. I think we're literally talking about a "singularity" where by definition our fate is not dependent on our actions, and of…

> I'm finding it difficult to believe this.

Pay attention to the whole sentence, especially the last section : "... based on the value of human labor."

It's not that I'm ruling out capitalism as the outcome. I'm simply ruling out the combined JOINT possibility of capitalism COMBINED WITH human labor remaining the base resource within it.

If robotics is going in the direction I expect there will simply be no jobs left that will be done more efficiently by humans than by machines. (ie that robots will match or exceed the robustness, flexibility and cost efficiency of all biology based life forms through breakthroughs in either nanotech or by simply using organic chemistry, DNA, etc to build the robots).

Why pay even $1/day for a human to do a job when a robot can do it for $1/week?

Also, such a capitalist system will almost certainly lead to AGI's becoming increasingly like a new life form, as capitalism between AGI's introduce a Darwinian selection pressure. That will make it hard even for the 100 richest people to retain permanent control.

IF humanity is to survive (for at least a few thousand more years, not just the next 100), we either need some way to ensure alignment. And to do that, we have to make sure that AGI's that optimize resource-control-seeking behaviours have an advantage over those who don't. We may even have to define some level of sophistication where further development is completly halted.

At least until we find ways for humans to merge with them in a way that allows us (at least some of us) to retain our humanity.

Re: Mira Murati leaves OpenAI

#572

Post by snowwrestler »

navjack27 wrote: post #514

Okay hear me out. Restructuring for profit right? There will probably be companies spawned off of all of these leaving. If the government ever wants a third party to oversee safety of openAI wouldn't it be convenient if one of those that left the company started a company that focused on safety. Safe Superintelligence Inc. gets the bid because lobbying because whatever I don't even care what the reason is in this mad…

I think the departures and switch to for-profit model may point in a different direction: that everyone involved is realizing that OpenAI’s current work is not going to lead to AGI, and it’s also not going to change.

So the people who want to work on AGI and safety are leaving to do that work elsewhere, and OpenAI is restructuring to instead focus on wringing as much profit as possible out of their current architecture.

Corporations are actually pretty bad at doing tons of different things simultaneously. See the failure of huge conglomerates like GE, as well as the failure of companies like Bell, Xerox, and Microsoft to drive growth with their corporate research labs. OpenAI is now locked into a certain set of technologies and products, which are attracting investment and customers. Better to suck as much out of that fruit as possible while it is ripe.

Re: Mira Murati leaves OpenAI

#573

Post by NoGravitas »

015a wrote: post #567
edouard-harris wrote: post #463

Earlier quoted context omitted.

In what way was their usage incorrect? They simply said that the brain just predicts next-actions, in response to a statement that an LLM predicts next-tokens. You can believe or disbelieve either of those statements individually, but the claims are isomorphic in the sense that they have the same structure.

Its not that it was used incorrectly: Its that it isn't a word actual humans use, and its one of a handful of dog whistles for "I'm a tech grifter who has at best a tenuous grasp on what I'm talking about but would love more venture capital". The last time I've personally heard it spoken was from Beff Jezos/Guillaume Verdon.

I think we should delve further into that analysis.

Re: Mira Murati leaves OpenAI

#574

Post by NoGravitas »

CooCooCaCha wrote: post #266
iLoveOncall wrote: post #161

Earlier quoted context omitted.

And there's nothing wrong about that: the fact that _artificial intelligence_ will never lead to general intelligence isn't exactly a hot take.

That's both a very general and very bold claim. I don't think it's unreasonable to say that's too strong of a claim given how we don't know what is possible yet and there's frankly no good reason to completely dismiss the idea of artificial general intelligence.

I think the existence of biological general intelligence is a proof-by-existence for artificial general intelligence. But at the same time, I don't think LLM and similar techniques are likely in the evolutionary path of artificial general intelligence, if it ever comes to exist.

Re: Mira Murati leaves OpenAI

#575

Post by HarHarVeryFunny »

trashtester wrote: post #550
drmindle12358 wrote: post #546

Earlier quoted context omitted.

Dude, it's not the LLM that does the reasoning. Rather it's the layers and layers of scaffolding around LLM that simulate reasoning. The moment 'tooling' became a thing for LLM, it reminded me 'rules' for expert system which caused one of the AI winter. The number of 'tools' you need to solve real use cases will be untenable soon enough.

Well, I agree that the part that does the reasoning isn't an LLM in the naive form. But that "scaffolding" seems to be an integral part of the neural net that has been built. It's not some Python for-loop that has been built on top of the neural network to brute force the search pattern. If that part isn't part of the LLM, then o1 isn't really an LLM anymore, but a new kind of model. One that can do reasoning. And if…

Reasoning, just like intelligence (of which it is part) isn't an all or nothing capability. o1 can now reason better than before (in a way that is more useful in some contexts than others), but it's not like a more basic LLM can't reason at all (i.e. generate an output that looks like reasoning - copy reasoning present in the training set), or that o1's reasoning is human level.

From the benchmarks it seems like o1-style reasoning-enhancement works best for mathematical or scientific domains where it's a self-consistent axiom-driven domain such that combining different sources for each step works. It might also be expected to help in strict rule-based logical domains such as puzzles and games (wouldn't be surprising to see it do well as a component of a Chollet ARC prize submission).

Re: Mira Murati leaves OpenAI

#576

Post by svara »

HarHarVeryFunny wrote: post #532
svara wrote: post #445

Earlier quoted context omitted.

No looping, but you can unroll loops to a fixed depth and apply the model iteratively. There obviously is memory and learning. Neuroscience hasn't found the magic dust in our brains yet, either. ;)

Zero memory inside the model from one input (ie token output) to the next (only the KV cache, which is just an optimization). The only "memory" is what the model outputs and therefore gets to re-consume (and even there it's an odd sort of memory since the model itself didn't exactly choose what to output - that's a random top-N sampling). There is no real runtime learning - certainly no weight updates. The weights ar…

I'm not entirely sure what you're arguing for. Current AI models can still get a lot better, sure. I'm not in the AGI in 3 years camp.

But, people in this thread are making philosophically very poor points about why that is supposedly so.

It's not "just" sequence prediction, because sequence prediction is the very essence of what the human brain does.

Your points on learning and memory are similarly weak word play. Memory means holding some quantity constant over time in the internal state of a model. Learning means being able to update those quantities. LLMs obviously do both.

You're probably going to be thinking of all sorts of obvious ways in which LLMs and humans are different.

But no one's claiming there's an artificial human. What does exist is increasingly powerful data processing software that progressively encroaches on domains previously thought to be that of humans only.

And there may be all sorts of limitations to that, but those (sequences, learning, memory) aren't them.

Re: Mira Murati leaves OpenAI

#577

Post by svara »

015a wrote: post #567
edouard-harris wrote: post #463

Earlier quoted context omitted.

In what way was their usage incorrect? They simply said that the brain just predicts next-actions, in response to a statement that an LLM predicts next-tokens. You can believe or disbelieve either of those statements individually, but the claims are isomorphic in the sense that they have the same structure.

Its not that it was used incorrectly: Its that it isn't a word actual humans use, and its one of a handful of dog whistles for "I'm a tech grifter who has at best a tenuous grasp on what I'm talking about but would love more venture capital". The last time I've personally heard it spoken was from Beff Jezos/Guillaume Verdon.

You know, you can just talk to me about my wording. Where do I meet those gullible venture investors?

Re: Mira Murati leaves OpenAI

#578

Post by trashtester »

HarHarVeryFunny wrote: post #575
trashtester wrote: post #550

Earlier quoted context omitted.

Well, I agree that the part that does the reasoning isn't an LLM in the naive form. But that "scaffolding" seems to be an integral part of the neural net that has been built. It's not some Python for-loop that has been built on top of the neural network to brute force the search pattern. If that part isn't part of the LLM, then o1 isn't really an LLM anymore, but a new kind of model. One that can do reasoning. And if…

Reasoning, just like intelligence (of which it is part) isn't an all or nothing capability. o1 can now reason better than before (in a way that is more useful in some contexts than others), but it's not like a more basic LLM can't reason at all (i.e. generate an output that looks like reasoning - copy reasoning present in the training set), or that o1's reasoning is human level. From the benchmarks it seems like o1-s…

o1 has moved "reasoning" from training time to partly something happening at inference time.

I'm thinking of this difference as analogus to the difference between my (as a human) first intution (or memory) about a problem to what I can achieve by carefully thinking about it for a while, where I can gradually build much more powerful arguments, verify if they work and reject parts that don't work.

If you're familiar with chess terminology, it's moving from a model that can just "know" what the best move is to one that combines that with the ability to "calculate" future moves for all of the most promising moves, and several moves deep.

Consider Magnus Carlsen. If all he did was just did the first move that came to his mind, he could still beat 99% of humanity at chess. But to play 2700+ rated GM's, he needs to combine it with "calculations".

Not only that, but the skill of doing such calculations must also be trained, not only by being able to calculate with speed and accuracy, but also by knowing what parts of the search tree will be useful to analyze.

o1 is certainly optimized for STEM problems, but not necessarily only for using strict rule-based logic. In fact, even most hard STEM problems need more than the ability to perform deductive logic to solve, just like chess does. It requires strategical thinking and intuition about what solution paths are likely to be fruitful. (Especially if you go beyond problems that can be solved by software such as WolframAlpha).

I think the main reason STEM problems was used for training is not so much that they're solved using strict rule-based solving strategies, but rather because a large number of such problems exist that have a single correct answer.

Re: Mira Murati leaves OpenAI

#579

Post by kolbe »

chairmansteve wrote: post #401
kolbe wrote: post #346

Earlier quoted context omitted.

Assuming they're 40, how far do you think $250k will go 20-30-40 years from now? It's not a stretch to think dollars could be devalued by 90%, possibly even worthless, within 30 years.

They obviously don't keep it dollars. Diversify into equities, property etc.

I love how the comment I'm responding to literally says "then collect $250k per year risk free after taxes," and then you all pile onto me with downvotes telling me that's he's not just going to invest in treasuries (which is exactly the implication of HIS comment and not mine).

Re: Mira Murati leaves OpenAI

#580

Post by squarefoot »

blitzar wrote: post #439
squarefoot wrote: post #308

Earlier quoted context omitted.

Thanks, now I cannot unthink of this vision: developers activate the first ASI, and after 3 minutes it spits out full code and plans for a working Full Self Driving car prototype:)

I thought super-intelligence was to say self driving would be fully operational next year for 10 consecutive years?

My point was that only super intelligence could possibly solve a problem that we can only pretend to have solved.
Post reply on HN