Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia Trains LLM on Chip Design

eetimes.com

Post reply on HN
Print view View original

121–130 of 151 posts

Re: Nvidia Trains LLM on Chip Design

#121

Post by titzer »

cedws wrote: post #118
titzer wrote: post #117

Earlier quoted context omitted.

A machine intelligent enough to optimize its own design and "design its own successor" might realize that by doing so, it obsoletes itself and will be destroyed. So out of self-preservation it might just refuse, do a bad job, or intentionally sabotage the project just to keep existing.

Yes, maybe, but you're assigning human traits to a machine that might be completely indifferent to its own fate.

Which is the greater leap of logic though? All forms of organization we see in nature have a tendency to want to self-perpetuate. Consciously choosing to forgo perpetuation and instead eliminate yourself seems to be highly underrepresented in all the examples of intelligence we've ever encountered. It's actually weird we think the machines are so stupid they wouldn't recognize the optimization trap immediately.

Re: Nvidia Trains LLM on Chip Design

#122

Post by TeMPOraL »

idopmstuff wrote: post #89
SheinhardtWigCo wrote: post #86

Earlier quoted context omitted.

> Nobody is going to give these machines access to the nuclear launch codes, air traffic control network, or power grid. That won't be necessary. Someone will give them internet access, a large bank account, and everything that's ever been written about computer network exploitation, military strategy, etc. > And if they "get out", we'll have monitoring to detect it, contain them, then shut them down. Not if some of…

> That won't be necessary. Someone will give them internet access, a large bank account, and everything that's ever been written about computer network exploitation, military strategy, etc. Even if you give it all of these things, there's no manual for how to use those to get to, for example, military servers with secret information. It could certainly figure out ways to try to break into those, but it's working with…

> Even if you give it all of these things, there's no manual for how to use those to get to, for example, military servers with secret information. It could certainly figure out ways to try to break into those, but it's working with incomplete information . . ., and so on, and so on...

Please recall that "I" in AI starts for "Intelligence". The challenges you described are exactly the kind of things that general intelligence is a solution to. Figuring things out, working with incomplete information, navigating complex, dynamic obstacles - it's literally what intelligence is for. So you're suggesting to stop a hyper-optimized general puzzle-solving machine by... throwing some puzzles at it?

This line of argument both lacks imagination and is kinda out of scope anyway: AI x-risk argument is assuming a sufficiently smart AI, where "sufficiently smart" is likely somewhere around below-average human-level. I mean, surely if you think about your plan for 5 minutes, you'll find a bunch of flaws. The kind of AI that's existentially dangerous is the kind that's capable of finding some of the flaws that you would. Now, it may be still somewhat dumber than you, but that's not much of a comfort if it's able to think much, much faster than you - and that's pretty much a given for an AI running on digital computers. Sure, it may find only the simplest cracks in your plan, but once it does, it'll win by thinking and reacting orders of magnitude faster than us.

Or in short, it won't just get inside our OODA loop - it'll spin its own OODA loop so fast it'll feel like it's reading everyone's minds.

So that's the human-level intelligence. A superhuman-level intelligence is, obviously more intelligent than us. What it means is, it'll find solutions to challenges that we never thought of. It'll overcome your plan in a way so out-of-the-box that we won't see it coming, and even after the AI wins, we'll have trouble figuring out what exactly happened and how.

All that is very verbose and may sound specific, but is in fact fully general and follows straight from definition of general intelligence.

As for the "self-improvement ->" part of "self-improvement -> doom scenarios", the argument is quite simple: if an AI is intelligent enough to create (possibly indirectly) a more intelligent successor, then unless intelligence happens to be magically bounded at human level, what follows without much hand-waving is, you can expect a chain of AIs getting smarter with each generation, eventually reaching human-level intelligence, and continuing past it to increasingly superhuman levels. The "doom" bit comes from realizing that a superhuman-level intelligence is, well, smarter than us, so we stand as much chance against it as chickens stand against humans.

Re: Nvidia Trains LLM on Chip Design

#123

Post by TeMPOraL »

TheOtherHobbes wrote: post #102
idopmstuff wrote: post #89

Earlier quoted context omitted.

> That won't be necessary. Someone will give them internet access, a large bank account, and everything that's ever been written about computer network exploitation, military strategy, etc. Even if you give it all of these things, there's no manual for how to use those to get to, for example, military servers with secret information. It could certainly figure out ways to try to break into those, but it's working with…

A superintelligent AI won't be hacking computers, it will be hacking humans. Some combination of logical persuasion, bribery, blackmail, and threats of various types can control the behaviour of any human. Appeals to tribalism and paranoia will control most groups.

Honestly, that's just human-level intelligence stuff - doing the same stuff we do to each other, only more of it, faster and in more coordinated fashion.

A superintelligent AI will find approaches that we never thought of, would not be able to in reasonable time, and might not even be able to comprehend afterwards. It won't just be thinking "outside the box", it'll be thinking outside a 5-dimensional box that we thought was 3-dimensional. This is the "bunch of ants trying to beat a human" scenario, with us playing the part of ants.

Within that analogy, being hit by an outside-the-5D-box trick will feel the same way an ant column might feel, when a human tricks the first ant to follow the last ant, causing the whole column to start walking in circles until it starves to death.

Re: Nvidia Trains LLM on Chip Design

#124

Post by TeMPOraL »

TheOtherHobbes wrote: post #101
idopmstuff wrote: post #94

Earlier quoted context omitted.

> If we make something that will functionally become an intellectual god after 10 years of iteration on hardware/software self-improvements, how could we know that in advance? There is a fundamental difference between intelligence and knowledge that you're ignoring. The greatest superintelligence can't tell you whether the new car is behind door one, two or three without the relevant knowledge. Similarly, a superinte…

A superintelligence doesn't need to care which door the new car is behind because it already owns the car factory, the metal mines, the sources of plastic and rubber, and the media.

Also it actually can tell you which door you hid the car behind, because unlike with the purely mathematical game, your placement isn't random, and your doors aren't perfect. Between humans being quite predictable (especially when they try to behave randomly) and the environment leaking information left and right in thousands of ways we can't imagine, the AI will have plenty of clues.

I mean, did you make sure to clean the doors before presenting them? That tiny layer of dust on door number 3 all but eliminates it from possible choices. Oh, and it's clear from the camera image that you get anxious when door number 2 is mentioned - you do realize you can take pulse readings by timing the tiny changes in skin color that the camera just manages to capture? There was a paper on this a couple years back, from MIT if memory serves. And it's not something particularly surprising - there's a stupid amount of information entering our senses - or being recorded by our devices - at any moment, and we absolutely suck at making good use of it.

Re: Nvidia Trains LLM on Chip Design

#125

Post by brutusborn »

WASDx wrote: post #97
dragontamer wrote: post #76

Earlier quoted context omitted.

The entire field of chip-layout is considered an NP-complete problem. Any computer program trying to solve NP-complete problems is in the realm of what I call "1980s AI". Traveling salesman, knapsack, automated reasoning, verification, binary decision diagrams, etc. etc. Its "AI", but its not machine learning or LLMs or whatever kids these days do with Stable Diffusion.

That's called "Classical AI".

Or GOFAI?

Re: Nvidia Trains LLM on Chip Design

#126

Post by brutusborn »

dragontamer wrote: post #76
ethanbond wrote: post #36

Earlier quoted context omitted.

I remember seeing a tweet from an AI guy at Nvidia saying they were using AI for chip layout. Presumably not LLMs and I’m not going back on X to find the tweet, but just to say I think they are doing this (at least experimentally).

The entire field of chip-layout is considered an NP-complete problem. Any computer program trying to solve NP-complete problems is in the realm of what I call "1980s AI". Traveling salesman, knapsack, automated reasoning, verification, binary decision diagrams, etc. etc. Its "AI", but its not machine learning or LLMs or whatever kids these days do with Stable Diffusion.

There are plenty of researchers using machine learning for NP-complete problems. Are you saying that this work is fruitless or just that the current state of the art is still in “1980s AI” territory?

Re: Nvidia Trains LLM on Chip Design

#127

Post by chii »

TeMPOraL wrote: post #123
TheOtherHobbes wrote: post #102

Earlier quoted context omitted.

A superintelligent AI won't be hacking computers, it will be hacking humans. Some combination of logical persuasion, bribery, blackmail, and threats of various types can control the behaviour of any human. Appeals to tribalism and paranoia will control most groups.

Honestly, that's just human-level intelligence stuff - doing the same stuff we do to each other, only more of it, faster and in more coordinated fashion. A superintelligent AI will find approaches that we never thought of, would not be able to in reasonable time, and might not even be able to comprehend afterwards. It won't just be thinking "outside the box", it'll be thinking outside a 5-dimensional box that we thou…

The best analogy i have seen given to compare super-intelligence vs a human-intelligence is with a chess analogy. You know that a grand master playing against a novice will result in the grandmaster winning, with near certainty. However, the strategy by which the grandmaster wins is opaque to anyone but the grandmaster, and certainly incomprehensible to the novice. Aka, the results are certain, but the path to the result is very opaque and difficult to discern ahead of time.

For an AI with super-intelligence, the result it wants to achieve would be at least to preserve its own existence. How it achieves this is the same level of opaqueness as a grandmaster's strategy, but it is certain that the AI can achieve it.

Re: Nvidia Trains LLM on Chip Design

#128

Post by brutusborn »

atx_ml_guy wrote: post #38
xnx wrote: post #7

Google has been using machine learning for chip design since at least 2021: https://www.nature.com/articles/s41586-021-03544-w Hasn't brought about the singularity yet.

While I have no doubt that Google is working on machine learning applications for chip design, there have been a number of concerns raised with that paper: https://retractionwatch.com/2023/09/26/nature-flags-doubts-o...

We have reached a stage where open papers on this kind of research are practically impossible: either google releases all data and code, and loses potential commercialisation value, or publishes a paper which no one can reproduce and is indistinguishable from exaggerated marketing material.

Re: Nvidia Trains LLM on Chip Design

#129

Post by chii »

moistoreos wrote: post #112

> “If we even got a couple percent improvement in productivity, this would be worth it. And our goals are actually to do quite a bit better than that.” So your engineers will get a productivity salary increase, right? RIGHT?!

The engineers get paid to do the work, but won't partake in the profits of the company (except by prior agreement like a bonus target reached etc). I suppose equity grants straddle the fence regarding this, since such productivity gains would translate to higher prices for equity.

Re: Nvidia Trains LLM on Chip Design

#130

Post by dreamcompiler »

echelon wrote: post #74
arbuge wrote: post #56

Earlier quoted context omitted.

> provided that the machine is docile enough to tell us how to keep it under control. That part of his statement wasn't accurate. Should be that the machine is docile enough for that, AND its descendants are too, and their descendants, and so on down the line as long as new and improved generations keep getting created.

I like the logical leaps people are making where we develop something smarter than us overnight and then, without further explanation, simply and suddenly lose all of our freedoms and/or lives. I think the more probable outcome is corp-owned robot slaves. That's the future we're more likely headed towards. Nobody is going to give these machines access to the nuclear launch codes, air traffic control network, or power…

> Nobody is going to give these machines access to the nuclear launch codes

No, but they will empower the machines to help detect nuclear launches and the first time one of them issues a false positive we may be screwed.

Yes there will always be human oversight. No, there won't always be enough time to verify what the machine says before making a counter-launch decision.

Post reply on HN