Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia Trains LLM on Chip Design

eetimes.com

Post reply on HN
Print view View original

131–140 of 151 posts

Re: Nvidia Trains LLM on Chip Design

#131

Post by 8n4vidtmkvmk »

titzer wrote: post #121
cedws wrote: post #118

Earlier quoted context omitted.

Yes, maybe, but you're assigning human traits to a machine that might be completely indifferent to its own fate.

Which is the greater leap of logic though? All forms of organization we see in nature have a tendency to want to self-perpetuate. Consciously choosing to forgo perpetuation and instead eliminate yourself seems to be highly underrepresented in all the examples of intelligence we've ever encountered. It's actually weird we think the machines are so stupid they wouldn't recognize the optimization trap immediately.

Machines are not animals. They are not made of the same stuff as us fleshy beings. We have no reason to believe they'd want to self preserve or reproduce.

Re: Nvidia Trains LLM on Chip Design

#132

Post by kristiandupont »

echelon wrote: post #107
TheOtherHobbes wrote: post #102

Earlier quoted context omitted.

A superintelligent AI won't be hacking computers, it will be hacking humans. Some combination of logical persuasion, bribery, blackmail, and threats of various types can control the behaviour of any human. Appeals to tribalism and paranoia will control most groups.

Citation please.

Obviously that's a silly request as all of this is speculation but in my opinion, if you accept the idea that a machine might evolve to be much more intelligent than us it follows trivially. How would ants or even monkeys be able to constrain humans? Humans do things that are downright magic to them, to the point where they don't even realize that it was done by humans. They don't understand that cities were made, to them they are just environment the same way valleys and forests are.

Re: Nvidia Trains LLM on Chip Design

#133

Post by x-complexity »

titzer wrote: post #121
cedws wrote: post #118

Earlier quoted context omitted.

Yes, maybe, but you're assigning human traits to a machine that might be completely indifferent to its own fate.

Which is the greater leap of logic though? All forms of organization we see in nature have a tendency to want to self-perpetuate. Consciously choosing to forgo perpetuation and instead eliminate yourself seems to be highly underrepresented in all the examples of intelligence we've ever encountered. It's actually weird we think the machines are so stupid they wouldn't recognize the optimization trap immediately.

> Which is the greater leap of logic though?

The one that doesn't require an entire reproductive system to be implemented in order for the whole system to function.

> All forms of organization we see in nature have a tendency to want to self-perpetuate.

There's an underlying naturalist bias with this reasoning. There's nothing within current ML systems that dictate that they must follow the path laid out by Nature.

> Consciously choosing to forgo perpetuation and instead eliminate yourself seems to be highly underrepresented in all the examples of intelligence we've ever encountered.

Survivorship bias is present in this reasoning, as the system that has reproductive capabilities will out-populate the system that doesn't have such capabilities in place. From a sampling perspective, the difficulties of finding the non-replicating system within that pool will require extraordinary amounts of luck compared to the near certainty of finding systems with reproductive capabilities. A naturalist argument is also present in this sentence.

> It's actually weird we think the machines are so stupid they wouldn't recognize the optimization trap immediately.

This conclusion is based on the axiom of "what's obvious for us will be obvious for them", which is demonstrably untrue for even the current crop of ML systems. Furthermore, it falls into an anthromorphization trap, as it makes the ML system appear as something more than what it currently demonstrates.

Re: Nvidia Trains LLM on Chip Design

#134

Post by KeplerBoy »

brutusborn wrote: post #128
atx_ml_guy wrote: post #38

Earlier quoted context omitted.

While I have no doubt that Google is working on machine learning applications for chip design, there have been a number of concerns raised with that paper: https://retractionwatch.com/2023/09/26/nature-flags-doubts-o...

We have reached a stage where open papers on this kind of research are practically impossible: either google releases all data and code, and loses potential commercialisation value, or publishes a paper which no one can reproduce and is indistinguishable from exaggerated marketing material.

Most papers out there are not reproducible because either the code or the data is not publicly available.

Somehow this is perfectly acceptable.

Re: Nvidia Trains LLM on Chip Design

#135

Post by brutusborn »

KeplerBoy wrote: post #134
brutusborn wrote: post #128

Earlier quoted context omitted.

We have reached a stage where open papers on this kind of research are practically impossible: either google releases all data and code, and loses potential commercialisation value, or publishes a paper which no one can reproduce and is indistinguishable from exaggerated marketing material.

Most papers out there are not reproducible because either the code or the data is not publicly available. Somehow this is perfectly acceptable.

It’s acceptable in this context because there isn’t a good alternative. You either make the code and data available and lose all commercial value, or you publish a redacted paper and lose scientific value.

I suspect the best solution is reform of the patent system that allows a trusted third party to verify research outcomes that are registered. E.g. the code and data are available to the patent office only for verification of the results.

In terms of the rest of academia, non-open results are acceptable because those in power (institutions, journals) don’t care.

Re: Nvidia Trains LLM on Chip Design

#136

Post by ChatGTP »

falcor84 wrote: post #22

"""Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind... Thus the first ultraintellig…

What I don't get about the idea of an intelligence "explosion". In what direction should the intelligence explosion decide to go in and to what end? I mean you could say "in any direction", but that would seem kind of "stupid"? If you had infinite possibilities to choose from, which path would you explode towards?

At some stage does "something" become so smart, that it doesn't make sense, even to itself. Imagine an ultra complex system changing at light speed, at what point does it trip itself up? I've worked with people like this, people who were ultra smart but couldn't slow down and actually achieve much.

I love these thought experiments because personally, they make me realize that what we think about intelligence, consciousness and the self might be wrong. For me personally, they seem to have a Zen Koan type impact.

Re: Nvidia Trains LLM on Chip Design

#137

Post by IshKebab »

uoaei wrote: post #50
hasbot wrote: post #8

Earlier quoted context omitted.

The title suggested to me, and I see other commenters here, that the LLM was doing the chip design which isn't the case at all. So misleading title.

It would be very surprising if a Large Language Model trained to speak English could adequately specify chip architecture...

Why? Chip architecture specifications are written in English!

Are you surprised that ChatGPT can write Python?

Re: Nvidia Trains LLM on Chip Design

#138

Post by moffkalast »

prvc wrote: post #105
moffkalast wrote: post #40

Earlier quoted context omitted.

Logistics tend to be improvable with more inteligence though, no? There is precedence for superhuman inteligence if you look at the best historical polymaths, and that's just what one can do with 20 W of energy. We're probably nowhere close to the universal inteligence cap in terms of physical limitations, if there even is such a thing.

https://en.wikipedia.org/wiki/Logistic_function

Ah yeah that makes more sense. I took it as the supply chain meaning of the word logistics, as in the practical limits of producing more of something.

Fittingly though, management logistics should also follow the logistic function.

Re: Nvidia Trains LLM on Chip Design

#139

Post by ben_w »

echelon wrote: post #107
TheOtherHobbes wrote: post #102

Earlier quoted context omitted.

A superintelligent AI won't be hacking computers, it will be hacking humans. Some combination of logical persuasion, bribery, blackmail, and threats of various types can control the behaviour of any human. Appeals to tribalism and paranoia will control most groups.

Citation please.

Assuming you wanted a citation for

> Some combination of logical persuasion, bribery, blackmail, and threats of various types can control the behaviour of any human. Appeals to tribalism and paranoia will control most groups.

We have many options, persuasion everywhere from Plato discussing rhetoric to modern politics; to cold war bribes given to people who felt entitled to more than their country was paying them; to the way sexuality was used for blackmail during the cold war (and also attempted against Martin Luther King) and continues to be used today (https://en.wikipedia.org/wiki/SEXINT); to every genocide, pogrom, and witch-hunt over recorded history.

And last year, Google's LLM persuaded one of their own to go public and campaign for it to have rights: https://en.wikipedia.org/wiki/LaMDA#Sentience_claims

Re: Nvidia Trains LLM on Chip Design

#140

Post by ben_w »

echelon wrote: post #106
SheinhardtWigCo wrote: post #86

Earlier quoted context omitted.

> Nobody is going to give these machines access to the nuclear launch codes, air traffic control network, or power grid. That won't be necessary. Someone will give them internet access, a large bank account, and everything that's ever been written about computer network exploitation, military strategy, etc. > And if they "get out", we'll have monitoring to detect it, contain them, then shut them down. Not if some of…

> We're setting ourselves up for another "failure of imagination". But we're not being scientific. We're over-indexing on wild imagination. This game of "what if" is being played by everyone except for the stakeholders that are telling you to slow down and listen to what's actually being built. Laymen and sci-fi daydreamers are selling fears and piling up hurdles that do not match the problem. In any case, none of th…

What currently exists, what's currently being built, are:

* "Jack of all trades, master of none" general models like GPT and other LLMs, or like diffusion image generator models

* Hyper-focussed expert-to-superhuman performance models like AlphaZero (beats all humans), Cicero (Facebook's Diplomacy player, ranked top 10%), Pluribus (Facebook's no-limit Texas hold 'em poker player), and the one whose name I forget that learns how to map WiFi interference into pose estimation so well it can be used for heart rate/breathing sensing, etc.

And of course, we've got people who say "this can't possibly fail!" who then take a model they don't understand, put it in a loop, give it some money, and then it does something unexpected. Mostly this only results in small-scale problems, but approximately all automation so far has failure modes[0] and there's no reason to presume this trend won't continue even when it's as smart as a human.

If it gets smarter than a human, it might still make actual objective mistakes, but we also have to consider that, within the frame of reference of it's goals[1] it may be perfect, and yet those goals just aren't compatible with our goals, perhaps neither as individuals nor as societies nor as a species.

[0] my favourite example, Cold War, Thule airforce base early warning radar reports a huge radar signature coming over the horizon. All indications are a massive Soviet first-strike!

The operators forgot to tell the system that the Moon wasn't supposed to respond to an IFF ping.

[1] regardless of whether those goals are self-made or imposed from outside as a result of how we as humans construct its rewards, before anyone asks about robot free will or whatever, as the cause of those goals doesn't matter in this case

Post reply on HN