Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There is no hard takeoff

geohot.github.io

Post reply on HN
Print view View original

151–160 of 190 posts

Re: There is no hard takeoff

#151

Post by FeepingCreature »

anileated wrote: post #123
FeepingCreature wrote: post #58

Earlier quoted context omitted.

We know exactly what it does. However, how it actually works we haven't the faintest.

If you really think so, I wouldn’t sleep at night if I were you. Thankfully, we actually understand fairly well how it works (cf. the “we’ve built it” part). That we may not know exactly how any particular output is produced is what sometimes happens when you build things that behave non-deterministically. Perhaps you give RNGs and bugs an undeserved aura of mystery.

We understand how Transformers "learn", ie. how the mechanism of its action operates. However, except for the most basic of cases, we don't understand how Transformers operate the skills they've acquired and can demonstrate, at all. See the field of interpretability for early attempts to change this.

For example, if you train a large network on lots of languages, and then fine-tune it to listen to instructions in English, it will, on its own, also listen to instructions in every other language it was trained on. Nobody knows why this is the case.

Re: There is no hard takeoff

#152

Post by KnobbleMcKnees »

fxtentacle wrote: post #139
KnobbleMcKnees wrote: post #77

Earlier quoted context omitted.

My problem with this isn't even that it's a bunch of strung together ideas without a clear narrative or conclusion; enough practice at this combined with actual deep field knowledge results in works like Godel, Escher, Bach . My problem is that GHotz is a shitty writer, has puddle deep knowledge of anything that isn't iPhone firmware, and watching him play at being a freelance tech journo and produce this kind of hol…

I believe you're being unnecessarily mean here. He managed to write enough of a low level GPU driver to compile and execute neural network kernels. Calling that "puddle deep knowledge" is pretty dismissive considering that CUDA is Nvidia's most profitable moat.

The implication was that his technical knowledge is very good and his writing is not, but you missed that while you were getting angry.

Re: There is no hard takeoff

#153

Post by KnobbleMcKnees »

greatpostman wrote: post #129
KnobbleMcKnees wrote: post #77

Earlier quoted context omitted.

My problem with this isn't even that it's a bunch of strung together ideas without a clear narrative or conclusion; enough practice at this combined with actual deep field knowledge results in works like Godel, Escher, Bach . My problem is that GHotz is a shitty writer, has puddle deep knowledge of anything that isn't iPhone firmware, and watching him play at being a freelance tech journo and produce this kind of hol…

Ask yourself why you’re so repulsed by him, it’s probably not due to his writing skill

I called his writing shitty. I like George and have used a lot the tools he's written over the years. I've made similar tools myself. Are you so offended that I have a single critical opinion of him? Because that's a reflection of you, not I.

Re: There is no hard takeoff

#155

Post by pusspuss »

Universe’s Jive"

(Complex instrumental sections with shifts in time signatures. Lyrics have a satirical undertone.)

Verse 1: Summoned a shadow or just a silicon brain? From chessboards to streets, it's all part of the game. Think we're the champs, but are we just inane? Compared to the cosmos, are our brains too lame?

Interlude: (Jazzy guitar riff mixed with an odd-metered percussion line.)

Chorus: The singularity’s knocking, or is it just hype? Caught in the loop, the stereotypical type. Predicting, speculating, swallowing the tripe, But can we decode the universe's archetype?

Verse 2: Markets and dreams, the billionaire’s dance, All looks rosy, till you're out of chance. Machines boast of a masterful glance, Yet, who pulls the strings in this vast expanse?

Bridge: MuZero's groove, feeling so elite, Thinks it’s got the rhythm, can't accept defeat. But against the universe, can it compete? Or just another tune, incomplete, obsolete?

Chorus: Singularity, they say, is the ultimate dive, But can we, mere mortals, really derive? Predictions abound, but can we survive? In the end, it’s about keeping the jive alive.

Verse 3: Computations, simulations, all in a grid, In this tech circus, we're just a tiny squid. Chasing echoes, in shadows we're hid, But the cosmic joke? It's just a quid.

Outro: (A whimsical woodwind section, possibly a kazoo solo.)

In this cosmic game, we strive and strive, Dancing on the edge, trying to thrive. But remember, as the stars contrive, It's not about the end, but the drive.

Re: There is no hard takeoff

#156

Post by cloverich »

nwoli wrote: post #47
reedf1 wrote: post #32

He makes a good point. The only real way to "summon the demon" is if AGI is centralized. If each of us has a maximally intelligent AGI in our phone - and it is us as individuals controlling the prompts, then the playing field is levelled. Unstoppable force meets unstoppable force. Gun meets gun. A tale as old as society.

We also already have laws against evil behaviour, no need to introduce AI specific regulation. People can poison the water supply, or make bio weapons already. Yet people don’t because there’s laws and the same will stop people from using AI for evil.

Poisoning the water supply / bio weapons are bits of knowledge most people don't typically have. I think a better analogy might be using a gun to shoot up a school. We have laws against it, but guns are so widely available and easy to obtain, that the tiny pool of sociopaths willing are enabled. Will AI be in the former or latter camp?

Re: There is no hard takeoff

#157

Post by cloverich »

jagged-chisel wrote: post #65
reedf1 wrote: post #32

He makes a good point. The only real way to "summon the demon" is if AGI is centralized. If each of us has a maximally intelligent AGI in our phone - and it is us as individuals controlling the prompts, then the playing field is levelled. Unstoppable force meets unstoppable force. Gun meets gun. A tale as old as society.

Tangent: We have some scifi about networked AI. If we have maximally intelligent AGI in our phones, we should probably keep them all firewalled.

Another tangent: This assumes the AGI on our phones are all discrete entities, rather than a singular entity. We know what it means for a human to die, and hence for independent agents to exist, compete, cooperate. What does it mean for an AGI to die? Can it?

Re: There is no hard takeoff

#158

Post by fisf »

KnobbleMcKnees wrote: post #152
fxtentacle wrote: post #139

Earlier quoted context omitted.

I believe you're being unnecessarily mean here. He managed to write enough of a low level GPU driver to compile and execute neural network kernels. Calling that "puddle deep knowledge" is pretty dismissive considering that CUDA is Nvidia's most profitable moat.

The implication was that his technical knowledge is very good and his writing is not, but you missed that while you were getting angry.

That's not really what "puddle deep knowledge of anything that isn't iPhone firmware" implies.

Re: There is no hard takeoff

#159

Post by leshow »

isodnhionio wrote: post #133
zenkat wrote: post #71

"And yet, here we are in 2023 and self driving cars still don’t work." LOL. Someone should tell the California CPUC that: https://www.theverge.com/2023/8/10/23827790/waymo-cruise-cpu...

This is yet another reason I want Musk in prison. Getting drivers off the road will save lives, and not a small number. Most companies investing in automated driving have built a system that is safe and largely effective. These systems have worked safely in the places they've been deployed. We should be heavily investing in expanding their capabilities. Yet Tesla's system is the most famous and most widely-deployed a…

It seems to me if you wanted to deliver on "economic benefits", save lives, reduce emissions, etc there is an obvious already existing option: subways/LRT. For inter-city travel build high speed rail. I just don't see driverless cars as the panacea that other people do, even if it worked— which it doesn't seem to.

Re: There is no hard takeoff

#160

Post by drawkbox »

andrewjl wrote: post #146
drawkbox wrote: post #124

Earlier quoted context omitted.

I think the point he is making is that competition will prevent others from being able to FOOM and 51% everyone else and "win" outright to the point that it is dangerous. Or competition will prevent one player from getting the "powder keg". Competition can do that if it is fair competition and a good game design. That is why rules and regulations are important in any game or it becomes Calvinball where the only rule…

> The flaw in the thinking is that there will be "fair" competition. Why does what OP concludes require fairness? Wouldn't the same logic apply if every player defects, in game theory terms?

OP doesn't really go into it other than competition will be a check on FOOM. Which I think is valid as in history any market with competition creates better products and keeps players in check.

I was arguing that competition if it is a more open market with fair rules/regulations, where competition is fair, that is true. Though many times it is a fixed market, or a cheat that stifles competition that might keep it in check.

Fixed markets happen more and more where the concentration is high and efficient players will game the system.

No products or markets are better where big fish solely control everything. That is why anti-trust or regulation need to expand to funding level not just surface company level. If you own entire industries across many companies, that is still oligarchy/monopoly if it is controlled by the same funding/sources.

Concentration needs to be broken up, for competition and better market and quality of life for everyone. Most people definitely don't want to make the authoritarian systems wealthier than open markets.

Concentration starts to take us away from a fair market and more towards a fixed/gamed market.

Everything in that gets so tuned that competition is very hard to enter. Very little margin and too much optimization/efficiency is bad for resilience. Couple that with private equity backed near leverage monopolies that control necessary supply and you have trouble.

HBS is even realizing too much optimization/efficiency is a bad thing. The slack/margin is squeezing out an ability to change vectors quickly.

The High Price of Efficiency, Our Obsession with Efficiency Is Destroying Our Resilience [1]

> Superefficient businesses create the potential for social disorder.

> A superefficient dominant model elevates the risk of catastrophic failure.

> *If a system is highly efficient, odds are that efficient players will game it.*

> sometimes power becomes so concentrated that political action is needed to loosen the stranglehold of the dominant players, as in the antitrust movement of the 1890s.

Couple massive wealth and concentration, even leaning authoritarian by then as tends to happen with fixed/controlled markets, with AI/AGI and you no longer have a "blue shell" because those players control the rules and thus the game.

[1] https://hbr.org/2019/01/the-high-price-of-efficiency

Post reply on HN