Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Will I ever own a zettaflop?

geohot.github.io

Post reply on HN
Print view View original

91–98 of 98 posts

Re: Will I ever own a zettaflop?

#91

Post by mysecretaccount »

markisus wrote: post #86
mysecretaccount wrote: post #36

Fun post, but I find the industry's obsession with compute to be rather vapid, and this is a good example: > One million Claudes. To be able to search every book in history, solve math problems, write novels, read every comment, watch every reel, iterate over and over on a piece of code until it’s perfect – spend a human year in 10 minutes. 50,000 people working for you, all aligned with you, all answering as one. We…

Don’t get hung up on the Claude part. We already know an algorithm that produces AGI: evolution. We don’t have the compute to run this algorithm because it requires simulating the whole Earth. But with enough compute, it becomes possible.

The "Claude part" is important here, though. If you believe we can produce AGI by simulating physics then the existing paradigm is far too slow. A zettaflop does not even get close, you can have a moon-sized computer and it probably will not be able to produce AGI using the current transformer-based, extremely slow, classical computing architecture. We need to improve the underlying computation paradigm.

This is why the focus on raw compute is a poor use of engineering time. We have plenty of it, we are just headed in the wrong direction.

Re: Will I ever own a zettaflop?

#92

Post by mysecretaccount »

TheOtherHobbes wrote: post #68
emaadm wrote: post #42

Earlier quoted context omitted.

> We are already near the limits of what we can do Hard disagree. If I had a million Claudes worth of compute I'd be livestreaming my entire reality feed to a local server 24/7 and having it organize my observations and thoughts, synthesize new ideas, implement prototypes and discard infeasible ones while I sleep. If you're in the business of knowledge creation, a million Claudes isn't enough. Text is an easy modalit…

I call this the Laurie Anderson fallacy, from a line in one of her songs: > "Heaven is exactly like where you are right now, but much, much better." If a million Claudes of compute were accessible, people would not be doing the same things they are now, but more so. They'd be doing very different things we likely can't imagine - in the same way that Alan Turing imagined machines learning from experience, but didn't i…

> If a million Claudes of compute were accessible, people would not be doing the same things they are now, but more so. They'd be doing very different things we likely can't imagine

You may as well make the same argument about commanding a billion Nintendo 64s. We cannot simply scale up Claude instances like you say, its ability to produce interesting output is bounded by the underlying model.

Re: Will I ever own a zettaflop?

#93

Post by sfn42 »

TheOtherHobbes wrote: post #81
sfn42 wrote: post #73

Earlier quoted context omitted.

There's nothing (except their capacity and your token budget I guess) stopping anyone from having a million simultaneous conversations with Claude right now. Maybe a million is a stretch but thousands is completely doable right now. That's thousands of Claudes. Knock yourself out.

But that's the point. No human has the capacity to handle a million simultaneous conversations, any million-sized workflow would have to be AI-managed itself, and it's not even clear what the goals would be. If it ever becomes possible to say "build me a unicorn" you're going to get millions of people trying to do the same thing, and you no longer have the same economy. Because the features that generate unicorns sto…

That's my point. Person above is like "if I had a million Claudes I'd be Rick Sanchez", I'm saying they can do that right now so go ahead.

Re: Will I ever own a zettaflop?

#94

Post by Nevermark »

limbicsystem wrote: post #65
Nevermark wrote: post #58

Earlier quoted context omitted.

> In particular, it has been shown that 1 kilogram of matter confined to 1 liter of space can perform at most 10^51 operations per second on at most 10^31 bits of information I believe the Bekenstein bound for holographic information on a 1 liter sphere, using space at the Planck scale for encoding, instead of matter, is about 6.7×10^67. I confess I got that number by taking round trips through multiple models to ens…

I think you would very much enjoy this book: https://share.google/boWcVLRiYz0c7EmKh They talk quite a bit about this sort of thing at the end...

I went to Amazon where it informed me I have bought that book. But never read it! Found it. Thanks!

Re: Will I ever own a zettaflop?

#95

Post by noduerme »

ecshafer wrote: post #85
noduerme wrote: post #50

Earlier quoted context omitted.

For one thing, most news websites would have to load at least 10,000x as much useless javascript to achieve the same performance.

Soon every website will have an LLM entirely in JS that will create the React code for the rest of the website, build it, deploy it, then create the article on the fly from a prompt, and present it.

Oof. You may have just named the next frontier in our becoming unmoored from base reality. That's gonna be a gold rush. And I hate that this just gave me a brutally demented idea for a dating platform. I have a conscience, damnit.

Re: Will I ever own a zettaflop?

#96

Post by eru »

saltcured wrote: post #89
eru wrote: post #38

Earlier quoted context omitted.

Please keep in mind that doubling isn't the only option. There's lots of numbers between 64 and 128.

Well, not so many if you assume some access alignment requirement for high-performance hardware designs...

Maybe. But from what I remember, many ostensibly 64 bit x86 computers were actually 48 bits in terms of memory addressing for a long time?

Re: Will I ever own a zettaflop?

#97

Post by saltcured »

eru wrote: post #96
saltcured wrote: post #89

Earlier quoted context omitted.

Well, not so many if you assume some access alignment requirement for high-performance hardware designs...

Maybe. But from what I remember, many ostensibly 64 bit x86 computers were actually 48 bits in terms of memory addressing for a long time?

x86 is a funny example because it supported unaligned access more than many designs. But ignoring that...

Many CPUs, not just x86, have a "physical bits" length that is less than the address size in the ISA. This saves transistor and power budget, since address buses can be smaller. Of course, it means there is a lower maximum RAM config for that design.

The software would still shuffle around the full ISA word/double word/quad word or whatever. In a typical OS, the MMU and page mapping logic would potentially interpret all the bits to map to the more limited physical address range. It didn't mean storing smaller pointers in your software data structures etc.

I'm not an expert, but I think it varies by ISA whether it is defined how the higher address order bits, above the physical range, are handled. Some may allow applications to set them for address tagging, while the CPU ignores those bits. But, others may require them to all be zeroed to get predictable behavior.

Re: Will I ever own a zettaflop?

#98

Post by danpalmer »

diogocp wrote: post #76
danpalmer wrote: post #72

Earlier quoted context omitted.

That someone thinks they can personally "fix search" in a few months at a multi-billion dollar social network that just fired half its engineering staff, however, does.

What if someone thinks they can personally fix the loading time of a multi-billion dollar game, while having no access to its source code? https://news.ycombinator.com/item?id=26296339

The difference there being that they did.

The GTA fix shows humility, literally the first sentence of the "recon" section is "First I wanted to check if someone had already solved this problem". Geohot wasn't interested in if anyone had tried and failed to "solve search", or why it might be a difficult problem. He assumed that Twitter were a bunch of idiots.

The whole approach of the GTA fix author is curious and humble. Very low ego.

Post reply on HN