211–220 of 334 posts
I like to think history is like the greatest book ever written. When we are born we get to open it somewhere in the middle, spend a while figuring out what’s going on, then have to close it and never know the end. But vinge clearly peaked ahead a few chapters.
> then have to close it and never know the end
That bit has irked me to no end since I was young!
Earlier quoted context omitted.
IMHO Marooned in Realtime is the best Vinge book. Besides being a dual mystery novel, it really explores the implications of bobble technology and how just a few hours of technology development near the singularity can be extreme.
Yep. I like it better than Fire Upon the Deep but I do like both of them. I didn’t like A Deepness in the Sky as it was feeling kinda grindy like Dune . (I wish we could just erase Dune so people could enjoy all of Frank Herbert’s other novels of which I love even the bad ones)
Earlier quoted context omitted.
with respect, we don’t know if he was spot on. Companies shoehorning language models into their products is a far cry from the transformative societal change he describes will happen. nothing like a singularity has yet happened at the scale he describes, and might not happen without more fundamental shifts/breakthroughs in AI research.
> Within thirty years, we will have the technological means to create superhuman intelligence. Blackwell. > o Develop human/computer symbiosis in art: Combine the graphic generation capability of modern machines and the esthetic sensibility of humans. Of course, there has been an enormous amount of research in designing computer aids for artists, as labor saving tools. I'm suggesting that we explicitly aim for a grea…
> Blackwell.
I'm fucking sorry but there is no LLM or "AI" platform that is even real intelligence, today, easily demonstrated by the fact that an LLM cannot be used to create a better LLM. Go on, ask ChatGPT to output a novel model that performs better than any other model. Oh, it doesn't work? That's because IT'S NOT INTELLIGENT. And it's DEFINITELY not "superhuman intelligence." Not even close.
Sometimes accurately regurgitating facts is NOT intelligence. God it's so depressing to see commenters on this hell-site listing current-day tech as ANYTHING approaching AGI.
Earlier quoted context omitted.
Already considered in my comment. > Imagine that there's a little computer inside each neuron that decides when it needs to do work Yah, there's -bajillions- of floating point operation equivalents happening in a neuron deciding what to do. They're probably not all functional. BUT, that's why I said the "useful parts" of the decision: It may take more than the equivalent of one floating point operation to decide whet…
I see. Do you think this is what Kurzweil was accounting for when he multiplied by 1000 connections?
There's basically a few axes you can view this on:
- Number of connections and complexity of connection structure: how much information is encoded about how to do the calculations.
- Mutability of those connections: these things are growing and changing -while doing the math on whether to fire-.
- How much calculation is really needed to do the computation encoded in the connection structure.
Basically, brains are doing a whole lot of math and working on a dense structure of information, but not very precisely because they're made out of meat. There's almost certainly different tradeoffs in how you'd build the system based on the precision, speed, energy, and storage that you have to work with.
Earlier quoted context omitted.
They might generate improvements, but I’m not sure why people think those improvements would be unbounded. Think of it like improvements to jet engines or internal combustion engines - rapid improvements followed by decades of very tiny improvements. We’ve gone from 32-bit LLM weights down to 16, then 8, then 4 bit weights, and then a lot of messy diminishing returns below that. Moore’s is running on fumes for proces…
Bro, Jensen Huang just unveiled a chip yesterday that goes 20 petaflops. Intel's latest raptorlake cpu goes 800 gigaflops. Can you really explain 25000x progress by the 2x larger die size? I'm sure reactionary America wanted Moore's law to run out of steam but the Taiwanese betrayal made up for all the lost Moore's law progress and then some.
Earlier quoted context omitted.
> Within thirty years, we will have the technological means to create superhuman intelligence. Blackwell. > o Develop human/computer symbiosis in art: Combine the graphic generation capability of modern machines and the esthetic sensibility of humans. Of course, there has been an enormous amount of research in designing computer aids for artists, as labor saving tools. I'm suggesting that we explicitly aim for a grea…
>> > Within thirty years, we will have the technological means to create superhuman intelligence. > Blackwell. I'm fucking sorry but there is no LLM or "AI" platform that is even real intelligence, today, easily demonstrated by the fact that an LLM cannot be used to create a better LLM. Go on, ask ChatGPT to output a novel model that performs better than any other model. Oh, it doesn't work? That's because IT'S NOT I…
I don't know where that "computationally sufficient" line is. It'll always be fuzzy (because you could have a very slow, but smart entity). And before we have a working AGI, thinking about how much computation we need always comes down to back of the envelope estimations with radically different assumptions of how much computational work brains do.
But I can't rule out the idea that current architectures have enough processing to do it.
Earlier quoted context omitted.
VV is up there with Stephenson and Gibson as the top 3. I don't put Asimov, etc in there since Asimov was hard sci-fi to the max and couldn't write a character to save his life, much like later Stephenson. I wish I could find something else like VV's work that's sort of under-the-radar. I do have to mention that things like The Three Body Problem get hype, but are several tiers below VVs work.
Vinge is certainly one of the greats but so is David Brin. I would not consider him under the radar though. Some of his best are Earth, The Heart of The Comet, Glory Season.
Earlier quoted context omitted.
> GPT4 does inference at 560 teraflops. Human brain goes 10,000 teraflops AFAICT, both are guesses. The low-end estimate I've seen for human brains are ~ 162 GFLOPS[0] to 10^28 FLOPS[1]; even just the model size for GPT-4 isn't confirmed, merely a combination of human inference of public information with a rumour widely described as a "leak", likewise the compute requirements. [0] https://geohot.github.io//blog/jekyl…
They're not guesses. We know they use A100s and we know how fast an A100 goes. You can cut a brain open and see how many neurons it has and how often they fire. Kurzweil's 10 petaflops for the brain (100e9 neurons * 1000 connections * 200 calculations) is a bit high for me honestly. I don't think connections count as flops. If a neuron only fires 5-50 times a second then that'd put the human brain at .5 to 5 teraflop…
Firstly, Kurzweil underestimates the number connections by order of magnitude.
Secondly, dentritic computation changes things. Individual dentrites and the dendritic tree as a whole can do multiple individual computations. logical operations low-pass filtering, coincidence detection, ... One neuronal activation is potentially thousands of operations per neuron.
Single human neuron can be equivalent of thousands of ANN's.