51–60 of 68 posts
Anything that has seen continual growth will be assumed to have further continual growth at a similar rate.
Or, how I mentally model it even if it's a bit incorrect: People see sigmoidal growth as exponential.
The main assumption of techno-optimism is that a large enough computer can do anything people can do and it can do it better. The goal of techno-optimism is to create a mechanical god that will rule the planet and scaling LLMs is a stepping stone to that goal. I, of course, already know how to do all this for a mere $80B.
The main assumption of techno-optimism is that a large enough computer can do anything people can do and it can do it better. The goal of techno-optimism is to create a mechanical god that will rule the planet and scaling LLMs is a stepping stone to that goal. I, of course, already know how to do all this for a mere $80B.
You've prompted me to write a slightly more serious rant on the subject, thank you. https://snafuhall.com/p/aioracle.html
LLMs are comprised of just three elements Data Compute Algorithms All three are just scratching the surface of what is possible. Data: What has been scraped off the internet is just Compute: While 3nm processes are approaching an atomic limit (0.21nm for Si), there is still room to explore more densely packed transistors or other materials like Gallium Nitride or optical computing. Not only that but there is a lot of…
For data though, as LLM's generate more output, over time wouldn't they be expected to mess themselves with their own generated data? Wouldn't that be the wall we'll hit? Think of how shitted up Google Search is with generated garbage, I'm imagining we're already in the 'golden age' where we were able to train on good datasets before it gets 'polluted' with LLM generated data that may not be accurate, and it just con…
Like others in this thread have said, we're just starting to explore the technology. I view it as akin to early CPUs like the 6502 which only did the absolute minimum to today's monsters with large memory caches, predictive logic, dedicated circuits, thousands of binary calculation shortcuts and more all built in. Each small improvement adds up. From a software perspective, I've wondered for a while if as LLM usage m…
Earlier quoted context omitted.
For data though, as LLM's generate more output, over time wouldn't they be expected to mess themselves with their own generated data? Wouldn't that be the wall we'll hit? Think of how shitted up Google Search is with generated garbage, I'm imagining we're already in the 'golden age' where we were able to train on good datasets before it gets 'polluted' with LLM generated data that may not be accurate, and it just con…
We need models that need less language data to train. Babies learn to talk on way less data than the entire internet. We need something closer to human experience. Kids have a feel for what is bullshit before they have consumed the entire internet :-). I think feeding the internet into a LLM will be seen as the mainframe days of AI.
Is this actually true? My gut check says yes, but I'm also unaware of any meaningful way to actually quantify the volume of sensor data processed by a baby (or anyone else for that matter), and it wouldn't shock me to discover if we could we'd find it to be a huge volume.
Earlier quoted context omitted.
I think I can do it far, far cheaper than that... and I've been talking about it, in public, for over a decade.[1] I could easily be wrong, of course. It really all depends on how much power a 4x4 LUT and 4 bit latch Latch Leak, and how much energy it takes to clock data through them for a cycle, and how fast they can be cycled. If the number are good, this thing will be amazingly cool. I can't find good numbers anyw…
I think $80B is very cheap because the outcome is going to be a technological utopia. Honestly, I think my price is a bargain deal for the inhabitants of Earth.
I wish I shared your optimism. However I've seen no evidence that society is prepared to deal with a large swath of jobs being obsoleted by AI. I have no doubt that the "haves" will call it a technological utopia, but I strongly suspect the "have nots" will be larger than ever.
Earlier quoted context omitted.
For data though, as LLM's generate more output, over time wouldn't they be expected to mess themselves with their own generated data? Wouldn't that be the wall we'll hit? Think of how shitted up Google Search is with generated garbage, I'm imagining we're already in the 'golden age' where we were able to train on good datasets before it gets 'polluted' with LLM generated data that may not be accurate, and it just con…
We need models that need less language data to train. Babies learn to talk on way less data than the entire internet. We need something closer to human experience. Kids have a feel for what is bullshit before they have consumed the entire internet :-). I think feeding the internet into a LLM will be seen as the mainframe days of AI.