911–920 of 1001 posts
Earlier quoted context omitted.
> LLMs seem to udnerstand language therefore they've trained a model of the world. This isn’t the claim, obviously. LLMs seem to understand a lot more than just language. If you’ve worked with one for hundreds of hours actually exercising frontier capabilities I don’t see how you could think otherwise.
> This isn’t the claim, obviously. This is precisely the claim that leads a of lot people to believe that all you need to reach AGI is more compute.
Andrej Karpathy seems to me like a national (world) treasure. He has the ability to explain concepts and thoughts with analogies and generalizations and interesting sayings that allow you to keep interest in what he is talking about for literally hours - in a subject that I don't know that much about. Clearly he is very smart, as is the interviewer, but he is also a fantastic communicator and does not come across as…
Earlier quoted context omitted.
> LLMs seem to udnerstand language therefore they've trained a model of the world. This isn’t the claim, obviously. LLMs seem to understand a lot more than just language. If you’ve worked with one for hundreds of hours actually exercising frontier capabilities I don’t see how you could think otherwise.
> This isn’t the claim, obviously. This is precisely the claim that leads a of lot people to believe that all you need to reach AGI is more compute.
Dwarkesh is AGI-pilled and would base his assumption of a world model on much more impressive feats than mere language understanding.
Earlier quoted context omitted.
I agree with this. A metaphor I like is that the reason why humans say the night sky is beautiful is because they see that it is, whereas an LLM says it because it’s been said enough times in its training data.
To play devil’s advocate, you have never seen the night sky. Photoreceptors in your eye have been excited in the presence of photons. Those photoreceptors have relayed this information across a nerve to neurons in your brain which receive this encoded information and splay it out to an array of other neurons. Each cell in this chain can rightfully claim to be a living organism in and of itself. “You” haven’t directly…
Of course, computers can be fitted with optical sensors, but our cognitive equipment has been carved over millions of years by these kind of interactions, so our familiarity with the phenomenon of light goes way deeper than that, shaping the very structure of our thought. Large language models can only mimic that, but they will only ever have a second-hand understanding of these things.
This is a different issue than the question of whether AI's are conscious or not.
Earlier quoted context omitted.
This is the key insight I believe. It is inherently unpredictable. There are species that pass the mirror test with a far fewer equivalent number of parameters than large models are using already. Carmack has said something to the effect that about 10ksloc would glue the right existing achictectures together in the right way to make agi, but that it might take decades to stumble on that way, or someone might find it…
> Carmack has said something to the effect that about 10ksloc would glue the right existing achictectures together in the right way to make agi What does he know about that?
Earlier quoted context omitted.
The iPhone came out less than 20 years ago, and now I: • Don't get out my debit card while shopping. • Don't get lost exploring a new city. • Have zero-cost video calls with anyone I want. • Use most spare moments of my time — walking to the shops, or on public transport, or while hiking in the countryside — learning something new. When I'm not too damp for the capacitive touch screen, that can be interactive lessons…
• Don't get out my debit card while shopping. You take out your phone though. How is taking your phone out of your pocket, logging in, and tapping it on a terminal significantly different from pulling a credit card or cash from your pocket and tapping the terminal or handing it to the checker? • Don't get lost exploring a new city. You're young, I guess. We had GPS in cars well before iPhone. GPS navigation in cars w…
Biometric ID to make the payment. I don't so much "log in" as "touch the fingerprint scanner built into the button that switches the screen on". Though if I cared to wear it, I do also have an Apple Watch and would therefore not even need to take anything out of my pocket.
> You're young, I guess. We had GPS in cars well before iPhone. GPS navigation in cars was taking off mid-90s to mid-2000s. I had a Garmin in 2002.
Just about to turn 42. I saw GPS in use only a little later than that, 2005 I think. But:
1) dedicated GPS was never in everyone's pocket until smartphones became normalised; and even then, location precision was mediocre until assisted GPS got phased in (IIRC the first consumer phone with A-GPS was about a year before the iPhone?)
2) the maps were incredibly bad; my experience in 2005 included it thinking we were doing 70 miles an hour through a field because the main road we were on was newer than the device's map.
3) Phone map apps also include traffic alerts, public transport info including live updates for delays, altitude data (useful for cyclists), ratings and hours for seemingly most of the cafes/restaurants/other attractions, and simply has a lot more detail because it can afford to (e.g. many of the public toilets).
> I was doing that on my laptop and desktop before iPhone. Heck, I was doing free video conferencing with European friends in 1995.
Critical point: "with anyone I want". Almost every independently functioning person in Europe, has a smartphone, and can be contacted without waiting for them to sit down at a desk terminal connected to a fixed line internet connection that was currently switched on.
Back in 1995, most people didn't have the internet at all, so no possibility at all to call them over the internet; those who did have it were either academics (yay JANET), had a relatively expensive wired ISDN line, or were on dialup (charged by the minute and had just about enough bandwidth for 3fps greyscale at 160x120 or so if the compression was what I think it was), and while mobile phones did exist back then, they were (1) unaffordable unless you were a yuppie, (2) didn't have cameras, (3) even worse bandwidth than dialup because 2G.
> This is an improvement over pocket electronic translators I was using in Japan in the early 2000s, but really the improvements are mostly in fidelity and usability, not in function.
I count "point camera at poster, see poster modified with translations overlaid over all text" as very much a change of function.
I mean, I don't need to translate Chinese, Japanese, Korean, or Arabic, but sometimes they come up in films and I get curious, but I can't type any of those alphabets in the first place so the only way to translate it is with something like Google Translate (and its predecessor Word Lens) that does it all as a video stream.
> focusing on when these tasks became easy and in everyone's hands, rather than when the tasks actually became possible and were in reasonably widespread use.
For much of this, that's the point. As the quote goes, "The future's already here, it's just not evenly distributed". I assumed it would be clear video calls can only be had with other people that also have video call equipment.
Or forward looking, look at how there are cars with no-steering-wheel-needed (even if Waymo has not actually removed them) full-self-drive, but they're geofenced. It's there, it's not everywhere.
With AI and human labour? Well, that's a two-part thing, the hardware and the software.
Hardware? I can buy a humanoid robot right now — it would be a bit silly, but I could, e.g.: https://de.aliexpress.com/item/1005009127396247.html
Software? The software running these robots can (just about) fold laundry, or tidy up litter and dishes — you know, all the things that people keep sarcastically listing to dismiss AI, saying "wake me up when they can XYZ": https://www.youtube.com/@figureai/videos
It's just… these robots are expensive, kinda slow, and the software gives me the same vibes I got from AI Dungeon (I think I saw it shortly after they changed away from GPT-2?), so I ask the same question of those today as I asked myself of a 3D printer in 2015, of an iPhone in 2010, of a multi-language electronic travel dictionary in 2009, of a dedicated GPS unit in 2005, of a laptop in 2002: can I really justify spending that much money on this thing? And my answer is the same: no.
I can't run the fanciest AI models on any of my devices, they won't fit, I'd have to buy a much beefier machine. There's a whole bunch of things that the SOTA AI models themselves can't do yet, but which can be done by tools that AI do know how to use, but I can't run all of those tools either. Any tool that gets invented in the next 20 years (or indeed ever), if it's documented at all in any language current LLMs can follow, those LLMs will be able to use them.
Now don't get me wrong, I'm not holding my breath or saying this will be soon. I've opined before that the minimum gap between "a level-5 self driving car" and "a humanoid robot that can get into any old car and drive it equally well" is 5-10 years just because of the smaller form factor having less room for compute and battery. Also, it seems obvious that "all human labour" is a harder problem than "can drive". If (if!) it is necessary to have humanoid robots in order to render all human labor obsolete, then I would be surprised if it takes any less than 15 years from today, but could be more — easily more, and by an arbitrarily large degree. I don't think humanoid robots are necessary for this, which reduces my lower bound, but at the same time it is just a lower bound.
Earlier quoted context omitted.
>An exception is a signal of uncertainty indicating that you need to learn more about your problem No, that would be a warning. Ab exceprion is a signal something failed and it was impossible to continue
Many exceptions are recoverable. This sometimes depends on the context, and on how well polished the software is
The execution couldn't continue in one path due to an error it needed to be caught in another path.
The difference with standard conditional mechanisms like if loops is mostly semantical. Exceptions are unforeseen errors, (technically they are sets of errors, which can have size 1, but the syntax is designed for catching groups of errors, if you want to react to a single error case you could also just use a condition with a return value and it ceases being an exception. )
Earlier quoted context omitted.
something that replaces humans doesn’t need to be 99.9999% reliable, it just has to be better than the humans it replaces.
But to be accepted by people, it has to be better than humans in the specific ways that humans are good at things . And less bad than humans in the ways that they're bad at things. When automated solutions fail in strange alien ways, it understandably freaks people out. Nobody wants to worry about if a car will suddenly serve into oncoming traffic because of a sensor malfunction. Comparing incidents-per-miles-driven…
Earlier quoted context omitted.
> Your brain is doing computation with neurotransmitters instead of transistors If it is, sure. But this isn't a given. We don't actually understand how the brain computes, as evidenced by our inability to simulate it. > Evolution didn't discover some mystical process that imbues meat with special properties Sure. But the complexity remains beyond our comprehension. Against the (nearly) binary action potential of a t…
Jonas & Kording showed that neuroscience methods couldn't reverse-engineer a simple 6502 processor [0]. If the tools can't crack a system we built and fully documented, our inability to simulate brains just means we're ignorant, not that substrate is magic. It also doesn't necessarily say great things for neuroscience! And "who said this?"... come on. Searle, Dreyfus, thirty years of "syntax isn't semantics," all the…
So, a decent approximation that only turned out to be wrong when we looked closely and found the mass flow was in the opposite direction, but otherwise the model basically worked?
That would be fantastic!
Earlier quoted context omitted.
The hubris here isn't CS people making comparisons, it's assuming biological substrate matters. Your brain is doing computation with neurotransmitters instead of transistors. So what? The "chemicals not electricity" distinction is pure carbon chauvinism, like insisting hydraulic computers can't be compared to electronic ones because water isn't electricity. Evolution didn't discover some mystical process that imbues…
> Your brain is doing computation with neurotransmitters instead of transistors If it is, sure. But this isn't a given. We don't actually understand how the brain computes, as evidenced by our inability to simulate it. > Evolution didn't discover some mystical process that imbues meat with special properties Sure. But the complexity remains beyond our comprehension. Against the (nearly) binary action potential of a t…
I'm sure I've seen people say this about language translation and playing go. Ditto chess, way back before Kasparov lost. I don't think I've seen anyone so specific as to say that about medical licensing exams, nor as vague as "write code", but on the latter point I do even now see people saying that software engineering is safe forever with various arguments given…