Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Adolescence of Technology

darioamodei.com

Post reply on HN
Print view View original

41–50 of 159 posts

Re: The Adolescence of Technology

#41

Post by marcosdumay »

jonas21 wrote: post #23
root_axis wrote: post #13

I don't think we have much to worry about in terms of economic disruption. At this point it seems pretty clear that LLMs are having a major impact on how software is built, but for almost every other industry the practical effects are mostly incremental. Even in the software world, the effect of being able to build software a lot faster isn't really leading to a fundamentally different software landscape. Yes, you ca…

> At this point it seems pretty clear that LLMs are having a major impact on how software is built, but for almost every other industry the practical effects are mostly incremental. Even just a year ago, most people thought the practical effects in software engineering were incremental too. It took another generation of models and tooling to get to the point where it could start having a large impact. What makes you…

> most people thought the practical effects in software engineering were incremental too

Hum... Are you saying it's having clear positive (never mind "transformative") impact somewhere? Can you point any place we can see observable clear positive impact?

Re: The Adolescence of Technology

#42

Post by Balgair »

tadfisher wrote: post #17
Balgair wrote: post #2

Initial thought about 1/5th of the way through: Wow, that's a lot of em-dashes! i wonder how much of this he actually wrote? Edit: Okay, section 3 has some interesting bits in it. It reminds me of all those gun start-ups in Texas that use gyros and image recognition to turn a C- shooter into an A- shooter. They all typically get bought up quite fast by the government and the tech shushed away. But the ideas are just…

Dario and Anthropic's strategy has been to exaggerate the harmful capabilities of LLMs and systems driven by LLMs, positioning Anthropic themselves as the "safest" option. Take from this what you will. As an ordinary human with no investment in the game, I would not expect LLMs to magically work around the well-known physical phenomena that make submarines hard to track. I think there could be some ability to augment…

> Dario and Anthropic's strategy has been to exaggerate the harmful capabilities of LLMs and systems driven by LLMs, positioning Anthropic themselves as the "safest" option. Take from this what you will.

Yeah, I've been feeling that as well. It's not a bad strategy at all, makes sense, good for business.

But on the nuclear issue, it's not a good sign that he's explicitly saying that this AGI future is a threat to nuclear deterrence and the triad. Like, where do you go up from there? That's the highest level of alarm that any government can have. This isn't a boy crying wolf, it's the loudest klaxon you can possibly make.

If this is a way to scare up dollars (like any tyre commercial), then he's out of ceiling now. And that's a sign that it really is sigmoiding internally.

Re: The Adolescence of Technology

#43

Post by nemomarx »

Lerc wrote: post #12

One of my formative impressions of AI came from the depiction of the Colligatarch from Alan Dean Foster's The I Inside. The AI in the book is almost feels like it is the main message masquerading as a subplot. Asimov knew the risks, and I had assumed until fairly recently that the lessons and explorations that he had imparted into the Robot books had provided a level of cultural knowledge of what we were about to fac…

I think people broadly feel like all of this is happening inevitably or being done by others. The alignment people struggle to get their version of AI to market first - the techies worry about being left behind. No one ends up being in a position to steer things or have any influence over the future in the race to keep up.

So what can you and I do? I know in my gut that imagining an ideal outcome won't change what actually happens, and neither will criticizing it really.

Re: The Adolescence of Technology

#45

Post by johnfn »

firasd wrote: post #36
johnfn wrote: post #34

Earlier quoted context omitted.

I find one-off anecdotal examples like this to be a bit like discourse around global warming - "Look at that ridiculous polar vortex we had this week! Global warming can't possibly be a thing!" Of course, a trend line comprises many points, and not every point falls perfectly in the center of the line! I'm not necessarily saying you are right or wrong, but your argument should address the line (and ideally give some…

Ah but I'm not arguing about the rate of change in the trend. I'm saying the signals are decoupled. That is to say an LLM can be as good as a programmer as Linus Torvalds without having even basic knowledge-generalization abilities we assume the median human with no specialized skills would have (when given the same knowledge an LLM has)

I think most LLM proponents would say that "basic knowledge-generalization abilities" is on a different, slower trend line.

I mean, you aren't very surprised that your CPU can crush humans at chess but can barely run an image classifier, right? But you probably wouldn't say (as you are saying with LLMs) that ability for a CPU to play chess is "decoupled" from classifying images. Increases in CPU speed improve both. You'd just say that one is a lot harder than the other.

Re: The Adolescence of Technology

#46

Post by BoiledCabbage »

firasd wrote: post #27
jonas21 wrote: post #22

Earlier quoted context omitted.

I have no idea what you are even asking Claude to do here.

I was asking it to see if the wikisource tools are working by looking up a Bible quote. There was no ambiguity about the task itself; what I'm saying is that Claude 'knows' a bunch of things (the Bible has different translations) that it doesn't operationalize when doing a task--issues that would would be glaringly obvious to a human who knows the same things

Maybe I'm missing the point as well, but what did it do wrong?

It seemed like you wanted to see if a search tool was working.

It looked to see. It tried one search using on data source KJ and found no matches. Next question would be is the quote not in there, is there a mis-remembering of the quote or is their something wrong with the data source. It tries an easier to match quote and finds nothing, which it finds odd. So next step in debugging is assume a hypotheses of KJ Bible datasource is broken, corrupted or incomplete (or not working for some other reason). So it searches for an easier quote using a different datasource.

It's unclear the next bit because it looks like you may have interrupted it, but it seems like it found the passage about Mary in the DR data source. So using elimination, it now knows the tool works (it can find things), the DR data source works (it can also find things), so back to the last question of eliminating hypotheses: is the quote wrong foe the KJ datasource, or is that datasource broken.

The next (and maybe last query I would do, and what it chose) was search for something guaranteed to be there in KJ version: the phrase 'Mary'. Then scan through the results to find the quote you want, then re-query using the exact quote you know is there. You get 3 options.

If it can't find "Mary" at all in KJ dataset then datasource is likely broken. If it finds mary, but results don't contain the phrase, then the datasource is incomplete. If it contains the phrase then search for it, if it doesn't find it then you've narrowed down the issue "phase based search seems to fail". If it does find and, and it's the exact quote it searched for originally then you know search has an intermittent bug.

This seemed like perfect debugging to me - am I missing something here?

And it even summarized at the end how it could've debugged this process faster. Don't waste a few queries up front trying to pin down the exact quote. Search for "Mary" get a quote that is in there, then search for that quote.

This seems perfectly on target. It's possible I'm missing something though. What were you looking for it to do?

Re: The Adolescence of Technology

#47

Post by psunavy03 »

Lerc wrote: post #12

One of my formative impressions of AI came from the depiction of the Colligatarch from Alan Dean Foster's The I Inside. The AI in the book is almost feels like it is the main message masquerading as a subplot. Asimov knew the risks, and I had assumed until fairly recently that the lessons and explorations that he had imparted into the Robot books had provided a level of cultural knowledge of what we were about to fac…

People can't even have a conversation about any kind of societal issues these days without pointing at the other political tribe and casting aspersions about "what they really meant" instead of engaging with what's actually being said.

Forgetting that if you really can hear a dogwhistle, you're also a dog.

Re: The Adolescence of Technology

#49

Post by atq2119 »

nemomarx wrote: post #43
Lerc wrote: post #12

One of my formative impressions of AI came from the depiction of the Colligatarch from Alan Dean Foster's The I Inside. The AI in the book is almost feels like it is the main message masquerading as a subplot. Asimov knew the risks, and I had assumed until fairly recently that the lessons and explorations that he had imparted into the Robot books had provided a level of cultural knowledge of what we were about to fac…

I think people broadly feel like all of this is happening inevitably or being done by others. The alignment people struggle to get their version of AI to market first - the techies worry about being left behind. No one ends up being in a position to steer things or have any influence over the future in the race to keep up. So what can you and I do? I know in my gut that imagining an ideal outcome won't change what ac…

In the large, ideas can have a massive influence on what happens. This inevitability that you're expressing is itself one of those ideas.

Shifts of dominant ideas can only come about through discussions. And sure, individuals can't control what happens. That's unrealistic in a world of billions. But each of us is invariably putting a little but of pressure in some direction. Ironically, you are doing that with your comment even while expressing the supposed futility of it. And overall, all these little pressures do add up.

Re: The Adolescence of Technology

#50

Post by torginus »

Philpax wrote: post #11
BinaryIgor wrote: post #9

Earlier quoted context omitted.

What makes you think that they will just keep improving? It's not obvious at all, we might soon hit a ceiling, if we haven not already - time will tell. There are lots of technologies that have been 99% done for decades; it might be the same here.

From the essay - not presented in agreement (I'm still undecided), but Dario's opinion is probably the most relevant here: > My co-founders at Anthropic and I were among the first to document and track the “scaling laws” of AI systems—the observation that as we add more compute and training tasks, AI systems get predictably better at essentially every cognitive skill we are able to measure. Every few months, public s…

I think the reference to scaling is a pretty big giveaway that things are not as they seem - I think it's pretty clear that we've run out of (human produced) data, so there's nowhere to scale to in that dimension. I'm pretty sure modern models are trained in some novel ways that engineers have to come up with.

It's quite likely they train on CC output too.

Yeah, there's synthethic data as well, but how do you generate said data is very likely a good question and one that many people have lost a lot of sleep over.

Post reply on HN