Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You are the scariest monster in the woods

jamie.ideasasylum.com

Post reply on HN
Print view View original

111–120 of 405 posts

Re: You are the scariest monster in the woods

#111

Post by jncfhnb »

lambaro wrote: post #101
jncfhnb wrote: post #40

Earlier quoted context omitted.

I think you could make AGI right now tbh. It’s not a function of intelligence. It’s just a function of stateful system mechanics. LLMs are just a big matrix. But what about a four line of code loop that looks like: ```while true: update_sensory_inputs() narrate_response() update_emotional_state() ``` LLMs don’t experience continuous time and they don’t have an explicit decision making framework for having any agency…

"STEP 2: Draw the rest of the owl."

I disagree.

Personally I found the definition of a game engine as

``` while True: update_state() draw_frame()```

To be a profound concept. The implementation details are significant. But establishing the framework behind what we’re actually talking about is very important.

Re: You are the scariest monster in the woods

#112

Post by simianparrot »

squidbeak wrote: post #79
kulahan wrote: post #31

Earlier quoted context omitted.

We still haven’t figured out what intelligence even is. Depending on what you care about, the second-smartest animal in the world varies wildly.

This is a bogus argument. There's a lot we don't understand about LLMs, yet we built them.

We built something we don’t understand by trial and error. Evolution took a few billion years getting to intelligence, so I guess we’re a few sprints away at least

Re: You are the scariest monster in the woods

#113

Post by nyeah »

myrmidon wrote: post #85
deadbabe wrote: post #36

Earlier quoted context omitted.

Language is only an end product. It is derived from intelligence. The intelligence is everything that created the language and the training corpus in the first place. When AI is able to create entire thoughts and ideas without any concept of language, then we will truly be closer to artificial intelligence. When we get to this point, we then use language as a way to let the AI communicate its thoughts naturally. Such…

I'm not saying that language necessarily is the biggest stumbling block on (our) road towards AI, but it is a very prominent feature that we have used to distinguish our capabilities from other animals long before AI was even conceived of. So the current successes with LLMs are highly encouraging. I'm not buying the "current AI is just a dumb parrot relying on human training" argument, because the same thing applies…

"I'm not buying the "current AI is just a dumb parrot relying on human training" argument [...]"

One difficulty. We know that argument is literally true.

"[...] because the same thing applies to humans themselves"

It doesn't. People can interact with the actual world. The equivalent of being passively trained on a body of text may be part of what goes into us. But it's not the only ingredient.

Re: You are the scariest monster in the woods

#114

Post by ACCount37 »

lo_zamoyski wrote: post #93
ACCount37 wrote: post #74

Earlier quoted context omitted.

I don't get how you can see one of those CLI coding tools in action and still parrot the "no agency" line. The goal-oriented behavior is rather obvious. Sure, they aren't very good at agentic behavior yet, and the time horizon is pretty low. But that keeps improving with each frontier release.

> The goal-oriented behavior is rather obvious. Obvious? Is an illusion obviously the real thing? There is nothing substantially different in LLMs from any other run of the mill algorithm or software.

In the same way there's "nothing substantially different" in humans from any other run of the mill matter.

I find that all this talk of "illusion" is nothing but anthropocentric cope. Humans want to be those special little snowflakes, so when an LLM does X, there are crowds of humans itching to scream "it's not REAL X".

Re: You are the scariest monster in the woods

#115

Post by ffsm8 »

jncfhnb wrote: post #99
Jensson wrote: post #68

Earlier quoted context omitted.

> ```while true: update_sensory_inputs() narrate_response() update_emotional_state() ``` You don't think that has already been made?

Sure, probably, to varying levels of implementation details.

It has been implemented for sure, just watch a little Neurosama.

That's most definitely not AGI

Re: You are the scariest monster in the woods

#116

Post by bee_rider »

myrmidon wrote: post #13

I struggle to understand how people can just dismiss the possibility of artificial intelligence. Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I'm not buying the whole "AI has no agency" line eith…

I think it is bad writing on the part of the author. Or maybe good writing for getting us engaged with the blog post, bad for making an argument though.

They include a line that they don’t believe in the possibility of AGI:

> I don’t really believe in the threat of AGI (Artificial General Intelligence—human-level intelligence) partly because I don’t believe in the possibility of AGI and I’m highly skeptical that the current technology underpinning LLMs will provide a route to it.

This is a basically absurd position to hold, I mean humans physically exist so our brains must be possible to build within the existing laws of physics. It is obviously far beyond our capabilities to replicate a human brain (except via the traditional approach), but unless brains hold irreproducible magic spirits (well we must at least admit the possibility) they should be possible to build artificially. Fortunately they immediately throw that all away anyway.

Next, they get to the:

> and I’m highly skeptical that the current technology underpinning LLMs will provide a route to it.

Which is, of course, at least a plausible thing to believe. I mean there are a bunch of philosophical questions about what “intelligence” even means so there’s plenty of room to quibble here. Then we have,

> But I also think there’s something we should actually be afraid of long before AGI, if it ever comes. […]

> Now, if you equip humans with a hammer, or sword, or rifle, or AI then you’ve just made the scariest monster in the woods (that’s you) even more terrifying. […]

> We don’t need to worry about AI itself, we need to be concerned about what “humans + AI” will do.

Which is like, yeah, this is a massively worrying problem that doesn’t involve any sci-fi bullshit, and I think it is what most(?) anybody who’s thought about this seriously at all (or even stupid people who haven’t, like myself). Artificial Sub-intelligences, things that are just smart enough to make trouble and too dumb or too “aligned” to their owner (instead of society in general) to push back are a big currently happening problem.

Re: You are the scariest monster in the woods

#117

Post by Chance-Device »

omnicognate wrote: post #88
boole1854 wrote: post #22

Earlier quoted context omitted.

If anyone knows of a steelman version of the "AGI is not possible" argument, I would be curious to read it. I also have trouble understanding what goes into that point of view.

If you genuinely want the strongest statement of it, read The Emperor's New Mind followed by Shadows of the Mind, both by Roger Penrose. These books often get shallowly dismissed in terms that imply he made some elementary error in his reasoning, but that's not the case. The dispute is more about the assumptions on which his argument rests, which go beyond mathematical axioms and include statements about the nature o…

To be honest, the core of Penrose’s idea is pretty stupid. That we can understand mathematics despite incompleteness theorem being a thing, therefore our brains use quantum effects allowing us to understand it. Instead of just saying, you know, we use a heuristic instead and just guess that it’s true. I’m pretty sure a classical system can do that.

Re: You are the scariest monster in the woods

#118

Post by snarf21 »

irjustin wrote: post #58
roxolotl wrote: post #27

Earlier quoted context omitted.

The point of the article isn’t that abstract super intelligent agi isn’t scary. Yes the author says that’s unlikely but that paragraph at the start is a distraction. The point of the article is that humans wielding LLMs today are the scary monsters.

But that's always been the case? Since we basically discovered... Fire? Tools?

The difference in my mind is scale and reach and time. Fire, tools, war are localized. AGI could have global and instant and complete control.

Re: You are the scariest monster in the woods

#119

Post by rootusrootus »

seniortaco wrote: post #48
jvanderbot wrote: post #11

About as helpful as "Guns don't kill people ... " And equally rebutted by Eddie Izzards "Well, I think the gun helps".

My thought as well. Nuclear weapons are also horrifying. And with LLMs, it's difficult to prevent the proliferation to bad actors. It seems like we're racing towards a world of fakery where nothing can be believed, even when wielded by good actors. I really hope LLMs can actually add value at a significant level.

> It seems like we're racing towards a world of fakery where nothing can be believed

Spend a couple minutes on social media and it is clear we are already there. The fakes are getting better, and even real videos are routinely called out as fake.

The best that I can hope for is that we all gain a healthy dose of skepticism and appreciate that everything we see could be fake. I don't love the idea of having to distrust everything I see, but at this point it seems like the least bad option.

But I worry that what we will experience will actually be somewhat worse. A sufficiently large number of people, even knowing about AI fakery, will still uncritically believe what they read and see.

Maybe I am being too cynical this morning. But it is hard to look at the state of our society today and not feel a little bleak.

Re: You are the scariest monster in the woods

#120

Post by jjkaczor »

nonethewiser wrote: post #105
jncfhnb wrote: post #40

Earlier quoted context omitted.

I think you could make AGI right now tbh. It’s not a function of intelligence. It’s just a function of stateful system mechanics. LLMs are just a big matrix. But what about a four line of code loop that looks like: ```while true: update_sensory_inputs() narrate_response() update_emotional_state() ``` LLMs don’t experience continuous time and they don’t have an explicit decision making framework for having any agency…

Where is the "what the thing cares about" part? When I look at that loop my thought is, "OK, the sensory inputs have updated. There are changes. Which ones matter?" The most naive response I could imagine would be like a git diff of sensory inputs. "item 13 in vector A changed from 0.2 to 0.211" etc. Otherwise you have to give it something to care about, or some sophisticated system to develop things to care about. E…

... well, humans are not always known for making correct, logical or sensical decisions when they update their input loops either...
Post reply on HN