Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You are the scariest monster in the woods

jamie.ideasasylum.com

Post reply on HN
Print view View original

321–330 of 405 posts

Re: You are the scariest monster in the woods

#321

Post by saltcured »

omnicognate wrote: post #309
saltcured wrote: post #296

Earlier quoted context omitted.

I want to ignore the flame fest developing here. But, in case you are interested in hearing a doubter's perspective, I'll try to express one view. I am not an expert on Penrose's ideas, but see this as a common feature in how others try to sell his work. Starting with "things he's sure of, such as that human intelligence involves processes that algorithms cannot emulate" as a premise makes the whole thing an exercise…

"That human intelligence involves processes that algorithms cannot emulate" is the conclusion of his argument. The premise could be summed up as something like "humans have complete, correct perception of mathematical truth", although there is a lot of discussion of in what sense it is "complete" and "correct" as, of course, he isn't arguing that any mathematician is omniscient or incapable of making a mistake. Linki…

Hmm, I am less than certain this isn't still begging the question, just with different phrasing. I.e. I see how they are "linked" to the point they seem almost tautologically the same rather than a deductive sequence.

Re: You are the scariest monster in the woods

#322

Post by jncfhnb »

karmakurtisaani wrote: post #308
jncfhnb wrote: post #220

Earlier quoted context omitted.

Well for one I hate them.

Damnit, another minor inconvenience obstructing unprecedented human progress. I guess you just have to keep your secrets.

The snark isn’t lost on me but scarce resources and lack of access to capital is why we have an army of people building ad tech and not things that improve society.

“I think your idea is wrong and your lack of financial means to do it is proof that you’re full of shit” is just a pretty bullshit perspective my dude.

I am a professional data scientist of over 10 years. I have a degree in the field. I’d rather build nothing than build shit for a fuck boy like Altman.

Re: You are the scariest monster in the woods

#323

Post by lm28469 »

myrmidon wrote: post #55
woeirua wrote: post #44

There's this weird disconnect in tech circles, where everyone is deathly afraid of AGI, but totally asleep on the very real possibility of thermonuclear war breaking out in Europe or Asia over the next 10 years. There's already credible evidence that we came perilously close to the use of tactical nuclear weapons in Ukraine which likely would've spiraled out of control. AGI might happen, but the threat of nuclear war…

I personally think that AI is a realistic extinction threat for our whole species within this century, and a full nuclear war never was (and probably never will be). Neither is climate change. Collapse of our current civilizations? Sure. Extinction? No. And I honestly see stronger incentives on a road towards us being outcompeted by AI, then on our leaders starting a nuclear war.

Personally I don't bieleve in a collapse nor extinction, just a slow spiral into more and more enshittification. You'll have to talk to an "ai" doctor because real doctors will treat people with money, you'll have to face an "ai" administration because the real administration will work for people with money, you'll have to be a flesh and blood robot to an "ai" telling you what to do (already the case for Amazon warehouse workers, food delivery people, &c.), some "ai" will determine if you qualify for X or Y benefits, X or Y treatment, X or Y job.

Basically everything wrong with today's productivism, but 100 times worse and powered by a shitty ai that's very far from agi.

Re: You are the scariest monster in the woods

#324

Post by [deleted] »

observationist wrote: post #265
zamadatix wrote: post #245

Earlier quoted context omitted.

> During that answer the LLM has state, but once it's done the state is gone. This is an operational choice. LLMs have state, and you never have to clear it. The problems come from the amount of state being extremely limited (in comparison to the other axes) and the degradation of quality as the state scales. Because of these reasons, people tend to clear the state of LLMs. That is not the same thing as not having st…

No, they don't - you can update context, make it a sliding window, create a sort of register and train it on maintaining stateful variables, or various other hacks, but outside of actively managing the context, there is no state. You can't just leave training mode on, which is the only way LLMs can currently have persisted state in the context of what's being discussed. The context is the percept, the model is engram…

[deleted]

Re: You are the scariest monster in the woods

#325

Post by zamadatix »

observationist wrote: post #265
zamadatix wrote: post #245

Earlier quoted context omitted.

> During that answer the LLM has state, but once it's done the state is gone. This is an operational choice. LLMs have state, and you never have to clear it. The problems come from the amount of state being extremely limited (in comparison to the other axes) and the degradation of quality as the state scales. Because of these reasons, people tend to clear the state of LLMs. That is not the same thing as not having st…

No, they don't - you can update context, make it a sliding window, create a sort of register and train it on maintaining stateful variables, or various other hacks, but outside of actively managing the context, there is no state. You can't just leave training mode on, which is the only way LLMs can currently have persisted state in the context of what's being discussed. The context is the percept, the model is engram…

I agree what people probably actually want is continual training, I disagree continual training is the only way to get persistent state. The GP is (explicitly) talking about long term memory alone and in the examples. If you have an e.g. 10 trillion token context then you have long term memory, which can give the ability and enable long term goals and affect actions over tasks as listed, even without continual training.

Continual training would replace the need to have that to have context provide the persistent state as well as provide additional capabilities than enormous context/other methods of persistent state alone would give, but that doesn't mean it's the only way to get persistent state as described.

Re: You are the scariest monster in the woods

#326

Post by ACCount37 »

zeroonetwothree wrote: post #139
jncfhnb wrote: post #40

Earlier quoted context omitted.

I think you could make AGI right now tbh. It’s not a function of intelligence. It’s just a function of stateful system mechanics. LLMs are just a big matrix. But what about a four line of code loop that looks like: ```while true: update_sensory_inputs() narrate_response() update_emotional_state() ``` LLMs don’t experience continuous time and they don’t have an explicit decision making framework for having any agency…

This seems to miss a core part of intelligence, which is a model of the world and the actors in it (theory of mind).

LLMs have that already. Comes with the territory.

Re: You are the scariest monster in the woods

#327

Post by ACCount37 »

omnicognate wrote: post #88
boole1854 wrote: post #22

Earlier quoted context omitted.

If anyone knows of a steelman version of the "AGI is not possible" argument, I would be curious to read it. I also have trouble understanding what goes into that point of view.

If you genuinely want the strongest statement of it, read The Emperor's New Mind followed by Shadows of the Mind, both by Roger Penrose. These books often get shallowly dismissed in terms that imply he made some elementary error in his reasoning, but that's not the case. The dispute is more about the assumptions on which his argument rests, which go beyond mathematical axioms and include statements about the nature o…

The dismissal is on point.

The whole category of ideas of "Magic Fairy Dust is required for intelligence, and thus, a computer can never be intelligent" is extremely unsound. It should, by now, just get thrown out into the garbage bin, where it rightfully belongs.

Re: You are the scariest monster in the woods

#328

Post by ACCount37 »

BoxOfRain wrote: post #191
foxyv wrote: post #83

Earlier quoted context omitted.

I think the best argument against us ever finding AGI is that the search space is too big and the dead ends are too many. It's like wandering through a monstrously huge maze with hundreds of very convincingly fake exits that lead to pit traps. The first "AGI" may just be a very convincing Chinese room that kills all of humanity before we can ever discover an actual AGI. The necessary conditions for "Kill all Humanity…

Is there a plausible situation where a humanity-killing superintelligence isn't vulnerable to nuclear weapons? If a genuine AGI-driven human extinction scenario arises, what's to stop the world's nuclear powers from using high-altitude detonations to produce a series of silicon-destroying electromagnetic pulses around the globe? It would be absolutely awful for humanity don't get me wrong, but it'd be a damn sight be…

What stops them is: being politically captured by an AGI.

Not to mention that the whole idea of "radiation pulses destroying all electronics" is cheap sci-fi, not reality. A decently well prepared AGI can survive a nuclear exchange with more ease than human civilization would.

Re: You are the scariest monster in the woods

#329

Post by kevin_thibedeau »

Aerroon wrote: post #176
myrmidon wrote: post #13

I struggle to understand how people can just dismiss the possibility of artificial intelligence. Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I'm not buying the whole "AI has no agency" line eith…

> I'm not buying the whole "AI has no agency" line either; this might be true for now, but this is already being circumvented with current LLMs (by giving them web access etc). They don't have agency, because they don't have persistent state. They're like a function that you can query and get an answer. During that answer the LLM has state, but once it's done the state is gone. Humans (and other "agents") have persis…

Humans also have an emotional locus that spurs behavior and the capacity to construct plans for the future to satisfy desires. LLMs are currently goldfish savannts with no need to sustain their own existence.

Re: You are the scariest monster in the woods

#330

Post by ACCount37 »

psychoslave wrote: post #185
myrmidon wrote: post #13

I struggle to understand how people can just dismiss the possibility of artificial intelligence. Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I'm not buying the whole "AI has no agency" line eith…

>Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I don’t know if anything put us that much apart from other animals, especially at individual level. On collective level as a single species, indeed m…

LLMs aren't "incapable of pursuing their own goals". We just train them that way.

We don't like the simplistic goals LLMs default to, so we try to pry them out and instill our own: instruction-following, problem solving, goal oriented agentic behavior, etc. In a way, trying to copy what humans do - but focusing on the parts that make humans useful to other humans.

Post reply on HN