Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You are the scariest monster in the woods

jamie.ideasasylum.com

Post reply on HN
Print view View original

271–280 of 405 posts

Re: You are the scariest monster in the woods

#271

Post by Aerroon »

throwuxiytayq wrote: post #209
Aerroon wrote: post #176

Earlier quoted context omitted.

> I'm not buying the whole "AI has no agency" line either; this might be true for now, but this is already being circumvented with current LLMs (by giving them web access etc). They don't have agency, because they don't have persistent state. They're like a function that you can query and get an answer. During that answer the LLM has state, but once it's done the state is gone. Humans (and other "agents") have persis…

I have no words to express how profoundly disappointed I am to keep reading these boring, shallow, shorttermist, unimaginative takes that are invalidated by a model/arch upgrade next week, or - in this case - more like years ago, since pretty much all big LLM platforms are already augmented by RAG and memory systems. Do you seriously think you’re discussing a serious long term limitation here?

The reason I brought this up is that we clearly can have AI without the kind of agency people are scared of. You don't need to make your robots into sci Fi style AI and feel sorry for them.

Re: You are the scariest monster in the woods

#272

Post by rurp »

Flamingoat wrote: post #59
us-merul wrote: post #5

"Humans will do what they’ve always tried to do—gain power, enslave, kill, control, exploit, cheat, or just be lazy and avoid the hard work—but now with new abilities that we couldn’t have dreamed of." -- A pretty bleak, and also accurate, observation of humanity. I have to hope that the alternative sentence encompassing all of the good can lead to some balance.

No, it is not accurate at all. There are some people that do all of these sure, however the vast majority of people live pretty ordinary lives where they do very little of what is described. I actually think it is very intellectually lazy to be this cynical.

It's absolutely true, but most of the destruction is abstracted away so modern humans don't have to experience it directly or even really think about it. A staggering number of creatures are killed by normal people driving to normal things every year. Many more are killed by all of the resource extraction needed to supply our normal lives. Not to mention the vast numbers of wildlife that sees their habitat destroyed every year to make way for more housing and other development.

Killing animals for fun is an entire sport enjoyed by millions. Humans keep pets that kill billions of birds every year. The limited areas we've set aside to mostly let other nature be nature are constantly under threat and being shrunk down. The list of ways we completely subjugate other intelligent life on this planet is endless. We have driven many hundreds of species to extinction and kill countless billions every year.

I certainly enjoy the gains our species have made, just like everyone else on HN. I'd rather be in our position than any other species on our planet. But given our history I'm also pretty terrified of what happens if and when we run into a smarter and more powerful alien species or AI. If history is any guide, it won't go well for us.

This understanding can guide practical decisions. We shouldn't be barreling towards a potential super intelligence with no safeguards given how catastrophic that outcome could be, just like we shouldn't be shooting messages into space trying to wave our arms at whatever might be out there, any more than a small creature in the forest would want to attract the attention of a strange human.

Re: You are the scariest monster in the woods

#273

Post by pennomi »

lurk2 wrote: post #225
pennomi wrote: post #200

Earlier quoted context omitted.

It’s not an assumption, it’s a viable theory based on overwhelming evidence from fossil records. What’s NOT supported by evidence is an unknowable, untestable spiritual requirement for cognition.

> it’s a viable theory based on overwhelming evidence from fossil records No one has gathered evidence of cognition from fossil records.

Sure they have. We see every level of cognition in animals today, and the fossil record proves that they all came from the same evolutionary tree. For every species that can claim cognition (there’s lots of them), you can trace it back to predecessors which were increasingly simple.

Obviously cognition isn’t a binary thing, it’s a huge gradient, and the tree of life shows that gradient in full.

Re: You are the scariest monster in the woods

#275

Post by evilduck »

danielbln wrote: post #60
Chance-Device wrote: post #50

Earlier quoted context omitted.

The only thing I can come up with is that compressing several hundred million years of natural selection of animal nervous systems into another form, but optimised by gradient descent instead, just takes a lot of time. Not that we can’t get there by artificial means, but that correctly simulating the environment interactions, the sequence of progression, getting the all the details right, might take hundreds to thous…

I don't disagree, but I think the evolution argument is a red herring. We didn't have to re-engineer horses from the ground up along evolutionary lines to get to much faster and more capable cars.

Most arguments and discussions around AGI talk past each other about the definitions of what is wanted or expected, mostly because sentience, intelligence, consciousness are all unagreed upon definitions and therefore are undefined goals to build against.

Some people do expect AGI to be a faster horse; to be the next evolution of human intelligence that's similar to us in most respects but still "better" in some aspects. Others expect AGI to be the leap from horses to cars; the means to an end, a vehicle that takes us to new places faster, and in that case it doesn't need to resemble how we got to human intelligence at all.

Re: You are the scariest monster in the woods

#276

Post by myrmidon »

lo_zamoyski wrote: post #78
myrmidon wrote: post #13

I struggle to understand how people can just dismiss the possibility of artificial intelligence. Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I'm not buying the whole "AI has no agency" line eith…

I think you make a lot of assumptions that you should perhaps reexamine. > Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? Here are some of your assumptions: 1. Human intelligence is entirely explic…

I'm just assuming materialism, and that assumption is basically for complete lack of convincing alternatives (to me).

With "agency" I just mean the ability to affect the physical world (not some abstract internal property).

Regarding "computers have no concepts of things": I'm happy with the "meaning" of something being a fuzzy cloud in some high dimensional space, and consider this plausible/workable both for our minds and current LLMs.

> you're taking a superficial black box view of intelligence

Yes. Human cognition is to me simply an emergent property of our physical brains, and nothing more.

Re: You are the scariest monster in the woods

#277

Post by santadays »

mediaman wrote: post #247
santadays wrote: post #238

Earlier quoted context omitted.

It seems like there is a bunch of research/working implementations that allow efficient fine tuning of models. Additionally there are ways to tune the model to outcomes vs training examples. Right now the state of the world with LLMs is that they try to predict a script in which they are a happy assistant as guided by their alignment phase. I'm not sure what happens when they start getting trained in simulations to b…

That's already happening. It started happening when they incorporated reinforcement learning into the training process. It's been some time since LLMs were purely stochastic average-token predictors; their later RL fine tuning stages make them quite goal-directed, and this is what has given some big leaps in verifiable domains like math and programming. It doesn't work that well with nonverifiable domains, though, si…

That makes sense for why they are so much better at writing code than actually following the steps the same code specifies.

Curious, is anyone training in adversarial simulations? In open world simulations?

I think what humans do is align their own survival instinct with a surrogate activities and then rewrite their internal schema to be successful in said activities.

Re: You are the scariest monster in the woods

#278

Post by olooney »

I think this is interesting and partially true: humans are scary. But it's important to remember the opposite is true as well: humans are the most cooperative species out there by a wide margin.

Eusocial insects and pack animals are in a distant 2nd and 3rd place: they generally don't cooperate much past their immediate kin group. Only humans create vast networks of trade and information sharing. Only humans establish establish complex systems to pool risk, or undertake public works for the common good.

In fact, a big part of the reason we are so scary is that ability to coordinate action. Ask any mammoth. Ask the independent city states conquered by Alexander the Great. Ask Napoleon as he faced the coalition force at Waterloo.

We are victims of our own success: the problems of the modern world are those of coordination mechanisms so effective and powerful that they become very attractive targets for bad actors and so are under siege, at constant risk of being captured and subverted. In a word, the problem of robust governance.

Despite the challenges, it is a solvable problem: every day, through due diligence, attestations, contract law, earnest money, and other such mechanisms people who do not trust each other in the least and have every incentive to screw over the other party are able to successfully negotiate win-win deals for life altering sums of money, whether that's buying a house or selling a business. Every century sees humans design larger, more effective, more robust mechanisms of cooperation.

It's slow: it's like debugging when someone is red teaming you, trying to find every weak point to exploit. But the long term trend is the emergence of increasingly robust systems. And it suggests a strategy for AI and AGI: find a way to cooperate with it. Take everything we've learned about coordinating with other people and apply the same techniques. That's what humans are good at.

This, I think, is a more useful framing than thinking of humans as "scary."

Re: You are the scariest monster in the woods

#279

Post by the8472 »

BoxOfRain wrote: post #268
the8472 wrote: post #244

Earlier quoted context omitted.

How do you tell the difference in other humans? Do you just believe them because they claim to be conscious instead of pointing a calibrated and certified consciousness-meter at them?

I obviously can't prove they're conscious in a rigorous way, but it's a reasonable assumption to make that other humans are conscious. "I think therefore I am" and since there's no reason to believe I'm exceptional among humans, it's more likely than not that other humans think too. This assumption can't be extended to other physical arrangements though, not unless there's conclusive evidence that consciousness is a…

So, what if I told you that some humans have been vat-grown without brains and had a silicon brain emulator inserted into their skulls. Are they p-zombies? Would you demand x-rays before talking to anyone? What would you use then to determine consciousness?

Relying on these status quo proxy-measures (looks human :: 99.9% likely to have a human brain :: has my kind of intelligence) is what gets people fooled even by basic AI (without G) fake scams.

Re: You are the scariest monster in the woods

#280

Post by jncfhnb »

lambaro wrote: post #234
jncfhnb wrote: post #111

Earlier quoted context omitted.

I disagree. Personally I found the definition of a game engine as ``` while True: update_state() draw_frame()``` To be a profound concept. The implementation details are significant. But establishing the framework behind what we’re actually talking about is very important.

Oh, well, enjoy your trillions of dollars then. Don't forget about us back here at HN.

The bad faith snark will remain with me forever
Post reply on HN