I agree with what you're worried about - humans using AI to do bad things. However, have you considered also being worried about what AI can do? (can you ever have too much existential dread?) You cast it as a tool, it may be the first thing we've made that goes beyond that.
You are the scariest monster in the woods
231–240 of 405 posts
Re: You are the scariest monster in the woods
#232I struggle to understand how people can just dismiss the possibility of artificial intelligence. Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I'm not buying the whole "AI has no agency" line eith…
> Human cognition was basically bruteforced by evolution You center cognition/intelligence on humans as if it was the pinacle of it, rather than include the whole lot of other species (that may have totally different, or adjacent cognition models). Why? How so? > As soon as profit can be made by transfering decision power into an AIs hand There's an ironic, deadly, Frankensteinesque delusion in this very premise.
> You center cognition/intelligence on humans as if it was the pinacle of it, rather than include the whole lot of other species (that may have totally different, or adjacent cognition models).
Why does that matter to their argument? Truly, the variety of intelligences on earth now only increases the likelihood of AGI being possible, as we have many pathways that don't follow the human model.
Re: You are the scariest monster in the woods
#233There's this weird disconnect in tech circles, where everyone is deathly afraid of AGI, but totally asleep on the very real possibility of thermonuclear war breaking out in Europe or Asia over the next 10 years. There's already credible evidence that we came perilously close to the use of tactical nuclear weapons in Ukraine which likely would've spiraled out of control. AGI might happen, but the threat of nuclear war…
>There's already credible evidence that we came perilously close to the use of tactical nuclear weapons in Ukraine which likely would've spiraled out of control.
I do agree nukes are a far more realistic threat. So this is kind of an aside and doesn't really undermine your point.
But I actually think we widely misunderstand the dynamic of using nuclear weapons. Nukes haven't been used for a long time and everyone kind of assumes using them will inevitably lead to escalation which spirals into total destruction.
But how would Russia using a tactical nuke in Ukraine spiral out of control? It actually seems very likely that it would not be met in kind. Which is absolutely terrifying in it's own right. A sort of normalization of nuclear weapons.
Re: You are the scariest monster in the woods
#234Earlier quoted context omitted.
"STEP 2: Draw the rest of the owl."
I disagree. Personally I found the definition of a game engine as ``` while True: update_state() draw_frame()``` To be a profound concept. The implementation details are significant. But establishing the framework behind what we’re actually talking about is very important.
Oh, well, enjoy your trillions of dollars then. Don't forget about us back here at HN.
Re: You are the scariest monster in the woods
#235Re: You are the scariest monster in the woods
#236I struggle to understand how people can just dismiss the possibility of artificial intelligence. Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I'm not buying the whole "AI has no agency" line eith…
The worst thing Star Trek did was convince a generation of kids anything is possible. Just because you imagine a thing doesn’t make it real or even capable of being real. I can say “leprechaun” and most people will get the same set of images in their head. They aren’t real. They aren’t going to be real. You imagined them.
That's not Star Trek, that's marketing.
Marketing grabbed a name (AI) for a concept that's been around in our legends for centuries and firmly welded it to something else. You should not be surprised that people who use the term AI think of LLMs as being djinn, golems, C3PO, HAL, Cortana...
Re: You are the scariest monster in the woods
#237I struggle to understand how people can just dismiss the possibility of artificial intelligence. Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I'm not buying the whole "AI has no agency" line eith…
I think dismissing possibility of evolving AI, is simply ignorance (and a huge blindspot)
This said, I think the author's point is correct. It's more likely that unwanted effects (risks) from the intentional use of AI by humans is something that precedes any form of "independent" AI. It already happens, it always has, it's just getting better.
Hence ignoring this fact makes the "independent" malevolent AI a red herring.
On the first point - LLMs have sucked almost all the air in the room. LLMs (and GPTs) are simply one instance of AI. They are not the beginning and most likely not the end (just a dead end) and getting fixated on them on either end of the spectrum is naive.
Re: You are the scariest monster in the woods
#238Earlier quoted context omitted.
> I'm not buying the whole "AI has no agency" line either; this might be true for now, but this is already being circumvented with current LLMs (by giving them web access etc). They don't have agency, because they don't have persistent state. They're like a function that you can query and get an answer. During that answer the LLM has state, but once it's done the state is gone. Humans (and other "agents") have persis…
In Context Learning (ICL) is already a rapidly advancing area. You do not need to modify their weights for LLMs to persist state. The human brain is not that different. Our long-term memories are stored separately from our executive function (prefrontal cortex), and specialist brain functions such as the hippocampus serve to route, store, and retrieve those long term memories to support executive function. Much of th…
It seems like there is a bunch of research/working implementations that allow efficient fine tuning of models. Additionally there are ways to tune the model to outcomes vs training examples.
Right now the state of the world with LLMs is that they try to predict a script in which they are a happy assistant as guided by their alignment phase.
I'm not sure what happens when they start getting trained in simulations to be goal oriented, ie their token generation is based off not what they think should come next but what should come next in order to accomplish a goal. Not sure how far away that is but it is worrying.
Re: You are the scariest monster in the woods
#239I struggle to understand how people can just dismiss the possibility of artificial intelligence. Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I'm not buying the whole "AI has no agency" line eith…
"As soon as profit can be made" is exactly what the article is warning about. This is exactly the "Human + AI" combination.
Within your lifetime (it's probably already happened) you will be denied something you care about (medical care, a job, citizenship, parole) by an AI which has been granted the agency to do so in order to make more profit.
Re: You are the scariest monster in the woods
#240"Humans will do what they’ve always tried to do—gain power, enslave, kill, control, exploit, cheat, or just be lazy and avoid the hard work—but now with new abilities that we couldn’t have dreamed of." -- A pretty bleak, and also accurate, observation of humanity. I have to hope that the alternative sentence encompassing all of the good can lead to some balance.
this is accurate because of the few who do it. however the cautionary and hopeful tale behind it is the majority when they stand up against it can change the distribution of power. today however we're comfortable and soft and too scared to do - so posts like this remind us to gain some courage to stand up for change.