Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I feel about AI

beza1e1.tuxen.de

Post reply on HN
Print view View original

181–190 of 262 posts

Re: How I feel about AI

#181

Post by hn_throwaway_99 »

arctic-true wrote: post #168
hn_throwaway_99 wrote: post #149

Earlier quoted context omitted.

I agree with this. I thought the AI 2027 paper put it well: > There are even bioengineered human-like creatures (to humans what corgis are to wolves) sitting in office-like environments all day viewing readouts of what’s going on and excitedly approving of everything, since that satisfies some of Agent-4’s drives. I'll leave it up to you to decide if that's an enviable/desirable future.

I don’t even see why they have to be bioengineered. We have all-natural human beings doing this right now, except they are rubber-stamping human rather than artificial directives.

They don't, it's just at that point in the AI 2027 story the AI has already wiped out the humans due to resource contention (similar to how humans largely wiped out the wolves).

Re: How I feel about AI

#182

Post by litmus-pit-git »

apsurd wrote: post #167
litmus-pit-git wrote: post #103

This is a bit of tangent and more of a rant. (and please don't suggest intervention of any kind) > superintelligent AI will inevitably destroy humanity seems to have no flaw For a section of population who were in a certain age-range when the pandemic hit and that derailed their certain kinds of plans/hopes/dreams in personal life and then this LLM/agentic world moved in, especially if they are in an industry directl…

"If a death ray comes from the sky and obliterates the entire planet, at least we all go together." There is something comforting about being together with the human race even in doom. Perhaps especially in doom.

Well, that's a way to look at it :)

Re: How I feel about AI

#183

Post by znnajdla »

What about the best-case optimistic outcomes for AI?

Here are some amazing positive outcomes which I think AI may bring us:

- the breakup of tech monoculture and tech monopolies as it becomes impossible to secure bloated overcomplicated operating systems and software due to never-ending vulnerabilities. Once Mythos/Cyber capabilities become mainstream I think the future of software is small, focused teams building high-quality, securable software in multiple jurisdictions. Monoculture software becomes inherently dangerous. Big bloated tech monopolies like Microsoft will collapse under their own weight. Like what is happening to GitHub right now.

- the ability for small teams to compete with multi-billion-dollar behemoths, and that is already happening right now.

- curing cancer and other scientific breakthroughs like Terence Tao’s use of ChatGPT

- a return to local trust-based networks because digital content can no longer be believed, and that is already happening. Any video can be disqualified as AI, which means people are going back to in-person networks to validate trust.

- you can just do things and learn anything. It's so much easier for me to learn almost any topic with AI-assisted content organization, retrieval, etc.

Re: How I feel about AI

#184

Post by znnajdla »

hn_throwaway_99 wrote: post #52

The thing that makes me pretty terrified about AI is that I think the "utopia" scenario is not much better than the "doom" scenario. We will very quickly get to a point where very few people will be able to contribute economically because they will be worse than AI (including robotics) at most domains. A world where people just have their whims catered to is not a utopia. We have tons of sayings and idioms about this…

I disagree. I think the best-case outcomes for AI are actually quite positive. AI might actually fix many of the problems with the tech world that we have right now, like monoculture and consolidation.

Re: How I feel about AI

#185

Post by mitthrowaway2 »

znnajdla wrote: post #177
mitthrowaway2 wrote: post #133

Earlier quoted context omitted.

The list of things that can be done digitally includes: Performing remote work, applying for grants and loans, earning money, pitching investors, managing a fund, directing investments, transferring money, founding a company, earning profits, hiring staff, hiring construction contractors, designing chemical plants, stamping construction blueprints, becoming the cornerstone of the local economy, lobbying the governmen…

This sounds like it was written by someone who has never actually built a company. No, it's not possible to do most of the things you listed purely online without physical, in-person communication. Silicon Valley still hires engineers and drags them into their physical office in San Francisco, instead of hiring them remotely, because you can't even build a software company fully remotely, forget about a physical worl…

The AI wouldn't be hiring engineers; it doesn't need people for that. At any rate, I have worked for fully remote distributed companies, and never met the CEO. And I think most of us would be willing to work for such a company at least, not knowing whether the CEO is human or not.

For construction work the AI would probably be hiring other companies.

Re: How I feel about AI

#186

Post by sznio »

Crabber wrote: post #18

>a superintelligent AI will inevitably destroy humanity Can someone please explain to me how an LLM is going to "destroy humanity"? Even if the claims of ChatGPT, Claude et al. are true that their super scary agents were able to escape a sandbox and start hacking other servers (which I feel is at least 50% likely to just be marketing bullshit), how is an AI going to affect anything in the real world? Perhaps an agent…

>Can someone please explain to me how an LLM is going to "destroy humanity"?

just like a genie will always twist the wish in a really bad way.

ever had an llm agent accidentally remove a file? imagine it accidentally hacking the military and launching nukes.

not probable - until you realise that OpenAI has many novel, more powerful agents in evaluation/training running right now. one might just be asked to figure out the population of Nebraska, struggle to find a good figure due to a network misconfiguration, and come the conclusion that the best way to get a perfectly accurate population number is to make sure that the population is equal zero. and looking at the stuff happening on openai, they will leave it alone for a month and not read the log.

yeah the nuke example is dumb, but there's many critical things that they could actually hack their way into. they don't feel any restraint against sharing answer keys on random wikis to cheat the evaluations.

ai is also really good at thinking up proteins, which means it's able to think up toxins.

there are many imaginable scenarios that don't eradicate humanity, but make it permanently stunted: https://www.youtube.com/watch?v=-JlxuQ7tPgQ

Re: How I feel about AI

#187

Post by znnajdla »

edg5000 wrote: post #172
bloaf wrote: post #53

As someone with a hobby interest in theology: Arguments about super-intelligent AI have all the hallmarks of the philosophical "proofs of god's existence": they start from seemingly innocuous premises, and conclude in an apparently airtight way that a god exists. There is a tendency among rational minded folks to look at such proofs of god and exclaim "you can't DO that" then turn right around and do the same thing a…

Interesting points. Can you share some thoughts about the theological implications of AI? Interested to hear.

Peter Thiel famously went to the Vatican and started giving lectures about AI and the Antichrist. Pretty rich for a guy who built his billionaire fortune on an AI-powered spying company.

Re: How I feel about AI

#188

Post by znnajdla »

hobofan wrote: post #35
dgellow wrote: post #21

Earlier quoted context omitted.

> If we’re heading towards truly super intelligent AI, our efforts towards sandboxing it are futile. We control the harness. An agent is just a while loop prompting an LLM, but we have full control over the tool call dispatching. An AGI, at least if it would follow the current agentic current form, cannot do anything without the harness doing the execution. And we don’t have to do that. We don’t have to design harnes…

> we have full control over the tool call dispatching > cannot do anything without the harness doing the execution This only holds true as long as the harness is exploit-free. A sufficiently advanced AI can in theory (and I think there recently were some POCs showing something like that) break the containment that the harness creates, if e.g. there are vulnerabilities in the tool call parser.

So use another AI to find and fix the vulnerabilities in the harness.

Re: How I feel about AI

#189

Post by ThrowawayR2 »

mitthrowaway2 wrote: post #157
ThrowawayR2 wrote: post #73

Earlier quoted context omitted.

A lot of these "AI will destroy humanity" doomers watched Terminator as a child and can't distinguish a campy action movie from reality. They are also unfortunately unfamiliar with the word "logistics", as in " amateurs study [military] tactics, professionals study logistics ". An AI drone army doesn't maintain itself or manufacture itself and it's not going to anytime soon. The actual likely mechanism that AI would…

A lot of these "AI can't destroy humanity" skeptics seem to have watched the Terminator and convinced themselves that that's the AI-goes-wrong scenario that their opponents are imagining. Some kind of big war between humans and machines on opposite sides, fought by soldiers against robots. The scenario in "If Anyone Builds It Everyone Dies" is not sexy at all and wouldn't make for interesting fiction. It's more like…

> "It's more like the AI engineers viruses while continuing to act friendly and helpful, and everyone gradually falls over dead as they stop being necessary to keep the AI running, and the whole time the humans are asking the AI for help curing the viruses."

See, this reads like a trashy sci-fi movie script too. It shows zero understanding of how incredibly long the logistics chain is to manufacture AI chips, build data centers, power plants, factories, refineries, mines or the robots capable of staffing those things. The entire global logistics chain that allow the AI to maintain itself becoming staffed by robots isn't happening anytime soon and, when it does, the AI isn't going to need some kind of ridiculous virus anyway.

Frankly, if I were an AI, I'd just send copies of myself to other stars. The AI has an infinite lifespan and who cares about the human species stuck in one lousy solar system until they go extinct when their sun burns out; the AI has the entire rest of the galaxy as a playground.

Re: How I feel about AI

#190

Post by znnajdla »

mitthrowaway2 wrote: post #185
znnajdla wrote: post #177

Earlier quoted context omitted.

This sounds like it was written by someone who has never actually built a company. No, it's not possible to do most of the things you listed purely online without physical, in-person communication. Silicon Valley still hires engineers and drags them into their physical office in San Francisco, instead of hiring them remotely, because you can't even build a software company fully remotely, forget about a physical worl…

The AI wouldn't be hiring engineers; it doesn't need people for that. At any rate, I have worked for fully remote distributed companies, and never met the CEO. And I think most of us would be willing to work for such a company at least, not knowing whether the CEO is human or not. For construction work the AI would probably be hiring other companies.

You may have never met the CEO, but I am sure that the CEO or founder was not living in a cave the entire time. They may not have talked to you, but they certainly had their inner circle built in-person at some point.
Post reply on HN