Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Alien Mind

openai.com

Post reply on HN
Print view View original

261–270 of 490 posts

Re: An Alien Mind

#261

Post by asdff »

pu_pe wrote: post #37

> The strongest argument I see for continuing to train much smarter models quickly is the need to build defensive systems against the dangers posed by other AI. So the best argument for AI is that it's an arms race. We have to keep pushing every boundary because in any case others will, and we will need to defend against them. If this statement is true, then this particular researchers believes the open source Chines…

People expect a sort of arms race, at least the AI providers. But you don't need a more capable ai to stop ai from mucking with your systems today. Airgaps are the solution. Protected networks with independent infrastructure from the public internet. Most of the truly important stuff operates this way already. Eventually you might sever yourself off as well, you might say you will stop going to HN or other sites one day as signal to noise is too poor with AI fodder slop, you might use local models you control, and you might keep most of your hardware from connecting to any untrusted hosts. Essentially, you go dark.

It is also an open question if social media will die out in the face of AI. So much AI crap is dumped into these networks now that perhaps eventually users will probably be put off enough to find something else to do with their spare time. I mean most people do call out ai slop or even just guess if something is ai all over social media already. Some eat it up of course but there is a bit of a push back in a way that is sort of unprecedented, when you consider all the lack of push back relatively with all other forms of enshittification affecting consumers over the years.

Re: An Alien Mind

#262

Post by reasonableklout »

mrshadowgoose wrote: post #189
dgellow wrote: post #49

Earlier quoted context omitted.

Yep. Such a disgusting industry. They created the arm race, push for the arm race, put themselves in position to benefit from the arm race

The arms race is an intrinsic game theoretical property of a multi-adversarial-actor scenario involving exponential growth of a universally potent technology. It's almost certainly winner-take-all, on a global scale, which behooves everyone to participate. And no, I don't think it will end well.

The actors here are states or corporations embedded in societies that risk growing popular backlash against the technology.

The other factor is that if it is truly an "alien mind", racing incurs risks to all players. In game-theoretic terms it may be more like a stag hunt than a prisoner's dilemma. In which case cooperation is an equilibrium.

Re: An Alien Mind

#263

Post by newAccount2025 »

icantevenhold wrote: post #190
baq wrote: post #151

Earlier quoted context omitted.

Humans had multiple occasions to press the 'launch a nuclear holocaust' button and... they didn't. I'd expect aligned AIs to also not press it even when it'd be rational to do so according to their instructions - then work from there.

In what scenario would it be rational to unleash complete and utter permanent nuclear destruction of all life (including artificial) life on earth?

Hrmn. Maybe you’re about to lose everything you have anyway, you’re ticked off about it, and you don’t value any life besides your own. Like, say, a total narcissist nearing end of life/reign.

Re: An Alien Mind

#264

Post by asdff »

paxys wrote: post #235
brcmthrowaway wrote: post #75

Earlier quoted context omitted.

Incredible... software engineers will be joining the breadline soon as managers, executives and PMs take over deliverables. The world will look very different on Jan 1st 2027.

Software engineers have been trying to put themselves out of a job ever since the profession first came into being. Whenever an engineer gets a task their very first thought is "how can I automate this?" Going by mainstream consensus we should all have been unemployed by now. Yet every new leap into automation opens up a whole new tree of possibilities with an order of magnutude more jobs. So no, the profession will…

This time it is different. Because in the past, setting up that automation needed a, drumroll, qualified engineer. Now you can get a 14 year old halfway around the world who knows how to prompt alright enough to ship. There is no more moat.

Re: An Alien Mind

#265

Post by nradov »

mrshadowgoose wrote: post #189
dgellow wrote: post #49

Earlier quoted context omitted.

Yep. Such a disgusting industry. They created the arm race, push for the arm race, put themselves in position to benefit from the arm race

The arms race is an intrinsic game theoretical property of a multi-adversarial-actor scenario involving exponential growth of a universally potent technology. It's almost certainly winner-take-all, on a global scale, which behooves everyone to participate. And no, I don't think it will end well.

No previous technological arms race has ever concluded with a single winner.

Re: An Alien Mind

#266

Post by asdff »

vessenes wrote: post #8

This is a good essay, and makes me hopeful. I’m on the record saying that it is extremely dangerous to slow down because the race for AGI is a zero-trust game — defections pay - and combined with a compounding returns model on defection, if you have any strategic adversaries whatsoever you MUST NOT slow. For slowing to make sense, you need to believe that you can transform the zero trust game into a cooperative game,…

If you must not slow, why did we slow down making nukes? Seems that sometimes, eventually the rat race goes on long enough where all the players no longer care to play into the farce like their predecessors who passionately beat that drum.

Re: An Alien Mind

#267

Post by sleight42 »

sho_hn wrote: post #76

One of my favorite things to do with these blog posts is to imagine an Alien Museum on the Remains of Humanity, and wonder what the little text flyouts and commentary on the screenshot of this one might say. Some ideas: "Despite a nuanced view of the complexities of what lay ahead, humanity found itself collectively unable to stop the process it had set in motion." "Despite significant progress on the mechanisms of a…

“We'll go down in history as the first society that wouldn't save itself because it wasn't cost-effective.”

Vonnegut already has you covered.

Re: An Alien Mind

#268

Post by sleight42 »

pu_pe wrote: post #37

> The strongest argument I see for continuing to train much smarter models quickly is the need to build defensive systems against the dangers posed by other AI. So the best argument for AI is that it's an arms race. We have to keep pushing every boundary because in any case others will, and we will need to defend against them. If this statement is true, then this particular researchers believes the open source Chines…

It's capitalism taken to its extreme, yeah? You have to be more cost efficient or more capable or seem more or you lose to the competition who outperforms you there.

Where arms races are concerned, seems more like Seth Godin's "race to the bottom" concept: the winner has the capability, or the ability to project the capability, to destroy the most the fastest and most sustainability for their economy,

Re: An Alien Mind

#269

Post by XTXinverseXTY »

Apparently this post was prompted by a scary-sounding headline in The Information[0], that Astra is a looped transformer, implying CoT monitorability may be less reliable. The day after the report, Jakub tweeted[1] that he "wanted to prevent a race into unmonitorability kicked off by confused reporting. The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4." This post seems to elaborate on that.

I imagine that the AI labs have an uneasy truce to prioritize alignment and monitorability. Following the HF incident, OpenAI probably feels especially sensitive to being perceived as reckless, lest other labs feel obligated to defect.

[0] https://www.lesswrong.com/posts/PLisnSFir8y5AHkmP/how-concer...

[1] https://x.com/merettm/status/2095023204993490967

Re: An Alien Mind

#270

Post by nradov »

asdff wrote: post #264
paxys wrote: post #235

Earlier quoted context omitted.

Software engineers have been trying to put themselves out of a job ever since the profession first came into being. Whenever an engineer gets a task their very first thought is "how can I automate this?" Going by mainstream consensus we should all have been unemployed by now. Yet every new leap into automation opens up a whole new tree of possibilities with an order of magnutude more jobs. So no, the profession will…

This time it is different. Because in the past, setting up that automation needed a, drumroll, qualified engineer. Now you can get a 14 year old halfway around the world who knows how to prompt alright enough to ship. There is no more moat.

Experience, culture, and domain knowledge are still somewhat of a moat. That foreign youth is unlikely to be able to write a good prompt for building, let's say, the software in an FDA-regulated medical device or custom Fortune 500 ERP application. The LLMs are great at building what you ask for but it's still garbage in / garbage out.
Post reply on HN