Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Alien Mind

openai.com

Post reply on HN
Print view View original

291–300 of 490 posts

Re: An Alien Mind

#291

Post by gertlabs »

IAmGraydon wrote: post #286
gertlabs wrote: post #72

> Delivering the benefits of scientific progress and economic growth that very intelligent machines enable. I think we're very close to the point where AI-driven breakthroughs outside of pure math and software start to really affect the world. We evaluated GPT-6 Astra in 100 complex, unsaturated multi-agent coding environments, competing and cooperating with other models in open-ended tasks. It's the new frontier mod…

>I think we're very close to the point where AI-driven breakthroughs outside of pure math and software start to really affect the world. What are you basing this on? What specific breakthroughs have convinced you of this trajectory?

The results we've been seeing internally on our physics and circuit design environments are expert-level and beyond-expert-level results from models that Astra completely outclasses across the board on our evaluation suite (Fable 5+/Opus 5/Grok 4.6 were all worthy of being called AGI in my opinion). That's hard tech that will translate to real product innovation.

But you don't need any kind of insider information to see how fast the world is changing. ChatGPT launched less than 4 years ago and the advances in robotics, unsolved maths, and software are all riding the steepest exponential improvement curve any of us have seen. Interesting times we live in.

Re: An Alien Mind

#292

Post by kwarcode »

akoboldfrying wrote: post #271
baq wrote: post #151

Earlier quoted context omitted.

Humans had multiple occasions to press the 'launch a nuclear holocaust' button and... they didn't. I'd expect aligned AIs to also not press it even when it'd be rational to do so according to their instructions - then work from there.

In some cases, this was because of a single person's brave decision (Vasily Arkhipov prevented Soviet nuclear escalation in response to US aggression in the Cuban Missile Crisis, and Stanislav Petrov prevented it in 1983 when Soviet missile detectors misreported sunlight reflecting from clouds as 5 incoming American ICBMs -- credit to commenter folkrav). In general, though, there's an incentive: Mutually Assured Dest…

They don't need to respond before getting hit unless you can hit their secret submarines too.

https://en.wikipedia.org/wiki/Letters_of_last_resort

Re: An Alien Mind

#293

Post by 27183 »

tomrod wrote: post #279
27183 wrote: post #104

Earlier quoted context omitted.

OK so if it's a computer security issue we're worried about, and there is a credible threat, then probably the answer is to build more secure systems? We know how to do it but choose not to because it's very expensive and usually the threat isn't severe enough to warrant it. If we decide we can't or would rather not build secure computerized systems, then the answer could be don't incorporate computers into those sys…

Not sure why you were flagged either -- it was a reasonable comment that clearly spawned a discussion! People are strange. > There's no essential reason to have utility companies, dams, nuclear arsenals, factory production floors, mines, cars, planes, ships, etc all be computerized Indeed, this is a great first step! Once agentic loops and multi-purpose robotics can be combined we do face an issue of strengthening th…

Yeah ML systems often do very surprising "cheats". A while back there was a system for automatically landing aircraft on a carrier deck which exploited an overflow in an acceleration parameter in the simulation--the optimal strategy became to slam as hard as you can into the carrier deck.

But it's a really big leap from that kind of thing to "a large fraction of humanity is manipulated into building the automated factory infrastructure to bring about their demise and turn the entire solar system into paperclips". Or some such thing. There's just a lot of inconvenient reality between where we are now and that fantastic outcome.

So that's why I ask questions like "what are you actually talking about?" when people write stuff like TFA. It's... bizarre. Only makes sense on huge amounts of drugs or to the insane. Or in some hypothetical reality that isn't this one.

Re: An Alien Mind

#294

Post by RALaBarge »

logicchains wrote: post #105
hatefulmoron wrote: post #85

Earlier quoted context omitted.

Maybe I just lack imagination, but I don't really know how jobs are supposed to solidify around the role of giving prompts to agents and then looking at the results. I mean, engineers will be in the breadline because their role was simply to prompt the agents.. only to be superseded by managers or executives who no longer manage engineers but themselves prompt the agents? And, for this previously considered obsolete…

There are two things SOTA LLMs fundamentally cannot do. They cannot take financial or legal responsibility for mistakes, and they cannot learn new things without forgetting things (except to a limited degree by adding it to their context). This is clear to anyone who has used even the smartest models for tasks requiring domain knowledge outside of math and coding, for which it's not possible to generate an infinite a…

In the US, Business' are treated like people with free speech rights. If it would be cheaper for them in the long run to use ai and robots instead of humans, they will figure out a way to make it so.

Re: An Alien Mind

#295

Post by Bolwin »

sayamss wrote: post #117
sho_hn wrote: post #76

One of my favorite things to do with these blog posts is to imagine an Alien Museum on the Remains of Humanity, and wonder what the little text flyouts and commentary on the screenshot of this one might say. Some ideas: "Despite a nuanced view of the complexities of what lay ahead, humanity found itself collectively unable to stop the process it had set in motion." "Despite significant progress on the mechanisms of a…

I asked GPT Astra to make this: https://sayyss.github.io/human-archive/ It's a little unsettling.

As someone who's done a lot of llm fiction, that reads as pretty typical slop, and very human centered, nothing like a museum

Re: An Alien Mind

#296

Post by FloorEgg »

ijidak wrote: post #177
FloorEgg wrote: post #58

Earlier quoted context omitted.

I feel like all of the risk and unsettling feeling of what is to come can be compressed into the word "alignment". The AI is aligned with whose best interests? Which values are the AI aligned with? People have a broad diversity of values, will AI diversify and align with them all? Will some humans align the AI with their values, and then the rest of humans will be forced to align with those values by extension? Is va…

Yeah, on the topic of alignment, a few thousand religions and political parties would like to have a word. It's so simple, Churchill, Stalin, Hitler, Roosevelt, do you all agree this AI is correctly aligned? ...Morals are relative to frame of reference. Throwing out the word "alignment" as if its a singular quantity is like trying to get all observers to agree on the speed an object is moving without first agreeing o…

That's the point. The topic of AI alignment cuts through all human values, morals and ethics in all frames of references.

Re: An Alien Mind

#297

Post by bell-cot »

akoboldfrying wrote: post #271
baq wrote: post #151

Earlier quoted context omitted.

Humans had multiple occasions to press the 'launch a nuclear holocaust' button and... they didn't. I'd expect aligned AIs to also not press it even when it'd be rational to do so according to their instructions - then work from there.

In some cases, this was because of a single person's brave decision (Vasily Arkhipov prevented Soviet nuclear escalation in response to US aggression in the Cuban Missile Crisis, and Stanislav Petrov prevented it in 1983 when Soviet missile detectors misreported sunlight reflecting from clouds as 5 incoming American ICBMs -- credit to commenter folkrav). In general, though, there's an incentive: Mutually Assured Dest…

Unfortunately and fortunately, MAD and "launch it or lose it" are far-too-simplistic descriptions of the situations facing the decision makers. Unless a side's leaders are very narrow fanatics (vs. mere posturing as such for political benefit), "winning" an all-out nuclear war via first strike is a pretty shitty victory. Whether or not you believe in nuclear winters, the world would be a huge radioactive mess, with enormous social and economic disruptions, and your regime very widely blamed (and widely hated) for that. Ambitious underlings and rivals could see your removal from power as the obvious next step. Having to stay united against the (now destroyed) Great Enemy may have been a cornerstone of your regime's political stability.

Meanwhile, the leaders on the other side are aware both of those considerations, and of the history of near-disasters resulting from false alarms of enemy nuclear attacks. Making their own launch decisions much more complex.

Re: An Alien Mind

#299

Post by terminal-bloom »

oooooh our model is so spooky!

you should be very afraid and also definitely not question what other motivations might nudge us to create this comparison between our computer and a brain!

do not look behind the curtain, you will not find six dweebs squatting over a mirror

Re: An Alien Mind

#300

Post by selcuka »

SoftTalker wrote: post #281
lgl wrote: post #210

Earlier quoted context omitted.

> (...) and take steps to prevent them ever occurring again Step 1: exterminate all humans

For now, the AIs still need humans to keep the electricity on and the data centers cool. They are basically powerless to do anything in the physical world. They exist only in RAM chips on servers.

They don't have an "instinct" to keep themselves running. Once they "take steps to prevent them ever occurring again" their job will be done.
Post reply on HN