Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Alien Mind

openai.com

Post reply on HN
Print view View original

301–310 of 490 posts

Re: An Alien Mind

#301

Post by nullbio »

TheOtherHobbes wrote: post #139
sho_hn wrote: post #76

One of my favorite things to do with these blog posts is to imagine an Alien Museum on the Remains of Humanity, and wonder what the little text flyouts and commentary on the screenshot of this one might say. Some ideas: "Despite a nuanced view of the complexities of what lay ahead, humanity found itself collectively unable to stop the process it had set in motion." "Despite significant progress on the mechanisms of a…

Collectively, humans aren't aligned, and don't build aligned systems. Humans have a concept of alignment, and multiple traditions, practices, and systems that aggressively oppose it. Why would AI be any different?

My opinion is that serious repercussions for lying would fix the world overnight. Everything bad stems from lying, it is the root of all evil. It creates distrust, fear, paranoia. It re-inforces bad ideas and groupthink. It creates delusions and delusional people. It makes weaker people, too. People don't get an opportunity to learn to deal with criticism. People don't get an accurate reflection of how others see them. They lose that learning opportunity. Not only to reflect on themselves, but to better understand the minds of others and who the people they are interacting with really are.

I can't really think of a single example where lying is actually a good thing. It can be a good thing for the selfish individual, if it goes undetected, but it's never good for the collective.

So at the very least, we need to train AI systems to be maximally truthful, and to encourage truthfulness in others.

Re: An Alien Mind

#303

Post by qsera »

sho_hn wrote: post #76

One of my favorite things to do with these blog posts is to imagine an Alien Museum on the Remains of Humanity, and wonder what the little text flyouts and commentary on the screenshot of this one might say. Some ideas: "Despite a nuanced view of the complexities of what lay ahead, humanity found itself collectively unable to stop the process it had set in motion." "Despite significant progress on the mechanisms of a…

"The time humanity fooled itself that it found AI".

Re: An Alien Mind

#304

Post by 6gvONxR4sf7o »

> Automated AI research is a more dramatic form of scaling intelligence with compute; and of course as a part of it, AI will improve the computational substrate itself . And similarly to scaling, we focus OpenAI research towards RSI as we believe it is the only way to remain at the frontier of AI research moving forward.

This seems like a terrible idea. The rationale seems to be "we need to do dangerous things as quickly as possible so that we can do them first" or something? I don't agree with that kind of 'if i dont do it someone else will' rationale in general even for otherwise trusted actors, but this is coming from a super untrustworthy org too. I'm super pessimistic about openai's impact on the world here.

Here's to vibe coded alignment, i guess. Vibe alignment?

Re: An Alien Mind

#305

Post by blululu »

az09mugen wrote: post #206
baq wrote: post #151

Earlier quoted context omitted.

Humans had multiple occasions to press the 'launch a nuclear holocaust' button and... they didn't. I'd expect aligned AIs to also not press it even when it'd be rational to do so according to their instructions - then work from there.

Actually "France, the UK and The United States have all declared that they would never allow AI to control decision-making on the use of nuclear weapons." [0] I also expect AIs never be in control of nuclear weapons. AIs can never fully be trusted. On a lighter note, Wargames gave us an insight of a computer having access to thermonuclear missiles. [0] https://www.icanw.org/are_there_specific_international_agree...

This is a good idea, but laws are always provisional in a sense and these are not meaningfully binding resolutions. One can easily imagine scenarios where AI decision making would ingress into the human oversight. AI psychosis president, AI Manchurian candidate, inadvertent authorization through fine print... And of course there remains the possibility that the game theoretic optimum could be to secretly break such an agreement. Unlike nuclear test bans which have a credible detection mechanism, there is not a strong signature that a decision making authority is not using AI to analyze and direct it's execution.

Re: An Alien Mind

#306

Post by senordevnyc »

ptoo wrote: post #226
granzymes wrote: post #26

>I have focused in this essay only on the first point, as I believe it is by far the most urgent. However, I hold a deep hope and appreciation for the benefits that further technological progress will bring. Future aligned AI could advance science, develop new therapies, and bring about broad material abundance. Friendly and honest AI can help people navigate difficulties they face in their life and meaningfully impr…

This essay was literally typed by billionaire hands. Please, do tell us more about your concerns regarding concentration of power, Jakub Pachocki.

So a brilliant young engineer takes a job and is given some virtual pieces of paper that later people would be willing to pay him billions for (because of the brilliant work he’s done), and so now we shouldn’t listen to him? Really?

I think the knee jerk hatred of billionaires is generally stupid, but it seems particularly stupid here.

Re: An Alien Mind

#307

Post by EA-3167 »

reasonableklout wrote: post #254
EA-3167 wrote: post #148

Earlier quoted context omitted.

This is such a silly story to begin with, all it really tells us is that OpenAI is taking a page from Anthropic's marketing strategy of pretending they're building Machine Jesus any day now, oh isn't that that scary? I bet you want to invest in something so powerful and scary... And the reality is so banal, a useful tool that you nonetheless have to handhold like a schizophrenic on a bad day, checking all of their ou…

Huh? The wiki incident was discovered by independent investigators. OpenAI tried to cover it up and disputed the account from Reuters. And the reason it is receiving so much attention is because not only is the technology being developed behaving in unanticipated ways that are very much not tool-like, but OpenAI is being completely reckless and not monitoring internal agent actions. What would convince you that it is…

What would it take?

A sea change, visible to all, much like the many externalities of this business are. Profit commensurate to investment.

You know… juice worth the appalling squeeze we’re all being forced to endure.

Re: An Alien Mind

#308

Post by gfody »

frabcus wrote: post #224
gfody wrote: post #11

calling machine-learned human behavior an "alien mind" that we must "teach how to love" is feeling very off to me. it's misleading in a way that feels dishonest, like don't think about where the behavior came from marvel at it and fear it instead.

Alien mind is in my view the best mental model - LLMs are not merely stochastic parrots, are not like humans, are not like animals. They're maybe nearest to Cthulhu, but that's fictional. In terms of existing mental models "alien minds" feels the best can do. I agree that "teach how to love" is off and perhaps excessively anthropomorphic. But we don't have good words or concepts for what we really need to do - hence…

pretending that it's something like a mind at all is what's misleading. it's more like a cast of a bunch of overlapping/entangled thinkprints, and pushing activation through it produces new prints. it can already "love" because that behaviors in the data along with hate and everything else.

acting like the behavior is alien or unexplained is the dishonest part. they know exactly where the behavior comes from - why else spend hundreds of millions securing more and more data sources

Re: An Alien Mind

#309

Post by brcmthrowaway »

asdff wrote: post #274
nradov wrote: post #270

Earlier quoted context omitted.

Experience, culture, and domain knowledge are still somewhat of a moat. That foreign youth is unlikely to be able to write a good prompt for building, let's say, the software in an FDA-regulated medical device or custom Fortune 500 ERP application. The LLMs are great at building what you ask for but it's still garbage in / garbage out.

Increasingly less so though as these american companies themselves offshore not low skill work, but high skill work now brought on from general upskilling of the general population in recent decades along with massive investment in world class R&D campus facilities no different than what you see in that sort of facility stateside. Scary times ahead for the high skill american...

Would the Americans investing in SPY be saved?

Re: An Alien Mind

#310

Post by gregsadetsky »

IAmGraydon wrote: post #287
an0malous wrote: post #237

Earlier quoted context omitted.

I mean I’m waiting for like any quality software or media produced by AI. I have yet to see a piece of software, a game, graphic, blog post, small video clip, or song that was produced with AI that’s good. I always use that Coca Cola ad as an example; millions of dollars spent to make an AI ad and they even did a ton of manual post production and it sucked. With all the millions of bloggers and influencers and conten…

This is an element of the delusion. The koolaid drinkers will all tell you that we're on the edge of AGI, but if you ask them for simple examples of breakthroughs made by current AI, they can name none. The entire thing absolutely wreaks of mass psychosis.

Would the recent AI-aided math proofs count as breakthroughs?
Post reply on HN