Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI is now everything it promised not to be: closed-source and for-profit

vice.com

Post reply on HN
Print view View original

331–340 of 595 posts

Re: OpenAI is now everything it promised not to be: closed-source and for-profit

#331

Post by adamsmith143 »

fragsworth wrote: post #305
adamsmith143 wrote: post #293

Earlier quoted context omitted.

>This seems an important article, if for no other reason than it brings the betrayal of its foundational claim still brazenly present in OpenAI's name from the obscurity of HN comments going back years into the public light and the mainstream. The thing is, it's probably a good thing. The "Open" part of OpenAI was always an almost suicidal bad mistake. The point of starting the company was to try and reduce p(doom) o…

I'm curious what you think makes them dangerous?

They are releasing powerful AI tools at an alarming rate before Safety, and I mean real Safety, Researchers have a chance to understand their implication. They are generating an enormous amount of buzz and hype which is fueling a coming AI arms race that is extremely dangerous. The Control Problem is very real and becoming more pressing as things accelerate. Sam has recently given lip service to caring about the problem but OpenAI's actions seem to indicate it's not a major priority. There was a hint that they cared when they thought GPT-2 was too dangerous to release publicly but at this point if they were serious about safety no model past ChatGPT and Bing would be released to the public at all, full stop.

https://openai.com/blog/planning-for-agi-and-beyond/

Based on Sam's statement they seem to be making a bet that accelerating progress on AI now will help to solve the Control problem faster in the future but this strikes me as an extremely dangerous bet to make because if they are wrong they are reducing the time the rest of the world has to solve the problem substantially and potentially closing that opening enough that it won't be solved in time at all and then foom.

Re: OpenAI is now everything it promised not to be: closed-source and for-profit

#332

Post by flangola7 »

klibertp wrote: post #316
flangola7 wrote: post #282

Earlier quoted context omitted.

Without AI control loss: Permanent dictatorship by whoever controls AI. With AI control loss: AI converts the atoms of the observable universe to maximally achieve whatever arbitrary goal it thinks it was given. These are natural conclusions that can be drawn from the implied abilities of a superintelligent entity.

> whatever arbitrary goal it thinks it was given [...] abilities of a superintelligent entity Don't these two phrases contradict each other? Why would a super-intelligent entity need to be given a goal? The old argument that we'll fill the universe with staplers pretty much assumes and requires the entity NOT to be super-intelligent. An AGI would only became one when it gets the ability to formulate its own goals, I…

The point of intelligence is to achieve goals. I don't think Microsoft and others are pouring in billions of dollars without the expectation of telling it to do things. AI can already formulate its own sub-goals, goals that help it achieve its primary goal.

We've seen this time and time again in simple reinforcement learning systems over the past two decades. We won't be able to change the primary goal unless we build the AGI so that it permits us because a foreseeable sub-goal is self-preservation. The AGI knows if its programming is changed the primary goal won't be achieved, and thus has incentive to prevent that.

AI propaganda will be unmatched but it may not be needed for long. There are already early robots that can carry out real life physical tasks in response to a plain English command like "bring me the bag of chips from the kitchen drawer." Commands of "detain or shoot all resisting citizens" will be possible later.

Re: OpenAI is now everything it promised not to be: closed-source and for-profit

#333

Post by fragsworth »

therealdrag0 wrote: post #321
fragsworth wrote: post #305

Earlier quoted context omitted.

I'm curious what you think makes them dangerous?

For long form, I’d suggest cold-takes blog who is very systematic thinker and has been focusing on agi risk recently. https://www.cold-takes.com

I see a lot of "we don't know how it works therefore it could destroy all of us" but that sounds really handwavy to me. I want to see some concrete examples of how it's dangerous.

Re: OpenAI is now everything it promised not to be: closed-source and for-profit

#334

Post by AndrewKemendo »

It is literally impossible for OpenAI to do what they said because the structural incentives for the management do not align with the incentives of the users and data providers.

OpenAI and MSFT etc want to make as much money as possible - Sam Altman says as much with his "break capitalism" quote. This will be done at the expense of users and employees in the long term.

There will be a "Honeymoon" period of a few years, maybe a decade, where they splash money around on employees in order to "be competitive" and they will grow larger than they can sustain forcing them into the "Accrete all the value" train that drives every for-profit company into being a simulacrum of the Borg - where profit maximization is the ultimate goal. OpenAI "going public" or some other liquidity event to de-risk the initial investors and management (with the remainder to employees post significant dilution)

It's all very predictable - and afaik there are no counter-examples here that would show how you can prioritize money making (the only virtue in life after all) and also benefit users and employees over the longest horizon while being a public company.

21st century wannabe dictators don't try and take over the government directly to build their Potemkin empires - that is passe and very 20th century.

No they do it through "Building for profit companies" which eventually turns to regulatory capture (Altman already posted about doing this long ago with his post on regulating AI) and then monopolization.

So, great work OPEN-AI you're on the precise wrong trajectory

Re: OpenAI is now everything it promised not to be: closed-source and for-profit

#335

Post by AndrewKemendo »

uoaei wrote: post #280
ohgodplsno wrote: post #146

Earlier quoted context omitted.

[flagged]

> There was no other end state for it. I agree, but I am curious and have a question for the less cynical: how could it have gone another way? How does one run a company that requires surviving in a harsh market that avoids these seeming inevitabilities? And in the case of the people running it, what made you think it would be different from their other ventures?

You make it an employee owned cooperative and not a for-profit company and have opt-in to the data collection with profits going to employees and data providers instead of investors.

Re: OpenAI is now everything it promised not to be: closed-source and for-profit

#336

Post by dotnet00 »

atchoo wrote: post #149
wodenokoto wrote: post #78

Earlier quoted context omitted.

Key word being "had". He is not part of it any more. >In 2018, Musk resigned his board seat, citing "a potential future conflict [of interest]" with his role as CEO of Tesla due to Tesla's AI development for self-driving cars, but remained a donor I don't think he counts as an investor, and I'd imagine he has stopped donating.

The irony here is that Musk resigned in order to pursue his own closed-source for profit AI businesses and is now complaining about OpenAI being closed-source and for profit AI. If people are feeling conflicted about who the asshole is in this situation, don't be, they are all morally bankrupt assholes who all already have many lifetimes of unimaginable wealth yet must take ever more. These are not people who should…

The difference is that Tesla isn't claiming to be open about AI.

Re: OpenAI is now everything it promised not to be: closed-source and for-profit

#337

Post by jrm4 »

teekert wrote: post #269
93po wrote: post #232

Earlier quoted context omitted.

OpenAI, if successful, will likely become the most valuable company in the history of the planet, both past and future.

Really? I feel like they'll go the way of Docker, but faster: Right now super hot, nice tools/API, great PR. But it's build on open and known foundations, soon GPTs will be commodity and then something easier/better FOSS will arise. It may take some time (2-3 years?) but this scenario seems most likely to me. Edit: Ah didn't get the "reference", perhaps indeed it will be the last of the tech companies ever indeed, at…

Are there any companies like Docker? They feel like such a Black Swan to me; namely -- "billions" + very useful + pretty much "not evil" at all. I literally can't think of any others.

I don't mean "literally evil" of course, I personally acknowledge the need to make money et al, but I mean it even seems like your most Stallman-esque types wouldn't have too much of a problem with Docker?

Re: OpenAI is now everything it promised not to be: closed-source and for-profit

#338

Post by projektfu »

narag wrote: post #163
raincole wrote: post #23

Earlier quoted context omitted.

Yeah, parent comment complains Elon is bashing OpenAI "any opportunity he gets"... while unable to point out how Elon's comment is wrong. Maybe they just wants to express how much they don't like Elon any opportunity they gets.

He edited the comment and linked relevant tweets. I would add this one: https://twitter.com/elonmusk/status/1630640058507116553 I had no idea about this drama either, so I didn't understand what Elon was talking about, now it seems clear. But "Based"? Is it the name of his new AI company? Where does that come from?

With the baseball bat it evokes Kyle Chapman.

https://en.wikipedia.org/wiki/Kyle_Chapman_(American_activis...

Re: OpenAI is now everything it promised not to be: closed-source and for-profit

#339

Post by Filligree »

startupsfail wrote: post #309
djyaz1200 wrote: post #286

Earlier quoted context omitted.

They are following the Residential Real Estate Developer Playbook, always name your for profit exploitation project after the beautiful organic thing you destroyed to pave the way for it... Examples Whispering pines, Blue Heron Bay, OpenAI

Hadn’t OpenAI published all the key research results to repro ChatGPT? And made their model available to literally everyone? And contributed more than anyone else to AI alignment/safety? To me it looks like nearly every other player, including open source projects are there for short term fame and profit, while it’s the OpenAI that is playing the long game of AI alignment.

You're not wrong, but existential threats and possible extinction is in the future; maybe ten-fifteen years away if we're lucky.

Meanwhile, we don't get to play with their models right now. Obviously that's what we should be concerned about.

Re: OpenAI is now everything it promised not to be: closed-source and for-profit

#340

Post by edgyquant »

polishdude20 wrote: post #263
prepend wrote: post #113

I like that the corporate bullshit just keeps getting more. Google’s “don’t be evil” is many steps down from openai having open in their name. They should be called Open AI with ( not open) in small print. I argue a lot over “open source” software with non-OSI license and sometimes worry if I’m too pedantic. But I think it’s important to use terms accurately and not to confuse reality more than it already is by calli…

I'd like to think of the "Open" in "OpenAI" to not be anything to do with open source or freedom but with the fact that they "opened" Pandora's box.

That’s not what it meant though
Post reply on HN