Live data from Hacker News

Why OpenAI's Structure Must Evolve to Advance Our Mission

openai.com

331–339 of 339 posts

Re: Why OpenAI's Structure Must Evolve to Advance Our Mission

#331
post #13

> A highly autonomous system that outperforms humans at most economically valuable work. That is not the real definition of AGI. The real "definition" can mean anything at this point and in their leaked report from the Information [0] they have defined it as "returning $100 billion or so in profits" In other words raise more money until they reach AGI. This non-profit conversion to for-profit is looking like a comple…

> "returning $100 billion or so in profits"

and do so without any human involvement

Re: Why OpenAI's Structure Must Evolve to Advance Our Mission

#333
post #106

Earlier quoted context omitted.

I think often company spin like this is more targeted towards internal employees than the outside world. Employees on the inside are always going to have a decent percentage of “true believers” who have cognitive dissonance if they don’t believe they’re making the world better. And so companies need to provide a narrative to keep that type of employee happy.

> I think often company spin like this is more targeted towards internal employees than the outside world. Probably is more ti employees than the general public, but it is even more targeted to the growing number of lawsuts against the conversion, since the charity nonprofit’s board is required to act in the interest of its charitable purpose even in a decision like this. It is directed at defeating the idea, express…

[dead]

Re: Why OpenAI's Structure Must Evolve to Advance Our Mission

#334

Earlier quoted context omitted.

I would guess they’re going to put as many expenses as possible on the nonprofit. For example, all the compute used for free tiers of ChatGPT will be charged to the nonprofit despite being a massive benefit to the for-profit. They may even charge the training costs, which will be in the billions, to the nonprofit as well

Why do this? They lose a deduction that way.

They’re doing it bc they’re obligated to keep the non profit part alive for PR purposes. They have to put money there so they may as well use it how they want

Re: Why OpenAI's Structure Must Evolve to Advance Our Mission

#335

Earlier quoted context omitted.

I want to put my opinion somewhere and as a reply to this seems like a good option. I am anti-capitalism and anti-profit and anti-Microsoft, but I can take a step back and recognize the truth to what’s being said here. They give the context that hundreds of billions are being spent on AI right now, and if OpenAI wants to remain competitive, they need significantly more money than they originally anticipated. They’re…

> They need money to stay competitive. The mission was never to stay competitive. The mission was to develop AGI. They need to be competitive if they want to make billions of dollars for themselves and their shareholders; they don't need to be competitive to develop AGI.

I meant competitive in terms of the roadmap to AGI, not in the marketplace. Feasible would have been a better word

Re: Why OpenAI's Structure Must Evolve to Advance Our Mission

#336
post #118

Earlier quoted context omitted.

The seasoned advice from experts is that AGI could very easily END humanity as a whole. But they need to pay their mortgage right now, and uhh... if we don't do it our competitors will. We're basically betting our species' future on these guys failing, because for a short period of time there's a massive amount of shareholder value to be made.

Isn't the idea that AGI could replace a bunch of labor allowing us to help more poor and increase net positive vibes or just have more leisure time? Obviously the way our economy and society are structured that's not what will happen, but I don't think that has much to do with tools and their tendency to increase our efficiency and output. Put another way, there are powerful benefits from AGI that we will squander be…

That is the idea being marketed by the companies, yes.

But AI does not come with glands, with brainstem instincts, with a limited physical strength, with childhood morality tales, with the ability to feel pain or empathy, with the desire for respect and public social approval, with the feeling of emotional connection with friends and family, with the feeling of helplessness and desire for freedom & variety, with the feeling of boredom and dislike of menial work, with the idealism of various religious and ethical mores.

It does not come packaged with any of the moral values or psychological quirks that constrain our behavior and allow us to work together. It is in fact, very difficult to teach these things to a machine - far more difficult than less subjective ideas like "Reason a way to do the thing that does the other thing". We do not have a mathematical proof of Correct Human Ethics, and we have reason to believe that this is a Difficult Problem.

It's arguably much easier to build a fully general AI that serves as what's termed a "paperclipper", which eats the world in pursuit of its task because we didn't formulate a reason for it not to do so and express it in the correct ten gigabyte matrix of floating point values, than to build a fully general AI which pursues what we actually implicitly want it to do for us in a way that the median eight year old would be able to follow.

The people studying AI alignment specifically were pessimistic fifteen years ago, and today the dominant stance is "Dying with dignity", give up the struggle to warn everybody and just try to enjoy what time you have.

Re: Why OpenAI's Structure Must Evolve to Advance Our Mission

#337
post #223

Earlier quoted context omitted.

Example: Covid does not hate you, nor does it love you, it simply follows an optimisation algorithm — that of genetic evolution — for the maximum reproductive success, and does so without regard to the damage it causes your body while it consumes you for parts. Covid is pretty stupid, it's just a virus. And yet: I've heard the mutation rate is 3.8 × 10e−6 / nucleotide / cycle, and at about 30,000 base pairs and 10e9…

I understand it how it applies to a virus, because a virus has a mechanism for action. In fact a virus is all mechanism for action; its behaviour is emergent from just being the surviving behaviour, rather than any grand strategizing. But for AI, they are all brains (or that's the claim) but no brawn. How do they act?

Computer viruses are already a thing.

Some of those viruses shut down hospitals, some damage uranium enrichment facilities, some mine or steal bitcoin, some steal normal money.

Polymorphic viruses have been around for decades, so it's not inconceivable that there's already some almost entirely evolved (simulated evolution is a kind of AI training mechanism) virus on the internet already, even before LLMs.

Right now, I'd expect an LLM to be fairly mediocre at writing a virus. But it doesn't need to be an LLM that does the writing, just as it doesn't need to be an LLM that is the payload once the infection has happened.

Re: Why OpenAI's Structure Must Evolve to Advance Our Mission

#338
post #53

I don't understand why they are spending so much time and effort trying to put a positive spin on this whole for-profit thing. No one is buying it. We all know what's going on. Just say "we want to make lots of money" and move on with your lives.

Because that is not their motivation and no amount of spin casting changes it.

There’s a simple counter-factual for people with this level of ability:

“Making money” can be easily pursued at BlackRock or any number of PE firms.

Creating historic technology, of unknown difficulty and scale of impact, is not something pure capitalism can or is designed to deliver. What would investors or customers buy at the start of OpenAI?

While OpenAI may end up a corporation like any other, it would not have achieved what it had as one.

Re: Why OpenAI's Structure Must Evolve to Advance Our Mission

#339
post #327

Earlier quoted context omitted.

> I'm pretty sure the difference is purely technical. It's really not - there is a ton of tax law that relies on e.g. the fair market value of hard-to-price assets or if all else fails and a penalty is due, there's an entire section of the CFR on how circumstances surrounding the underpayment can reduce or eliminate the liability. https://www.law.cornell.edu/cfr/text/26/1.6664-4 If you've only ever filed a personal t…

Waiving penalties depending on the circumstance is interesting but not directly relevant to the question on hand. You would have already been found to have evaded taxes before the penalty could be assessed. So the circumstance/motivation/explanation is still not the differentiator between avoidance and evasion in this case - just the penalty.

If tax law were black and white / “purely technical” they wouldn’t make accommodation to reduce your penalties if you made good faith errors when filing was my only point.
Post reply on HN