Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI and Elon Musk

openai.com

Post reply on HN
Print view View original

971–980 of 1001 posts

Re: OpenAI and Elon Musk

#971

Post by edgyquant »

qarl wrote: post #928
smoldesu wrote: post #892

Earlier quoted context omitted.

> Working in this space (however you want to say it) costs a lot of money. Not necessarily? OpenAI deliberately chooses to scale their models vertically, which incumbents like Google and Meta had largely avoided. Their "big innovation" was GPT-4, an LLM of unprecedented scale that brute-forced it's way to the top. The Open Source 72b+ models of today generally give it a run for it's money, the party trick is over. Ni…

> Not necessarily? Well then, I expect we'll see a bunch of small projects that beat the big players any day now. I won't hold my breath.

Again you haven’t a clue of the point here. They were created as a research lab and instead after taking a paper created at Google they went closed source and are simply scaling that instead of working to create new and improved models that can actually run on reasonable hardware.

Small transformers being able to beat the same models but scaled up is unrelated to anything being discussed and you just seem like a fanboy at this point

Re: OpenAI and Elon Musk

#972

Post by remram »

vwkd wrote: post #965
habosa wrote: post #716

Regardless of who is right here, I think the enormous egos at play make OpenAI the last company I’d want to develop AGI. First Sam vs. the board, now Elon vs. everyone … it’s pretty clear this company is not in any way being run for the greater good. It’s all ego and money with some good science trapped underneath.

Who runs a company “for the greater good”? Surely, anyone taking on and enduring the pain of running a company does so for egoistic reasons. Your implicit assumption is that altruism exists. In the limit, every living being is egoistic. Anything you do is ultimately for egoistic reasons - even if you do it “for others” at first sight, it ultimately benefits you in some way, even only to make you feel better. A common…

It's not supposed to be a company, it is supposed to be a non-profit organization with an altruistic goal.

If it's not possible maybe they shouldn't have set it up this way, but they did and here they are.

Re: OpenAI and Elon Musk

#973

Post by ClarityJones »

theptip wrote: post #906
ClarityJones wrote: post #806

Earlier quoted context omitted.

Would you please try to explain why ?

This would be orders of magnitude more than any charity has ever raised, and is also an uncommonly huge raise even among _for-profit_ companies where investors expect returns. Even when the EA community was flush with crypto billionaires there was no appetite for this level of spend.

> more than any charity has ever raised

The Novo Nordisk foundation has a $120 Billion endowment, the Bill & Melinda Gates Foundation has $50 Billion, the Welcome Trust has $42 Billion, ... There are about 19 charities that still have more than $10 Billion, let alone those who have raised (and spent) that during their existence.

https://en.wikipedia.org/wiki/List_of_wealthiest_charitable_...

> an uncommonly huge raise even among _for-profit_ companies where investors expect returns.

So, offer a return on investment. Charities that take loans pay interest. Charities that hire staff pay salaries. What's with this idea that charities cannot pay a reasonable market rate for services (e.g. short-term funding)?

Re: OpenAI and Elon Musk

#974

Post by qarl »

edgyquant wrote: post #971
qarl wrote: post #928

Earlier quoted context omitted.

> Not necessarily? Well then, I expect we'll see a bunch of small projects that beat the big players any day now. I won't hold my breath.

Again you haven’t a clue of the point here. They were created as a research lab and instead after taking a paper created at Google they went closed source and are simply scaling that instead of working to create new and improved models that can actually run on reasonable hardware. Small transformers being able to beat the same models but scaled up is unrelated to anything being discussed and you just seem like a fanb…

> Again you haven’t a clue of the point here

I guess we're just going to have to agree to disagree.

Re: OpenAI and Elon Musk

#975

Post by Judgmentality »

hef19898 wrote: post #960
bbor wrote: post #897

Earlier quoted context omitted.

In this note: HIGHLY recommend “Rigor of Angels”, which (in part) details Heisenbergs life and his moral qualms about building a bomb. He just wanted to be left alone and perfect his science, and it’s really interesting to see how such a laudable motivation can be turned to such deplorable, unforgivable (IMO) ends. Long story short they claim they thought the bomb was impossible, but it was still a large matter of co…

Heisenberg makes you think, doesn't he? As the developer of Hitler's bomb, which never was a realistic thing to begin with, he never employed slave labour for example. Nor was any of his stuff used during warfare. And still, he is seen by some as some tragic figure, at worst as man behind Hitler's bomb. Wernher vin Braun on the other hand got lauded for his contribution to space exploration. His development of the V2…

To be reductionist - history is written by the victors.

https://www.smbc-comics.com/comic/status-2

Re: OpenAI and Elon Musk

#976

Post by smoldesu »

qarl wrote: post #928
smoldesu wrote: post #892

Earlier quoted context omitted.

> Working in this space (however you want to say it) costs a lot of money. Not necessarily? OpenAI deliberately chooses to scale their models vertically, which incumbents like Google and Meta had largely avoided. Their "big innovation" was GPT-4, an LLM of unprecedented scale that brute-forced it's way to the top. The Open Source 72b+ models of today generally give it a run for it's money, the party trick is over. Ni…

> Not necessarily? Well then, I expect we'll see a bunch of small projects that beat the big players any day now. I won't hold my breath.

You're basically describing what happened to GPT-2 when T5-flan came out. Not to mention, the incumbent model at the time (BERT) was extremely competitive for it's smaller size. Pretty much nobody was talking about OpenAI's models back then because they were too large to be realistically useful.

So yeah, I do actually anticipate smaller projects that devalue turtle-tower model scaling.

Re: OpenAI and Elon Musk

#977

Post by qarl »

smoldesu wrote: post #976
qarl wrote: post #928

Earlier quoted context omitted.

> Not necessarily? Well then, I expect we'll see a bunch of small projects that beat the big players any day now. I won't hold my breath.

You're basically describing what happened to GPT-2 when T5-flan came out. Not to mention, the incumbent model at the time (BERT) was extremely competitive for it's smaller size. Pretty much nobody was talking about OpenAI's models back then because they were too large to be realistically useful. So yeah, I do actually anticipate smaller projects that devalue turtle-tower model scaling.

Cool. Send me a note when that happens.

Re: OpenAI and Elon Musk

#978

Post by enriquec »

hef19898 wrote: post #950
WalterBright wrote: post #942

Earlier quoted context omitted.

> 4 of those 5 countries have significant welfare programs Welfare does not correspond with prosperity. It's hard to see how paying people to not work contributes to prosperity.

So, you are saying it would be good to have people working? Kind of like, I don't know, propose a programm or policy to very strongly motivate them, for example like a law, to take essentail jobs? I don't know, maybe during wartime to meet production goals? Just asking, because FDR wanted exactly that in his free market economy only produxing for the war effort to goverment set gials and prices in 1945. Welfare progr…

The welfare administrators pay no price for being wrong and thus the waste, fraud, and abuse has collapsed them all, throughout all of history.

> Welfare programs mean that life expectabcy is higher, as is quality of life. The US for examole is loosong on both metrics against Europe and ither parts of the world with solid programs. Germanys ascent to economic hight can be in part be traced back to Soziale Marktwirtschaft, combining very solid welfare and social.programs with a free market economy.

You're just saying this when all the evidence is to the contrary. Is Ukraine depending on Germany or the US? (Rhetorical.)

Re: OpenAI and Elon Musk

#979

Post by ActorNightly »

initplus wrote: post #577
ActorNightly wrote: post #471

Earlier quoted context omitted.

>They're probably right that building AGI will require a ton of computational power, and that it will be very expensive Eh. Humans have about 1.5 * 10 ^ 14 synapses (i.e connections between neurons). Assume all the synapses are firing (highly unlikely to be the case in reality), and the average firing speed is 0.05ms (there are chemical synapses that are much slower, but we take the fastest speed of the electrical sy…

Inference compute costs and training compute costs aren’t the same. Training costs are an order of magnitude higher.

Training will be significantly cheaper and take less time once we have the correct software for the 4090s

Right now, the idea is that every time you build an LLM, you start the training from scratch, because thats all we know how to do.

Human like AI will most definitely not be trained like that. Humans can look at a piece of information once or twice and remember it.

Just like the attention paper, at some point someone will publish a paper that describes a methodology of feeding one piece of data back through the network only a few times to fully train it.

Re: OpenAI and Elon Musk

#980

Post by codexb »

nebula8804 wrote: post #910
codexb wrote: post #775

Earlier quoted context omitted.

Yes and no. Elon has ego, but I also take him at his word when he says he wants to open source AI. He did the same thing with Tesla's patents.

Didn't the stipulation for the patents require other automakers to share their patents as a condition of using Tesla patents? "I'd like some of your turkey please and in exchange I will offer this half eaten chicken bone."

You mean like the GPL open source license?
Post reply on HN