Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 details leaked?

threadreaderapp.com

Post reply on HN
Print view View original

401–410 of 648 posts

Re: GPT-4 details leaked?

#401

Post by carabiner »

vouaobrasil wrote: post #80
carabiner wrote: post #59

Earlier quoted context omitted.

We must destroy OpenAI!

Not sure if you're being sarcastic but I agree. Humans should not have AI research or advanced AI at all. It (a) removes purpose from people, (b) presents a situation that is too alien for human minds to handle, (c) increases the addictiveness of technology and thereby pushes us further into growing the technological system, (d) crosses the "adaptability threshhold", i.e. the point at which the PACE of technological…

Serious as a heart attack. I hope that we don't have to resort to military force, but it may be the only option: https://futurism.com/ai-expert-bomb-datacenters

Re: GPT-4 details leaked?

#402

Post by hattmall »

blitz_skull wrote: post #388
criley2 wrote: post #372

Earlier quoted context omitted.

It is not the scientific method at all. You didn't even include an analyze or a publish step, two critical components to the scientific method. The point is that science is a thoughtful, methodical, recorded, repeatable process that is scrutinized by not just your friends but by the entire world, including those who compete against you. In science, it's normal for your competitor to recreate your experiment exactly t…

Umm.. no. I’m pretty sure when science was “invented” (whatever that means), there was no way to share your research with “the world”. You wrote it down in a book in you cared to reproduce it. Stop gate-keeping science. Observe, Predict, Test, Repeat. That is science. Kids do science when they watch adults do things. Just because you aren’t sitting in a lab publishing things no one reads doesn’t mean you can’t “do sc…

Just simply learning something isn't "doing science." The scientific method is a natural learning process it's not the only thing that defines science. This is incredibly light gate keeping. It's just maintaining a basic professional definition.

Re: GPT-4 details leaked?

#403

Post by paxys »

YeGoblynQueenne wrote: post #239
xeckr wrote: post #135

If this is true, then: 1. Training took 21 yottaflops. When was the last time you saw the yotta- prefix for anything? 2. The training cost of GPT-4 is now only 1/3 of what it was about a year ago. It is absolutely staggering how quickly the price of training an LLM is dropping, which is great news for open source. The google memo was right about the lack of a moat.

>> The training cost of GPT-4 is now only 1/3 of what it was about a year ago. It is absolutely staggering how quickly the price of training an LLM is dropping, which is great news for open source. The google memo was right about the lack of a moat. That really doesn't change anything at all. The more training large models gets cheaper, the more large corporations are able to train larger models than everyone else. S…

> if I had a million dollars and you had a thousand dollars, I could still buy a thousand times more rice than you.

And all that rice would be useless since you could only eat one cup a day.

The richest person in the world and someone who is solidly middle class both use the exact same iPhone. After a point more dollars doesn't necessarily mean better or more useful technology. If training "good enough" models becomes cheap enough to be achievable by small-time developers then OpenAI/Google/Anthropic etc. will definitely lose some of their edge in the space.

Re: GPT-4 details leaked?

#404

Post by renlo »

londons_explore wrote: post #367
jph00 wrote: post #9

Earlier quoted context omitted.

The previous posts are to a twitter thread that's been taken down, and the preview of a post that requires a $1000 subscription. This post however is freely available (for now at least).

And the tweeter of the twitter thread paid the $1000, copied the useful info to twitter, and then did a credit card chargeback.

Seems he summarized it and didn't copy it

Re: GPT-4 details leaked?

#405

Post by Xeoncross »

immibis wrote: post #382
Xeoncross wrote: post #359

Earlier quoted context omitted.

> pressure the government into making it Not really, Capitalism is free-trade between two parties. The more government involvement you have the more it moves towards socialism or communism where the government controls trade. At least this was the historical meaning. These days Capitalism is being redefined to mean private (non-government) Communism. That is, power concentrated in the hands of a few.

Capitalism is not free trade between two parties. That's free market. Capitalism is control by capital.

There is no free market without Capitalism. Capitalism is where the private citizens controls the capital. Not the government.

It's not possible to have a free market without competitive markets, price systems, private property, property rights recognition, and voluntary exchange.

Re: GPT-4 details leaked?

#406

Post by idopmstuff »

krackers wrote: post #17
CSMastermind wrote: post #2

Previously posted about here: https://news.ycombinator.com/item?id=36671588 and here: https://news.ycombinator.com/item?id=36674905 With the original source being: https://www.semianalysis.com/p/gpt-4-architecture-infrastruc... The twitter guy seems to just be paraphrasing the actual blog post? That's presumably why the tweets are now deleted. --- The fact that they're using MoE was news to me and very interesting. I…

Interesting on a meta point that the more clickbaity title "GPT-4 details leaked" won out over the more dispassionate but drier "GPT-4 Architecture, Infrastructure, Training Dataset, Costs".

I think in this case it's a better title in terms of highlighting the relevant info. What's really interesting is the source - these are important details that actually came from within OpenAI. Because the latter title emphasizes the type of info but not where it came from, I'd probably assume that's a blog post with some industry expert speculating about those things.

Re: GPT-4 details leaked?

#407

Post by rurp »

> The conspiracy theory that the new GPT-4 quality had been deteriorated might be simply because they are letting the oracle model accept lower probability sequences from the speculative decoding model.

Whether or not this specific theory is true something along these lines seems like the most likely explanation for the quality degradation that many have noticed; where OpenAI's claims about not changing the model are both technically true and conpletely misleading.

Re: GPT-4 details leaked?

#408

Post by [deleted] »

TradingPlaces wrote: post #336
PostOnce wrote: post #19

"Open" AI, a charity to benefit us all by pushing and publishing the frontier of scientific knowledge. Nevermind, fuckers, actually it's just to take your jobs and make a few VCs richer. We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. https://github.com/ggerganov/llama.cpp https://github.com/openlm-research/open_llama https://huggingface.co/TheBlok…

Keep in mind, that was the idea originally. Then in 2018 Elon decided he wanted to be CEO, the board rejected that idea for now obvious reasons, and he reneged on 90% of a promised $1b donation. The only way forward was to become a for-profit company and do a normal funding round, which is what happened with Microsoft. Elon’s rug pull is why this happened.

[deleted]

Re: GPT-4 details leaked?

#409

Post by mdp2021 »

DeathArrow wrote: post #243
YeGoblynQueenne wrote: post #229

Earlier quoted context omitted.

>> We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. Just to be clear, there's no science being kept secret because there is no science being done. OpenAI's is a feat of engineering, borne aloft by a huge budget supporting a large team whose expertise lies in tuning neural net systems, and not in doing science. Machine learning, as it is practiced to…

Still, the guys developing ML algorithms like Neural nets, Linear regression, Decision tree, SVM, Naive Bayes, KNN etc., weren't doing science?

The note is interesting and in a way a time taking puzzle,

but do not forget the apparent point in the original poster's is the lacking approach below the implementations of above said techniques.

The techniques you listed in a substantial way expanded our knowledge ("oh, we can do this ... and solve more problems") while raising little more questions.

The said implementations on topic give results but raise many more unanswered questions.

(This post is unfortunately an "immature" reply - just a provisional tentative point. The matter of "ML algorithms ∩ science" requires more time and concentration and consultation of the texts of the Great Ones.)

Re: GPT-4 details leaked?

#410

Post by jeron »

YeGoblynQueenne wrote: post #229
PostOnce wrote: post #19

"Open" AI, a charity to benefit us all by pushing and publishing the frontier of scientific knowledge. Nevermind, fuckers, actually it's just to take your jobs and make a few VCs richer. We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. https://github.com/ggerganov/llama.cpp https://github.com/openlm-research/open_llama https://huggingface.co/TheBlok…

>> We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. Just to be clear, there's no science being kept secret because there is no science being done. OpenAI's is a feat of engineering, borne aloft by a huge budget supporting a large team whose expertise lies in tuning neural net systems, and not in doing science. Machine learning, as it is practiced to…

> racing cool-looking sports cars.

I like the analogy attempt but there's a lot of science behind Formula 1

Post reply on HN