Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 details leaked?

threadreaderapp.com

Post reply on HN
Print view View original

391–400 of 648 posts

Re: GPT-4 details leaked?

#391

Post by UncleEntity »

weird-eye-issue wrote: post #274
YeGoblynQueenne wrote: post #239

Earlier quoted context omitted.

>> The training cost of GPT-4 is now only 1/3 of what it was about a year ago. It is absolutely staggering how quickly the price of training an LLM is dropping, which is great news for open source. The google memo was right about the lack of a moat. That really doesn't change anything at all. The more training large models gets cheaper, the more large corporations are able to train larger models than everyone else. S…

> Yet, if I had a million dollars and you had a thousand dollars, I could still buy a thousand times more rice than you. And?

And...

...the market for rice will totally collapse because it would cost more to transport it than the farmer would make by selling it. Feel free to substitute "rice" for whatever commodity which becomes "too cheap to meter".

The "invisible hand" has a tendency to bitchslap people who don't have an even modest understanding of economic principles.

Re: GPT-4 details leaked?

#392

Post by blitz_skull »

0xDEF wrote: post #328
arthur_sav wrote: post #241

Earlier quoted context omitted.

I would love to use these... but they suck. They don't even come close to what OpenAI offers.

I think most of HN has only tried ChatGPT with GPT-3.5 in December 2022 when the OpenAI servers were getting hammered. That is why they're impressed by these slow and low quality local models and mistakenly think they're on par with OpenAI's offerings. Honestly even reddit and teenagers on TikTok have a more accurate view on OpenAI vs. local LLMs than HN.

There’s not a single thing out there that even comes close to GPT-4. Not one that I’ve used anyway. Benchmarks be damned, it’s the experience that matters and I’ve yet to have an LLM blow my mind the way GPT-4 does.

Re: GPT-4 details leaked?

#393

Post by captainbland »

ben_w wrote: post #236
wrasee wrote: post #228

Earlier quoted context omitted.

I don’t understand this. Won’t that hurt their progress on GPT-5?

The public position (as opposed to the rumour mills) is that they're not working on a 5, and don't intend to at least until they can figure out how to do it safely.

I think the reality is that training a "GPT-5" which is as much of a bump as GPT-4 was over GPT-3 or that was over GPT-2 would cost in the billions of dollars using today's hardware so they're choosing to wait it out until the risk of them screwing it up doesn't cost as much as a state of the art microfabrication plant.

Re: GPT-4 details leaked?

#394

Post by mdp2021 »

criley2 wrote: post #372
mensetmanusman wrote: post #319

Earlier quoted context omitted.

{Hypothesis, test, loop} is the scientific method, and I can guarantee it is being used when fine tuning an LLM.

It is not the scientific method at all. You didn't even include an analyze or a publish step, two critical components to the scientific method. The point is that science is a thoughtful, methodical, recorded, repeatable process that is scrutinized by not just your friends but by the entire world, including those who compete against you. In science, it's normal for your competitor to recreate your experiment exactly t…

Do not forget that with ANNs we are still dealing with technologies with obscure sides: function approximators where the original function, and what it means, is oftentimes not reconstructed.

Replication of the experiment, "verification" of the law, can still be within a largely preliminary side - of the steps that describe behavioural details, patterns over "protocols", without the explanatory side (that which cause understanding, e.g. enabling deduction).

Re: GPT-4 details leaked?

#395

Post by steve76 »

PoignardAzur wrote: post #280
johnnyworker wrote: post #183

Earlier quoted context omitted.

The point is that it's infinitely better in not being there "just to take your jobs and make a few VCs richer". Nobody even claimed it's more performant. It's like the difference getting nothing, but keeping your land, and getting glass pearls, losing your land. You have to completely ignore the meat of the argument to even pretend there is a contest. And this is without considering what happened if we stopped feedin…

That argument seems more political than practical. If on one hand you have a tool that you can actually use to help with your job, and another that sounds like a very advanced chatbot but doesn't actually provide value, well the second tool being open-source doesn't change that it's doesn't provide value . (Also, assuming that open-source tools aren't going to upend a ton of people's jobs seems really naive. These pe…

[dead]

Re: GPT-4 details leaked?

#396

Post by tensor »

DeathArrow wrote: post #243
YeGoblynQueenne wrote: post #229

Earlier quoted context omitted.

>> We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. Just to be clear, there's no science being kept secret because there is no science being done. OpenAI's is a feat of engineering, borne aloft by a huge budget supporting a large team whose expertise lies in tuning neural net systems, and not in doing science. Machine learning, as it is practiced to…

Still, the guys developing ML algorithms like Neural nets, Linear regression, Decision tree, SVM, Naive Bayes, KNN etc., weren't doing science?

There is actually an old argument about whether Computer Science is math or science. The lack of hypotheses and physicality drives the "it's math" camp. I tend to fall into that camp. Most computer science is either math or applied math.

That said, most of machine learning is not just engineering. If anything, there is little engineering done at all.

Re: GPT-4 details leaked?

#397

Post by blitz_skull »

BoorishBears wrote: post #50
PostOnce wrote: post #38

Earlier quoted context omitted.

It's several reasons, first it's the lies and the abuse of a charity, that wouldn't be an issue if they had began as a private company instead of robbing a charity. But secondly, even if they were a private company, it's dishonest and reprehensible to claim to congress that you want to "protect the public" when you really only give a shit about protecting your moat, I'm not happy about that either. I'm also tired of…

Did you actually read my comment or did you have a preplanned diatribe? They didn't start the private company until the person who promised them $1B reneged. That person reneged because they tried to forcefully takeover and were rebuked. They were running out of money and forced to raise funds in a very for-profit way or fold. - Edit: Rate limited because the hivemind has decided that it's unacceptable to insult thei…

Lol, I’ve never heard a single person suggest that we’d be living in some LLM utopia if OpenAI was gone.

Who have you been talking to?

Re: GPT-4 details leaked?

#398

Post by blitz_skull »

Brometheus wrote: post #195
astrange wrote: post #159

Earlier quoted context omitted.

I recommend reading comments like this and substituting "a baby" for AI. A baby also can't be aligned and is capable of deciding to destroy the world. It's not gonna do it though.

A baby doesn't output propaganda at 6GB/s in computer readable text.

Not with that attitude! /s

Re: GPT-4 details leaked?

#399

Post by ly3xqhl8g9 »

notahacker wrote: post #348
ly3xqhl8g9 wrote: post #340

Earlier quoted context omitted.

If it's genuinely news to you that exploitation is a normal day-to-day part of any industry, perhaps these pointless replies should end here. Yes, writers have been working for free [1]. [1] "In October 2015, Wil Wheaton created a stir when he declared that he had turned down an offer to write for the Huffington Post. He refused, according to him, because they had declined to pay for his work, in keeping with their p…

Strangely, it is not news to me that exploitation is a normal day-to-day part of any industry . (This is why I have not made any statements to that effect, never mind done anything as ludicrous as post articles about unionised screenwriters seeking to negotiate a higher pay rate as evidence that "most" of them earned "nothing") And this is also why I am not advocating a copyright-free world in which HuffPo has the ri…

Hope at least you are getting paid and you don't deploy such ridiculous amounts of bad faith for free.

My reply from above "Most of them already don't get paid, hence the strike" was specifically in the context of the streaming platforms not paying residuals in the same manner the studios do, hence the reference to the article.

What I find most funny is that you don't even know or care about alternative solutions, you just assume the copyright laws is for the best in the best of all possible worlds [1] and anything else would be chaos.

[1] Tout est pour le mieux dans le meilleur des mondes possibles, https://en.wikipedia.org/wiki/Candide

Re: GPT-4 details leaked?

#400

Post by fallingknife »

immibis wrote: post #370
lolinder wrote: post #354

Earlier quoted context omitted.

Just a reminder that LLaMA is not open—in order to use it legally you have to agree to Meta's terms, which currently means research use only. The versions circulating on torrents are essential pirated, and while I don't have an ethical problem with that at all you can't use it safely in a business. The open replacements for LLaMA have yet to reach 30B, let alone 65B.

If anyone has a copyright claim to an LLM, the creators of the input data have more of a copyright claim than the company that trained it. There's a good chance they are not copyrightable at all. I'd bet there's a lot of people willing to take on that risk. However, they might still fall under trade secret law.

Why would an LLM be any less copyrightable than any other piece of software?
Post reply on HN