Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 details leaked?

threadreaderapp.com

Post reply on HN
Print view View original

271–280 of 648 posts

Re: GPT-4 details leaked?

#271

Post by JumpCrisscross »

ly3xqhl8g9 wrote: post #253
JumpCrisscross wrote: post #240

Earlier quoted context omitted.

> Most benefits from the copyright system are accrued to large corporations Citation fucking needed. Among those who study copyright and inequality, none suggest abandoning it [1][2]. Within the context of machine learning, one of the only pillars buttressing individuals against multi-trillion dollar corporations is copyright [3]. [1] https://journals.library.wustl.edu/lawreview/article/id/5108... [2] https://www.jst…

To nitpick, the second article is from 1970, back then a concept such as the Amazon Antitrust Paradox [1] would cause burning at the stake in the legal field. Similarly, we also have an Elsevier Antitrust Paradox, a Nature(.com) Antitrust Paradox, and so on. Just the fact that I cannot freely read an article published 53 years ago is beyond ludicrous. And also, if I pay JSTOR ($19.5/month) and read the 1970 article,…

> the fact that I cannot freely read an article published 53 years ago is beyond ludicrous

And has as much to do with individual versus corporate power as the health of a single tree can speak for a forest.

Nobody is saying the current situation is good or even sustainable. OP just made a big claim for which there is no evidence, despite many looking for it.

Re: GPT-4 details leaked?

#272

Post by classified »

theLiminator wrote: post #170
xeckr wrote: post #135

If this is true, then: 1. Training took 21 yottaflops. When was the last time you saw the yotta- prefix for anything? 2. The training cost of GPT-4 is now only 1/3 of what it was about a year ago. It is absolutely staggering how quickly the price of training an LLM is dropping, which is great news for open source. The google memo was right about the lack of a moat.

The real moat is an abundance of high quality data.

... stolen without regard for copyright and licensing.

Re: GPT-4 details leaked?

#273

Post by notahacker »

ly3xqhl8g9 wrote: post #263
notahacker wrote: post #255

Earlier quoted context omitted.

Ask someone on the screenwriters' strike if they'd rather copyright was eliminated so they didn't get paid at all...

Most of them already don't get paid, hence the strike [1]. [1] "Writers say they want a living wage as streaming devalues their work even as it demands more of their time.", https://www.indiewire.com/news/business/writers-strike-2023-...

No, they get paid but would, understandably, like to be paid more. Something which would be even less likely to happen if copyright was abolished so anybody could use their output at zero cost.

Re: GPT-4 details leaked?

#274

Post by weird-eye-issue »

YeGoblynQueenne wrote: post #239
xeckr wrote: post #135

If this is true, then: 1. Training took 21 yottaflops. When was the last time you saw the yotta- prefix for anything? 2. The training cost of GPT-4 is now only 1/3 of what it was about a year ago. It is absolutely staggering how quickly the price of training an LLM is dropping, which is great news for open source. The google memo was right about the lack of a moat.

>> The training cost of GPT-4 is now only 1/3 of what it was about a year ago. It is absolutely staggering how quickly the price of training an LLM is dropping, which is great news for open source. The google memo was right about the lack of a moat. That really doesn't change anything at all. The more training large models gets cheaper, the more large corporations are able to train larger models than everyone else. S…

> Yet, if I had a million dollars and you had a thousand dollars, I could still buy a thousand times more rice than you.

And?

Re: GPT-4 details leaked?

#276

Post by ben_w »

namaria wrote: post #267
ben_w wrote: post #236

Earlier quoted context omitted.

The public position (as opposed to the rumour mills) is that they're not working on a 5, and don't intend to at least until they can figure out how to do it safely.

And safely here means non disruptive to established businesses. If they create a tool so powerful that it can replace entire professional classes, that would just mean people can bypass the employers and get their value directly from an API. OpenAI needs to make sure they have a business model where they can charge enterprise fees and provide value to corporations, not directly to people. ChatGPT was merely a publici…

> And safely here means non disruptive to established businesses

Why would OpenAI care about that?

No, it means safety, as in not giving out dangerous answers that get people killed.

Re: GPT-4 details leaked?

#277

Post by bionhoward »

YeGoblynQueenne wrote: post #229
PostOnce wrote: post #19

"Open" AI, a charity to benefit us all by pushing and publishing the frontier of scientific knowledge. Nevermind, fuckers, actually it's just to take your jobs and make a few VCs richer. We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. https://github.com/ggerganov/llama.cpp https://github.com/openlm-research/open_llama https://huggingface.co/TheBlok…

>> We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. Just to be clear, there's no science being kept secret because there is no science being done. OpenAI's is a feat of engineering, borne aloft by a huge budget supporting a large team whose expertise lies in tuning neural net systems, and not in doing science. Machine learning, as it is practiced to…

Pretty sure the idea of scaling is under test here, just because it’s relatively straightforward to use larger models doesn’t make it less of an application of the scientific method. Also, process supervision, devops is experimental for a startup like OpenAI, and the mixture of experts really hasn’t been used with MHA models at scale before.

I’d argue it’s pedantic to assume all science has to be completely novel or revolutionary. More neurons good, is a perfectly reasonable reason to experiment. I think you’re being a bit “snooty” in setting such a high bar to gatekeep science…

Re: GPT-4 details leaked?

#278

Post by mhandley »

borissk wrote: post #153
vouaobrasil wrote: post #95

Earlier quoted context omitted.

I would love to see your rebuttals, especially since I have never seen any strong arguments in favour of AI being a net benefit to society, and I have thought and read about this at lenght. Of course, I always expect downvotes on my posts here since there is a strong tendency towards loving technology here. But what I find most interesting is that there is absolutely no taking of responsibility of any technological c…

If we were a rational civilization we'd stop all scientific research immediately. First there's a good chance the great filter is ahead of us and will be triggered by a technology break-trough. Second with nuclear weapons we got lucky in that it's extremely hard to separate fission capable isotope of uranium from mineral ores; if in the future we invent a powerful weapon that's easy to produce organizations like al-Q…

The great filter is definitely ahead of us if we stop all scientific research. Just ask the dinosaurs.

Re: GPT-4 details leaked?

#279

Post by namaria »

ben_w wrote: post #276
namaria wrote: post #267

Earlier quoted context omitted.

And safely here means non disruptive to established businesses. If they create a tool so powerful that it can replace entire professional classes, that would just mean people can bypass the employers and get their value directly from an API. OpenAI needs to make sure they have a business model where they can charge enterprise fees and provide value to corporations, not directly to people. ChatGPT was merely a publici…

> And safely here means non disruptive to established businesses Why would OpenAI care about that? No, it means safety, as in not giving out dangerous answers that get people killed.

Because they belong to Microsoft. Because they are funded by rich people who own established businesses. Because they are the privileged .01% who are invested in the status quo. Because their whole networks consist of people with entrenched business interests. Pick one or more. Why would you think rich American engineers are in general any more worried about the overall safety of the world then their own self interest? That's what they show day in day out with their decisions.

Re: GPT-4 details leaked?

#280

Post by PoignardAzur »

johnnyworker wrote: post #183
koheripbal wrote: post #137

Earlier quoted context omitted.

4-bit quantization removes a lot of the model's sophistication, and 60B parameters is still smaller than what GPT4 is using.

The point is that it's infinitely better in not being there "just to take your jobs and make a few VCs richer". Nobody even claimed it's more performant. It's like the difference getting nothing, but keeping your land, and getting glass pearls, losing your land. You have to completely ignore the meat of the argument to even pretend there is a contest. And this is without considering what happened if we stopped feedin…

That argument seems more political than practical.

If on one hand you have a tool that you can actually use to help with your job, and another that sounds like a very advanced chatbot but doesn't actually provide value, well the second tool being open-source doesn't change that it's doesn't provide value.

(Also, assuming that open-source tools aren't going to upend a ton of people's jobs seems really naive. These people aren't going to be any less bitter that their jobs are taken by freelance nerds instead of corporate nerds.)

Post reply on HN