Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 details leaked?

threadreaderapp.com

Post reply on HN
Print view View original

251–260 of 648 posts

Re: GPT-4 details leaked?

#251

Post by yyyk »

antupis wrote: post #92
KronisLV wrote: post #76

Earlier quoted context omitted.

> You can use the above without paying OpenAI. You don't even need a GPU. There are no license issues like with the facebook llama. I actually wrote about getting an LLM chatbot up and running a while ago: https://blog.kronis.dev/tutorials/self-hosting-an-ai-llm-cha... It's good that the technology and models are both available for free, and you don't even need a GPU for it. However, there are still large memory requ…

I bet that open models win in the end because porn. There is already very weird and vibrant community creating "waifus" and tinkering with these models.

Perhaps this could happen to make Stable Diffusion win against Midjourney/Adobe. LLMs however have more than enough commercial support to overcome weird communities, and these communities care less about text than images.

Re: GPT-4 details leaked?

#252

Post by mihaic »

Incipient wrote: post #227
ZiiS wrote: post #209

Earlier quoted context omitted.

Most of the ram is for storing the model once it is loaded it is read only so will not harm an SSD.

It only reads from memory,not swap directly. If it needs to read something from swap, it'll write out something from memory to swap, then read the swap into memory. Reading 1gb of swap, will essentially write 1gb to the ssd too. (rough numbers) Correct me if I misunderstand swap?

If the underlying data hasn't changed, the page isn't written to disk. CPUs do keep track of writes that mark a memory page as "dirty".

Re: GPT-4 details leaked?

#253

Post by ly3xqhl8g9 »

JumpCrisscross wrote: post #240
dwallin wrote: post #213

Earlier quoted context omitted.

Copyright does very little for individuals. Most benefits from the copyright system are accrued to large corporations.

> Most benefits from the copyright system are accrued to large corporations Citation fucking needed. Among those who study copyright and inequality, none suggest abandoning it [1][2]. Within the context of machine learning, one of the only pillars buttressing individuals against multi-trillion dollar corporations is copyright [3]. [1] https://journals.library.wustl.edu/lawreview/article/id/5108... [2] https://www.jst…

To nitpick, the second article is from 1970, back then a concept such as the Amazon Antitrust Paradox [1] would cause burning at the stake in the legal field. Similarly, we also have an Elsevier Antitrust Paradox, a Nature(.com) Antitrust Paradox, and so on.

Just the fact that I cannot freely read an article published 53 years ago is beyond ludicrous. And also, if I pay JSTOR ($19.5/month) and read the 1970 article, do you think the money will go to Stephen Breyer [2], the author? No, 100% of the $19.5/month will go to Ithaka Harbors, parent company of JSTOR [3], that's where the "accrued to large corporations" lies.

[1] 2017, https://www.yalelawjournal.org/note/amazons-antitrust-parado..., Lina M. Khan is the chairwoman of the Federal Trade Commission since 2021.

[2] https://en.wikipedia.org/wiki/Stephen_Breyer

[3] "Ithaka's total revenue was $105 million in 2019, most of it ($79 million) from JSTOR service fees", https://en.wikipedia.org/wiki/Ithaka_Harbors

Re: GPT-4 details leaked?

#254

Post by iraqmtpizza »

bayindirh wrote: post #176
rebolek wrote: post #147

Earlier quoted context omitted.

We had nuclear weapons for almost 80 years and the world still hasn't ended. And I think that nuclear weapons are way more dangerous than Markov chains on steroids.

I can't launch a tactical nuke because somebody wronged me, but can create a disinformation campaign with the tools I have and optionally 2-3 smart, motivated individuals, for free. Both can be equally devastating. Or, if I want to go the extra mile, I can use the latter to create motivation for the utilization of the former. e.g. I may say that a country has WMDs, and maybe try to manufacture consent for destruction…

>e.g. I may say that a country has WMDs, and maybe try to manufacture consent for destruction

too soon

Re: GPT-4 details leaked?

#255

Post by notahacker »

ly3xqhl8g9 wrote: post #152
BelleOfTheBall wrote: post #119

Earlier quoted context omitted.

That is a very dangerous way to approach copyright laws. They are definitely abused by corporations like Disney, infamously so, but abolishing them is absolutely not the answer. Art makers are already struggling en masse, taking away their ability to earn money off their work isn't an answer, especially if it's just to train a predictive text generator.

Already copyright laws do not offer protection for writers (see the current [screen]writers strike [1]), visual artists, musicians (see the recent Taylor Swift re-recording debacle [2]), not to speak of the "lower" arts, crafts & merch, where colossi such as H&M and Zara steal art, designs, concepts regularly [3] (not to even mention the sweatshops). Copyright laws, or in fact generally laws, are for the rich and pow…

Ask someone on the screenwriters' strike if they'd rather copyright was eliminated so they didn't get paid at all...

Re: GPT-4 details leaked?

#256

Post by ethanbond »

amelius wrote: post #245
YeGoblynQueenne wrote: post #229

Earlier quoted context omitted.

>> We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. Just to be clear, there's no science being kept secret because there is no science being done. OpenAI's is a feat of engineering, borne aloft by a huge budget supporting a large team whose expertise lies in tuning neural net systems, and not in doing science. Machine learning, as it is practiced to…

It's more a science than pure math or CS, since they are producing falsifiable models.

In what sense are they falsifiable?

Re: GPT-4 details leaked?

#257

Post by ethanbond »

ben_w wrote: post #236
wrasee wrote: post #228

Earlier quoted context omitted.

I don’t understand this. Won’t that hurt their progress on GPT-5?

The public position (as opposed to the rumour mills) is that they're not working on a 5, and don't intend to at least until they can figure out how to do it safely.

[or until someone else starts catching up]

The same dynamic that incentivized a mass rollout of these unaligned systems will be perfectly sufficient to incentivize mass rollout of stronger, also unaligned systems.

Re: GPT-4 details leaked?

#258

Post by DeathArrow »

jona-f wrote: post #167
wiz21c wrote: post #148

Earlier quoted context omitted.

What are all the researchers in universities doing ? Couldn't they improve these models (they do have big brains after all) with tax payer's money and put the results under some cool open source license...

Yes, they are doing the improving, but then you need loads of money to do the learning no university can afford. So now big tech is hiring promising university researchers for good money to scale up their research. This could be solved by massive decentralization where millions of users provide compute with their gpus and i think it will be at some point, cause i believe foss is more powerful than this openai bs. The…

> There are people working on this, but afaik the techniques aren't quite there. You need a different kind of model with much more parallelization then what is currently used.

What if crypto is switching from mindless hashing as proof-of-work to training AI models as proof-of-work? That would mean suddenly big computing resources are available.

Re: GPT-4 details leaked?

#259

Post by varispeed »

DeathArrow wrote: post #243
YeGoblynQueenne wrote: post #229

Earlier quoted context omitted.

>> We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. Just to be clear, there's no science being kept secret because there is no science being done. OpenAI's is a feat of engineering, borne aloft by a huge budget supporting a large team whose expertise lies in tuning neural net systems, and not in doing science. Machine learning, as it is practiced to…

Still, the guys developing ML algorithms like Neural nets, Linear regression, Decision tree, SVM, Naive Bayes, KNN etc., weren't doing science?

It's like asking whether a carpenter is a scientist because they developed a cabinet...

Re: GPT-4 details leaked?

#260

Post by thifhi »

YeGoblynQueenne wrote: post #239
xeckr wrote: post #135

If this is true, then: 1. Training took 21 yottaflops. When was the last time you saw the yotta- prefix for anything? 2. The training cost of GPT-4 is now only 1/3 of what it was about a year ago. It is absolutely staggering how quickly the price of training an LLM is dropping, which is great news for open source. The google memo was right about the lack of a moat.

>> The training cost of GPT-4 is now only 1/3 of what it was about a year ago. It is absolutely staggering how quickly the price of training an LLM is dropping, which is great news for open source. The google memo was right about the lack of a moat. That really doesn't change anything at all. The more training large models gets cheaper, the more large corporations are able to train larger models than everyone else. S…

Surely there are diminishing returns for the AI computing though? I mean, is a model with 10x the parameter count 10x better? I think it is still possible that the training costs will be irrelevant for all players at some point with this non-linear scale. Access to data is another story
Post reply on HN