Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 details leaked?

threadreaderapp.com

Post reply on HN
Print view View original

151–160 of 648 posts

Re: GPT-4 details leaked?

#151

Post by swyx »

behnamoh wrote: post #26
krackers wrote: post #17

Earlier quoted context omitted.

Interesting on a meta point that the more clickbaity title "GPT-4 details leaked" won out over the more dispassionate but drier "GPT-4 Architecture, Infrastructure, Training Dataset, Costs".

Clickbait has its time and place. Despite my hatred towards it, sometimes it's really needed.

is it needed when you pay for the blogpost and then immediately chargeback the card like this dude did? https://twitter.com/untitled01ipynb/status/16786550120150712...

what a colossal asshole

Re: GPT-4 details leaked?

#152

Post by ly3xqhl8g9 »

BelleOfTheBall wrote: post #119
ly3xqhl8g9 wrote: post #115

Earlier quoted context omitted.

From the post: [Rumors that start to become lawsuits] Some speculations are: - LibGen (4M+ books) - Sci-Hub (80M+ papers) - All of GitHub This is the most funny, but in the end sad aspect. If ChatGPT was indeed trained on pirated content and is able to be(come) such a powerful tool, then the copyright laws should have been abolished yesterday. If ChatGPT was not trained on all these resources out there, then think ho…

That is a very dangerous way to approach copyright laws. They are definitely abused by corporations like Disney, infamously so, but abolishing them is absolutely not the answer. Art makers are already struggling en masse, taking away their ability to earn money off their work isn't an answer, especially if it's just to train a predictive text generator.

Already copyright laws do not offer protection for writers (see the current [screen]writers strike [1]), visual artists, musicians (see the recent Taylor Swift re-recording debacle [2]), not to speak of the "lower" arts, crafts & merch, where colossi such as H&M and Zara steal art, designs, concepts regularly [3] (not to even mention the sweatshops).

Copyright laws, or in fact generally laws, are for the rich and powerful, perhaps even for the corporations, not the individuals, since they paid for them to be so [4], it's just us, individually, the hoi polloi, who are still trapped into believing that there is rhyme or reason to our current system.

The joke is that while our system extolls itself as the most efficient system in history, based on competition of equals and free markets (two contradictions in terms contradicting each other), it harbours terrible inefficiencies such as the copyright laws, the patent system, and so on.

[1] https://en.wikipedia.org/wiki/2023_Writers_Guild_of_America_...

[2] https://www.vox.com/culture/22278732/taylor-swift-re-recordi...

[3] https://www.boredpanda.com/zara-stealing-designs-copying-ind...

[4] The Power Corporations Have In Changing Laws, https://www.npr.org/2021/04/02/983925056/the-power-corporati...

Re: GPT-4 details leaked?

#153

Post by borissk »

vouaobrasil wrote: post #95
M3L0NM4N wrote: post #87

Earlier quoted context omitted.

This is such a pessimistic view to take that I cannot even begin to describe how you are wrong.

I would love to see your rebuttals, especially since I have never seen any strong arguments in favour of AI being a net benefit to society, and I have thought and read about this at lenght. Of course, I always expect downvotes on my posts here since there is a strong tendency towards loving technology here. But what I find most interesting is that there is absolutely no taking of responsibility of any technological c…

If we were a rational civilization we'd stop all scientific research immediately. First there's a good chance the great filter is ahead of us and will be triggered by a technology break-trough. Second with nuclear weapons we got lucky in that it's extremely hard to separate fission capable isotope of uranium from mineral ores; if in the future we invent a powerful weapon that's easy to produce organizations like al-Qaeda will destroy every city on Earth.

Re: GPT-4 details leaked?

#155

Post by dr_dshiv »

wiz21c wrote: post #148
kyledrake wrote: post #65

Earlier quoted context omitted.

Unfortunately I've found the current OSS models to be vastly inferior to the OpenAI models. Would love to see someone actually get close to what they can do with GPT-3.5/4, except capable of running on commodity GPUs. What's the most impressive open model so far?

What are all the researchers in universities doing ? Couldn't they improve these models (they do have big brains after all) with tax payer's money and put the results under some cool open source license...

They are busily working on all the other large scale engineering projects that they are so good at producing and maintaining.

Re: GPT-4 details leaked?

#156

Post by holoduke »

behnamoh wrote: post #27
potatoman22 wrote: post #11

Google has been doing research into mixture of experts for scaling LLMs. Their GLaM model published in 2022 has 1.7 trillion parameters and 64 experts. https://icml.cc/media/icml-2022/Slides/17378.pdf

Google is jokingly behind in terms of LLMs. They've done a pretty good job at incorporating vision and audio ML models into their ecosystem, but they underestimated language.

Their translation service is based on llms and is commercially a successful product.

Re: GPT-4 details leaked?

#157

Post by albertzeyer »

wiz21c wrote: post #148
kyledrake wrote: post #65

Earlier quoted context omitted.

Unfortunately I've found the current OSS models to be vastly inferior to the OpenAI models. Would love to see someone actually get close to what they can do with GPT-3.5/4, except capable of running on commodity GPUs. What's the most impressive open model so far?

What are all the researchers in universities doing ? Couldn't they improve these models (they do have big brains after all) with tax payer's money and put the results under some cool open source license...

They don't have enough money and thus computing resources. It's a huge gap.

Re: GPT-4 details leaked?

#158

Post by astrange »

CamperBob2 wrote: post #67
valval wrote: post #62

Earlier quoted context omitted.

If it involves Elon being a bad guy, it is certain to have HNers salivating at the thought.

At this point, after attempting to start a literal fight with Zuckerberg, Musk is proposing a penis-measuring contest. He appears to be decompensating in real time. If that's salivation fodder, so be it, but it just makes me sad. You hate to see it happen... or at least I do.

He's joking but making less than no effort to communicate like someone who owns a large company normally would.

Re: GPT-4 details leaked?

#159

Post by astrange »

vouaobrasil wrote: post #78
BoorishBears wrote: post #34

Earlier quoted context omitted.

Why the vitriol towards OpenAI? If Elon hadn't pulled the rug out from under them after they refused his forceful takeover*, they wouldn't have had to go to Microsoft and they'd still be open. * a takeover which he predicated on the claim that OpenAI was "doomed to fail"

Every company who promotes and develops AI is morally responsible for the coming disaster that it will bring on us. If I could have one wish it would be that every trace of AI research is destroyed.

I recommend reading comments like this and substituting "a baby" for AI. A baby also can't be aligned and is capable of deciding to destroy the world. It's not gonna do it though.

Re: GPT-4 details leaked?

#160

Post by dr_kiszonka »

DANmode wrote: post #46
DanAtC wrote: post #42

Earlier quoted context omitted.

I meant things like: - parameters - layers - "Mixture Of Experts" - tokens That's about as far as I made it

Parameters: In the context of AI and language models, parameters refer to the internal settings or variables that an AI model uses to make predictions or generate responses. Think of them as the knobs and switches that can be adjusted to fine-tune how the AI understands and generates language. These parameters are learned during the training process, where the AI model analyzes vast amounts of data to optimize its pe…

Great explanations! How about Multi-Query Attention?
Post reply on HN