GPT-4 details leaked? • Hacker News Board

Re: GPT-4 details leaked?

#331

Post by numpad0 » Tue, Jul 11, 2023, 12:47 PM UTC

ly3xqhl8g9 wrote: post #253

JumpCrisscross wrote: post #240

Earlier quoted context omitted.

> Most benefits from the copyright system are accrued to large corporations Citation fucking needed. Among those who study copyright and inequality, none suggest abandoning it [1][2]. Within the context of machine learning, one of the only pillars buttressing individuals against multi-trillion dollar corporations is copyright [3]. [1] https://journals.library.wustl.edu/lawreview/article/id/5108... [2] https://www.jst…

To nitpick, the second article is from 1970, back then a concept such as the Amazon Antitrust Paradox [1] would cause burning at the stake in the legal field. Similarly, we also have an Elsevier Antitrust Paradox, a Nature(.com) Antitrust Paradox, and so on. Just the fact that I cannot freely read an article published 53 years ago is beyond ludicrous. And also, if I pay JSTOR ($19.5/month) and read the 1970 article,…

I'm not sure how that has to do with abolishing copyright, when that mostly has to do with weak and skewed protections current laws offer. They're not allowed to take names off copyrighted text, which is partly why that name is still on that paywalled text. They're not effectively forbidden by law to purchase to become a corporate owner of copyright, which is how they get to paywall it in their name.

If they, in loose usage of the pronoun, were allowed to deface an art, papers, code, anything copyrighted, they will. Stripping copyright text could even become an "etiquette". Conversely, if they were not allowed to profit off of a thing, copyrighted, with appropriate enforcement, they won't. These should go without saying for core Internet users and software engineers.

I mean, haven't you heard what Oracle just said?

Re: GPT-4 details leaked?

#332

Post by notahacker » Tue, Jul 11, 2023, 12:51 PM UTC

ly3xqhl8g9 wrote: post #320

notahacker wrote: post #309

Earlier quoted context omitted.

I'm not sure quite why you're continuing to post articles explaining why writers get paid less as streaming services' revenue is less linked to the ratings of new shows as an argument in favour of marking down the value of the intellectual property they create to zero... My original point still stands. Ask the writers. They're not getting paid nothing yet , and they won't approve of your passionate advocacy of a futu…

I'm not quite sure why you are not understanding that with the streaming model of business the writers, and even the actors, no longer get residuals, here is another article [1]: yes, they get paid nothing. I am advocating for the abolishment of copyright laws, not for not being rewarded for one's work. [1] Euphoria actress Sydney Sweeney revealed in a recent interview with The Hollywood Reporter that "They don’t pay…

I'm not sure why you are not understanding that not getting residuals is not the same as not getting a salary or a per writing contribution payment. Or that production companies are not paying these salaries (and the rest of the production budget) out of the kindness of their hearts, but out of revenues accrued from networks and streaming services having to pay to screen their shows.

Or that writers will not be paid anything (and certainly not by streaming services) in a copyright-free world in which anybody is allowed to use any creative work in any way free of charge. The abolition of copyright laws is literally the abolition of virtually[1] all existing rewards for creating IP.

This is not complicated stuff: if it's genuinely news to you that writers aren't working for free, perhaps you are not the person to lecture us on how copyright should work.

[1]I guess writers could still ask for donations. They could do this already, but they prefer negotiating with networks for higher pay packages...

Re: GPT-4 details leaked?

#333

Post by mdp2021 » Tue, Jul 11, 2023, 12:52 PM UTC

mensetmanusman wrote: post #325

mdp2021 wrote: post #323

Earlier quoted context omitted.

...but in a technical context, not in a scientific one. One thing is identifying (age of copper; age of bronze) the best ways of smelting ore to obtain the metal through trial and error, another is to try and understand the nature of materials.

Those steps are both necessary and usually take decades to build understanding.

But the activity of "attempting to find explanations" is specific.

The attempt to formulate laws (the "better faring hypoteses") can bring to "universal laws" or to more humble "patterns". It will contribute to the scientific effort - it does not exhaust it.

Re: GPT-4 details leaked?

#334

Post by pas » Tue, Jul 11, 2023, 12:59 PM UTC

jona-f wrote: post #304

DeathArrow wrote: post #258

Earlier quoted context omitted.

> There are people working on this, but afaik the techniques aren't quite there. You need a different kind of model with much more parallelization then what is currently used. What if crypto is switching from mindless hashing as proof-of-work to training AI models as proof-of-work? That would mean suddenly big computing resources are available.

That would indeed be neat, no idea if that is possible safely. Blockchain technology sure looks like a good fit for the organization of such a decentralized model. The reaction of the people of hn to 'blockchain-ai', probably won't be kind though, lol.

Blockchains depend on small inputs and outputs, and easy verifiability of the computation. AI training is none of that. It's a massively interconnected problem, Google and NVIDIA built traditionalish fast-interconnect supercomputers to do it.

Maybe it's possible to implement it in a decentralized way with not-completely-useless performance, but by then OpenAI and others will be even more ahead. (Sure, it'd be good to have such an implementation, so maybe enthusiasts will do it eventually, but that's mostly for fun.)

Re: GPT-4 details leaked?

#335

Post by blueprint » Tue, Jul 11, 2023, 1:00 PM UTC

varispeed wrote: post #259

DeathArrow wrote: post #243

Earlier quoted context omitted.

Still, the guys developing ML algorithms like Neural nets, Linear regression, Decision tree, SVM, Naive Bayes, KNN etc., weren't doing science?

It's like asking whether a carpenter is a scientist because they developed a cabinet...

I hate to break it to you but that is part of science. Perhaps the major part of it too. Hypothesis: I can build a cabinet with these materials which will bear some load range. Experiment: I built it and it obviously works.

Now change cabinet to particle accelerator that a giant team of other theorists and engineers designed. Am I not doing science by participating in building it? So experimental scientists arent scientists?

Re: GPT-4 details leaked?

#336

Post by TradingPlaces » Tue, Jul 11, 2023, 1:00 PM UTC

PostOnce wrote: post #19

"Open" AI, a charity to benefit us all by pushing and publishing the frontier of scientific knowledge. Nevermind, fuckers, actually it's just to take your jobs and make a few VCs richer. We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. https://github.com/ggerganov/llama.cpp https://github.com/openlm-research/open_llama https://huggingface.co/TheBlok…

Keep in mind, that was the idea originally. Then in 2018 Elon decided he wanted to be CEO, the board rejected that idea for now obvious reasons, and he reneged on 90% of a promised $1b donation. The only way forward was to become a for-profit company and do a normal funding round, which is what happened with Microsoft. Elon’s rug pull is why this happened.

Re: GPT-4 details leaked?

#337

Post by baanlox » Tue, Jul 11, 2023, 1:02 PM UTC

0xDEF wrote: post #328

arthur_sav wrote: post #241

Earlier quoted context omitted.

I would love to use these... but they suck. They don't even come close to what OpenAI offers.

I think most of HN has only tried ChatGPT with GPT-3.5 in December 2022 when the OpenAI servers were getting hammered. That is why they're impressed by these slow and low quality local models and mistakenly think they're on par with OpenAI's offerings. Honestly even reddit and teenagers on TikTok have a more accurate view on OpenAI vs. local LLMs than HN.

[dead]

Re: GPT-4 details leaked?

#338

Post by mt_ » Tue, Jul 11, 2023, 1:03 PM UTC

shahules wrote: post #13

This guy doesn't have any idea what he is talking about. He consistently posts such bullshit on twitter. Mostly copy paste with added spice mix.

It's Twitter, why would you think otherwise?

Re: GPT-4 details leaked?

#339

Post by pas » Tue, Jul 11, 2023, 1:09 PM UTC

CSMastermind wrote: post #2

Previously posted about here: https://news.ycombinator.com/item?id=36671588 and here: https://news.ycombinator.com/item?id=36674905 With the original source being: https://www.semianalysis.com/p/gpt-4-architecture-infrastruc... The twitter guy seems to just be paraphrasing the actual blog post? That's presumably why the tweets are now deleted. --- The fact that they're using MoE was news to me and very interesting. I…

Interestingly Google was using ~2000 experts back in the first Trasnformer architecture (if I understand correctly) https://www.youtube.com/watch?v=9P_VAMyb-7k&t=6m42s [sparsely-gated mixture of experts layer]

Re: GPT-4 details leaked?

#340

Post by ly3xqhl8g9 » Tue, Jul 11, 2023, 1:15 PM UTC

notahacker wrote: post #332

ly3xqhl8g9 wrote: post #320

Earlier quoted context omitted.

I'm not quite sure why you are not understanding that with the streaming model of business the writers, and even the actors, no longer get residuals, here is another article [1]: yes, they get paid nothing. I am advocating for the abolishment of copyright laws, not for not being rewarded for one's work. [1] Euphoria actress Sydney Sweeney revealed in a recent interview with The Hollywood Reporter that "They don’t pay…

I'm not sure why you are not understanding that not getting residuals is not the same as not getting a salary or a per writing contribution payment. Or that production companies are not paying these salaries (and the rest of the production budget) out of the kindness of their hearts, but out of revenues accrued from networks and streaming services having to pay to screen their shows. Or that writers will not be paid…

If it's genuinely news to you that exploitation is a normal day-to-day part of any industry, perhaps these pointless replies should end here.

Yes, writers have been working for free [1].

[1] "In October 2015, Wil Wheaton created a stir when he declared that he had turned down an offer to write for the Huffington Post. He refused, according to him, because they had declined to pay for his work, in keeping with their policy of reimbursing writers with "exposure" in lieu of payment." https://www.vox.com/2016/2/26/11106006/writing-for-free Guess what writers who aren't famous have to do? Work for free, for "exposure".