Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama and ChatGPT Are Not Open-Source

spectrum.ieee.org

Post reply on HN
Print view View original

51–60 of 130 posts

Re: Llama and ChatGPT Are Not Open-Source

#51

Post by philipov »

barbariangrunge wrote: post #32
bick_nyers wrote: post #9

I personally do not want the companies to release training data (at least for a while) because then it gives people leverage to neuter it. I don't want a sanitized LLM, and I don't have $60M lying around to train my own. Copyrighted material, sexual content, political opinions, throw it all in and release it please! Yes, reducing bias in the models is a noble goal, but introducing new bias and blindspots to do it is…

> Copyrighted material, sexual content, political opinions, throw it all in and release it please! Why copyrighted material? Could we stop celebrating how tech is going to steal everyone's copyrighted works in a massive effort to replace the artists who made it? Why does everyone here hate artists so much? Do they not deserve any rights over their IP, eg, the right to say no when someone wants to make derivative work…

We know who copyright is meant to protect, and it's not artists: it's Disney's shareholders. Pay writers and actors a fair cut, and then maybe people will take seriously that copyright is for artists.

Re: Llama and ChatGPT Are Not Open-Source

#52

Post by bick_nyers »

kristopolous wrote: post #42
bick_nyers wrote: post #21

Earlier quoted context omitted.

Why is it a binary choice? "Most students would answer X, due to this common misconception about Y". As we have seen from history, there is not often an absolute truth to questions, only clusters of truths. We want our LLM to be able to perform reasoning, mathematics, and science, but expecting absolute truths in anything outside of those fields is a bit much. Wikipedia often takes a good approach here and strikes th…

That's not how bullshit works. An expertise is needed from the reader when presented with false material. That's why so many people think parody news articles are real. If you have ever gone through any comment thread on the internet about climate change you'll find a large volume of climate deniers citing God, greedy liberals, George Soros, whatever. The people working with these fictions can always just add more fi…

Right but I feel like in your argument there is an implicit assumption that the only way to get ChatGPT etc. to understand say a global climate system is by omitting false information entirely. Why can't an LLM extract a signal from the noise? It seems as though it does that already.

If you're worried that there is more text on the internet saying that the Earth is flat than there is text saying that the Earth is round, then I understand where you are coming from, but then how would ANYONE come to believe that the Earth is round on their own? By learning how to reason.

Edit: My point is that if the LLM can't weed through bullshit properly then how can we expect it to get anything right at all? We can't just feed it a continuously updated list of facts that are true and facts that are false until the end of time.

Re: Llama and ChatGPT Are Not Open-Source

#53

Post by nightski »

Andrex wrote: post #50
LightHugger wrote: post #46

Earlier quoted context omitted.

Modern copyright is not a good, it's an evil.

That depends on the jurisdiction, but in my view I err on the side of "but it's necessary." A society without copyright would be a much poorer one, and humans figured that out a long time ago.

I don't think you can look to the past to picture a future with AI.

Re: Llama and ChatGPT Are Not Open-Source

#54

Post by eropple »

bick_nyers wrote: post #52
kristopolous wrote: post #42

Earlier quoted context omitted.

That's not how bullshit works. An expertise is needed from the reader when presented with false material. That's why so many people think parody news articles are real. If you have ever gone through any comment thread on the internet about climate change you'll find a large volume of climate deniers citing God, greedy liberals, George Soros, whatever. The people working with these fictions can always just add more fi…

Right but I feel like in your argument there is an implicit assumption that the only way to get ChatGPT etc. to understand say a global climate system is by omitting false information entirely. Why can't an LLM extract a signal from the noise? It seems as though it does that already. If you're worried that there is more text on the internet saying that the Earth is flat than there is text saying that the Earth is rou…

> Right but I feel like in your argument there is an implicit assumption that the only way to get ChatGPT etc. to understand say a global climate system is by omitting false information entirely. Why can't an LLM extract a signal from the noise? It seems as though it does that already.

Because it's not thinking, it's looking at probabilities. If you don't prune branches, you've created a disinformation radiator if that information is sufficiently populous within your corpus to be a pretty-good-bet.

The best analogy I've come up with is "it's like if you had an objective reality based on Wikipedia edit wars."

Re: Llama and ChatGPT Are Not Open-Source

#55

Post by threeseed »

LightHugger wrote: post #46
barbariangrunge wrote: post #32

Earlier quoted context omitted.

> Copyrighted material, sexual content, political opinions, throw it all in and release it please! Why copyrighted material? Could we stop celebrating how tech is going to steal everyone's copyrighted works in a massive effort to replace the artists who made it? Why does everyone here hate artists so much? Do they not deserve any rights over their IP, eg, the right to say no when someone wants to make derivative work…

Modern copyright is not a good, it's an evil.

Authors, artists, actors etc whose work have been ripped off by mega corporations like Meta, OpenAI, Google etc would disagree.

Copyright may need to be updated to cater for the new world of AI but that doesn't mean as a concept it is evil.

Re: Llama and ChatGPT Are Not Open-Source

#56

Post by accrual »

barbariangrunge wrote: post #35
version_five wrote: post #15

Earlier quoted context omitted.

And posted the link to your subscribers only medium page twice in this thread

devils advocate: if it wasn't subscribers only, google's crawler would be stealing it right now to train an ai with, ie, illegally creating derivative works based on their articles

Maybe there's a market in the future for "AI-restricted" content. You can send your payment/tokens/etc. to access the content and the user is bound by a no-AI-clause. Because this content would be rare and can't be found via models, it offers a niche market.

Re: Llama and ChatGPT Are Not Open-Source

#57

Post by kristopolous »

ta93754829 wrote: post #41
kristopolous wrote: post #17

Earlier quoted context omitted.

We really need to somehow separate a bias towards accuracy as distinct from some bias towards say, a sports team. Using everything would be like taking a bunch of students final exams and then claiming the most common answers are the correct ones. This isn't how expertise and accuracy works. Most things worth doing are not only genuinely hard and complicated but something that only a minority subset of accomplished p…

When you dig into it, there are very few truths. Who "won" the last US election? Where did covid come from? Humans will argue the right answer until their last days. It's frustrating how on-the-fence chatgpt can be. It's pretty interesting too, because in a professional environment one of the most important things you need to do is have an opinion and take a position otherwise you can't execute.

That's not true though. Every word of this sentence, for instance, has a correct spelling and grammatical rules as to where the words and punctuation go.

To English learners, that might be very difficult. I'm learning a new language now and I feel the pain. The textbook and instructor, in this case, is way more correct than the collective opinion of my fellow students.

The vast majority of things actually follow this pattern instead. Knowing what a programming function would output given a set of inputs isn't some gray area world of opinion.

When things are unknown, such as say Covid origin, again there's people way beyond my pay-grade doing investigative work here. Youtube comment threads and tiktok opinions are actually not as valuable as peer reviewed scholarship on the topic EVEN IF there's say, 10 youtube comments saying one thing and 5 papers in the Lancet saying another. The youtube comments don't become correct by volume here.

It's not some giant equal playing field.

Re: Llama and ChatGPT Are Not Open-Source

#58

Post by threeseed »

philipov wrote: post #51
barbariangrunge wrote: post #32

Earlier quoted context omitted.

> Copyrighted material, sexual content, political opinions, throw it all in and release it please! Why copyrighted material? Could we stop celebrating how tech is going to steal everyone's copyrighted works in a massive effort to replace the artists who made it? Why does everyone here hate artists so much? Do they not deserve any rights over their IP, eg, the right to say no when someone wants to make derivative work…

We know who copyright is meant to protect, and it's not artists: it's Disney's shareholders. Pay writers and actors a fair cut, and then maybe people will take seriously that copyright is for artists.

It's both. Because we don't live in a child-like, black & white world.

Copyright like many regulatory devices can be used for good, evil and everything in between.

Re: Llama and ChatGPT Are Not Open-Source

#59

Post by __loam »

bick_nyers wrote: post #47
barbariangrunge wrote: post #32

Earlier quoted context omitted.

> Copyrighted material, sexual content, political opinions, throw it all in and release it please! Why copyrighted material? Could we stop celebrating how tech is going to steal everyone's copyrighted works in a massive effort to replace the artists who made it? Why does everyone here hate artists so much? Do they not deserve any rights over their IP, eg, the right to say no when someone wants to make derivative work…

Pretty much everything nowadays is copyrighted, by omitting such materials, what are you really left with? LLM is a tool much like the internet is a tool. Yes, someone can use it to steal, but stealing is against the law. Instead of encoding a criminal justice system into an LLM by omitting the possibility of stealing an artists work or omitting the knowledge of physics so someone can't learn how to build a bomb, we…

You're not addressing the massive abuse of the commons this still represents. If artists don't have the right to tell you to fuck off for using their work in training data, they're less likely to publicly show that work, which hurts them because they become less visible and hurts the AI because the training data gets worse.

Re: Llama and ChatGPT Are Not Open-Source

#60

Post by __loam »

LightHugger wrote: post #46
barbariangrunge wrote: post #32

Earlier quoted context omitted.

> Copyrighted material, sexual content, political opinions, throw it all in and release it please! Why copyrighted material? Could we stop celebrating how tech is going to steal everyone's copyrighted works in a massive effort to replace the artists who made it? Why does everyone here hate artists so much? Do they not deserve any rights over their IP, eg, the right to say no when someone wants to make derivative work…

Modern copyright is not a good, it's an evil.

People should be paid for their labor.
Post reply on HN