Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Finetune GPT-Like Large Language Models on a Custom Dataset

lightning.ai

Post reply on HN
Print view View original

121–126 of 126 posts

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#121

Post by mind-blight »

sanxiyn wrote: post #38
cookieperson wrote: post #11

Earlier quoted context omitted.

Well they can sue you and bankrupt you by delaying trial for a decade. That's how the US patent system works anyways...

Sue on what grounds? It will be quickly dismissed.

That is not how the US legal system works. You can sue someone for and regardless of merit, and they will have to defend themselves. That costs time and legal fees. If they lose, they can appeal, and continue appealing. If it's baseless, they'll lose, but you still spend a lot of money and time dealing with the lawsuits.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#122

Post by anticensor »

itake wrote: post #69
tsunamifury wrote: post #68

Earlier quoted context omitted.

You understand these things have a very very wide interpretation scope here that has yet to be tested in court. I wouldn’t make these statements so confidently as courts tend to reinterpret the law significantly for the balance of societal factors when serious technology changes occur.

AI generated work is not copyright-able. I guess the courts later could disagree though. https://www.copyright.gov/ai/

They are in the UK:

https://www.gov.uk/government/consultations/artificial-intel...

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#123

Post by cookieperson »

sanxiyn wrote: post #38
cookieperson wrote: post #11

Earlier quoted context omitted.

Well they can sue you and bankrupt you by delaying trial for a decade. That's how the US patent system works anyways...

Sue on what grounds? It will be quickly dismissed.

If by quickly you mean 5 to 10 years of paying a retainer on a lawyer sure. Even if you win the case you lose in life. Most individuals can't afford 500k in legal fees to have them be reimbursed years later. Big companies have lawyers on staff at a discount and they play these games every day.

This happens with illegal things all the time. IE manager sexually harasses someone on video or something, it's some CEOs nephew who did it, so they fire the person who got harassed. The person who got harassed now has to aquire legal counsel on top of paying relocation claw backs etc. Few years ago by and the person who was in the right is trying to hold down a job, a family, and the stress of the legal battle. The company offers to settle two years in for 50k and 99% of people take it, sometimes at a loss. Also, getting employed is a lot harder when a background check reveals suing a previous employer or really any company, because shocker, most companies do illegal shit regularly... So it's almost always best to settle

I realize I painted a picture pretty far from my previous statement but I figured you were new in your career and could benefit from an allegory of how stuff like this goes down.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#124

Post by RockyMcNuts »

swalsh wrote: post #77
nico wrote: post #67

What is the main difference between training and fine tuning? Can you start with a model trained only in producing the letter a, and then fine tune it to learn b, then c, then words, sentences, etc?

Not an expert, but my high level understanding is this: If a model is a set of inputs, some middle layers, and a set of outputs. Fine tuning concentrates on only the output layers. Useful for taking a generic model with a base level of knowledge, and tuning it so the output is more useful for an application specific use case.

not strictly true I think

- you could add new units throughout and train those while freezing existing units (adapter-based fine-tuning)

- you could train all units and use e.g. low-rank adaptation to limit how much they can change

- you could do prefix tuning and train an input to add at every layer

see e.g. - https://lightning.ai/pages/community/article/understanding-l...

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#125

Post by tsunamifury »

nl wrote: post #120
tsunamifury wrote: post #88

Earlier quoted context omitted.

In the history of media law I’ve seen judged lean into whatever interpretation balances the ecosystem more than what is “literally the law”. The law is meant to serve people not the other way around. I hope judges will understand the contribution and theft can’t just be “haha fuck humanity love, openAI”

I want to train my own LLM on public but copyrighted data. I think this is serving humanity (and fucking OpenAI). I also think it is ethical because there's a big difference between "learning from" and "copying". Your proposed reading of the law means only big tech will be able to afford the license fees to train on large amounts of data.

How do YOU plan on compensating those whose labor helped you? I bet you don’t. Same thing you are just imagining being David rather than Goliath makes it ok for you.

Re: How to Finetune GPT-Like Large Language Models on a Custom Dataset

#126

Post by mrtranscendence »

tsunamifury wrote: post #125
nl wrote: post #120

Earlier quoted context omitted.

I want to train my own LLM on public but copyrighted data. I think this is serving humanity (and fucking OpenAI). I also think it is ethical because there's a big difference between "learning from" and "copying". Your proposed reading of the law means only big tech will be able to afford the license fees to train on large amounts of data.

How do YOU plan on compensating those whose labor helped you? I bet you don’t. Same thing you are just imagining being David rather than Goliath makes it ok for you.

It's not always necessary to compensate those whose labor helped you. I haven't compensated many of the open source projects I use, for example, even those who clearly want me to (with nagging pop-ups). If the use of copyrightable material to train a model is legal, and it does not legally require compensation, it might be difficult to argue that the use of such material should be compensated or else. It would depend IMO on whether there are norms in place for this kind of thing, and I don't necessarily see wide agreement.
Post reply on HN