Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Facebook LLAMA is being openly distributed via torrents

github.com

Post reply on HN
Print view View original

201–210 of 719 posts

Re: Facebook LLAMA is being openly distributed via torrents

#201

Post by basch »

colinmorelli wrote: post #88
RobotToaster wrote: post #54

Earlier quoted context omitted.

IANAL, but I imagine it's a legal grey area if the weights can be copyrighted? Works produced by purely mechanical means don't normally meet the threshold of originality.

I am quite confident that using this model for commercial purposes will, if detected, land you in quite a legal quagmire that almost certainly sides in favor of Meta. And even if it did not, Meta certainly has a more capable legal team with more cash to spend than the average HN user.

Has any photo or art ever been found to have been illegal due to pirated photoshop?

Re: Facebook LLAMA is being openly distributed via torrents

#202

Post by justinclift »

Brian_K_White wrote: post #158
bayindirh wrote: post #105

Earlier quoted context omitted.

Or BSD distros.

These are actually trivial and silly examples. The bulk of really valuable commercial code is not self contained or portable like those. Where is the torrent with a runnable copy of paypal, or amazon?

Heh, on that note:

https://news.ycombinator.com/item?id=34525936

Looking down that page, there are some valid magnet links.

Re: Facebook LLAMA is being openly distributed via torrents

#203

Post by bayindirh »

Brian_K_White wrote: post #196
bayindirh wrote: post #193

Earlier quoted context omitted.

I guess you have never compiled a Linux/BSD distribution from scratch and supported it alongside its infrastructure and lead its maintenance process. Even without that, if you want downloadable and runnable software platforms, look to public Git repositories. Some of the people who have no financial motivation will release what they do alongside installation procedures and quality of life scripts and architecture doc…

"I guess you have never compiled a Linux/BSD distribution from scratch" You guess, and so perhaps do other things as well, with poor acuity. The incalculable value of open source software has approximately no bearing on this assertion. Yes I love linux and bsd too I'm not defacing your religion. I'm actually quite Stallmanesque in making my own life harder by only using ooen source software as much as possible and be…

Nice, it makes us two tech leads then.

Re: Facebook LLAMA is being openly distributed via torrents

#204

Post by fnordpiglet »

micromacrofoot wrote: post #197
weberer wrote: post #153

Earlier quoted context omitted.

/g/ is one of the SFW boards

I can put on a paper bag and call it pants, but that doesn't make it a smart thing to wear to work

That’s precisely why return to office is a stupid idea

Re: Facebook LLAMA is being openly distributed via torrents

#205

Post by dmix »

forgotOldLogin wrote: post #163
sebzim4500 wrote: post #89

Earlier quoted context omitted.

No you are probably overestimating the cost by 1-2 orders of magnitude. GPT-3 probably cost under $5 million, and this model is smaller and there have been algorithmic improvements to training transformers since then.

In February 2018, OpenAI signed a two-year compute contract with Google that had a $63M minimum spend. See last page of their most recent available audited financials. https://rct.doj.ca.gov/Verification/Web/Download.aspx?saveas...

So do they estimate how much computing power/time they will need and then find some upper tier minimum $ amount to get the maximum discount possible or getting a certain resource availability commitment from Google? That's an interesting accounting problem.

Re: Facebook LLAMA is being openly distributed via torrents

#206

Post by mysterydip »

dmix wrote: post #183
narrator wrote: post #151

Earlier quoted context omitted.

It's funny that part of the 4chan excitement over this is that they think they'll get back the AI girlfriend experience of when character.ai was hooked up to uncensored GPT-3. All that has been thoroughly shut down by character.ai and Replika and they just want their girlfriends back.

The Repilka subreddit became one of the weirdest places on the internet when their model got capped for adult content. https://www.reddit.com/r/replika/ Hundreds of men (and yes women) full on acting like they lost a spouse and posting constantly about it for weeks. AI is going to create some unusual social situations the general public isn't ready to grasp. And we're only in the early alpha stages.

> ELIZA's (1966) creator, Weizenbaum, intended the program as a method to explore communication between humans and machines. He was surprised, and shocked, that individuals attributed human-like feelings to the computer program, including Weizenbaum's secretary.

(citations and further info in the wikipedia article https://en.m.wikipedia.org/wiki/ELIZA)

Re: Facebook LLAMA is being openly distributed via torrents

#207

Post by londons_explore »

londons_explore wrote: post #65
px43 wrote: post #57

Earlier quoted context omitted.

Hell, I'd love to be able to buy a $30k server to run these models. I think to run BLOOM required something more along the lines of a $200k server.

With code modifications, it should be possible to run this with a very modest machine as long as you're happy for performance to suck. Transformer models typically need to read all the weights per 'word' output, so if your model is 20GB and you have not enough ram or vram, but have an SSD that reads 1GB/sec, expect 3 words per minute output speed. However, code changes are necessary to achieve that, although they won…

There is a neat potential speedup here for the case where the bandwidth to your model weights is the limiting factor.

If you have a guess what the model will output, then you can verify that your guess is correct very cheaply, since you can do it in parallel.

That means there is the possibility to have a highly quantized small model in RAM, and then use the big model only from time to time. You might be able to get a 10x speedup this way if your small model agrees 90% of the time.

Re: Facebook LLAMA is being openly distributed via torrents

#208

Post by pas »

brookst wrote: post #147
pas wrote: post #128

Earlier quoted context omitted.

... how is that a good test of anything? I mean what if I click on a /b/ link "at work"? Does that make my work output immediately tainted and the company has to immediately file for bankruptcy?

Nah, it just reflects poor judgment that may extend to other areas. It’s a little like the Van Halen M&M test: if you don’t follow that rule, people have to wonder what other expectations you won’t meet.

They don't have to, I very loudly proclaim which rules I have a lot of disregard for :)

> little like the Van Halen M&M test

Hah, yes, though in this case I apply it "inversely". Anyone who gets lost in the process, instead of considering the people in it, is out. (That's why, usually, my conflicts/problems with my late bosses/employers had something to do with them being a bit too cavalier when it came formalities like ... paying in time.) Trade-offs, trade-offs are hard.

Re: Facebook LLAMA is being openly distributed via torrents

#209

Post by gpm »

koheripbal wrote: post #130
px43 wrote: post #64

Earlier quoted context omitted.

Yeah, their recent papers show the smaller LLAMA models outperforming the major LLMs today, and they also have bigger models. This isn't just an alternative, it's a multi order of magnitude optimization. https://aibusiness.com/meta/meta-s-llama-language-model-outp...

Can I spend $5K and run it at home? What GPU(s) do I need?

In principal you can run it on just about any hardware with enough storage space. It's just a question of how fast it will run. This readme has some benchmarks with a similar set of models (and the code has support for even swapping data out to disk if needed): https://github.com/FMInference/FlexGen

And here are some benchmarks running OPT-175B purely on (a very beefy) CPU machine. Note that the biggest llama model is only 65.2B: https://github.com/FMInference/FlexGen/issues/24

Re: Facebook LLAMA is being openly distributed via torrents

#210

Post by joseph_grobbles »

pavlov wrote: post #41

It’s interesting that these models are both massively expensive to produce and self-contained to a degree that you can distribute the end product in a torrent. This has not been the case for most commercial software for the past 20 years, during the cloud era. If you could steal a dump of random Facebook source code, it would be 99% useless because it’s so closely tied to the infrastructure. There’s almost nothing yo…

(My posts are dead by default, but for the good showdead people if someone knows and can answer in a sibling reply)

Are models like this copyrightable? It seems like this falls under the realm of "fact", which can't be copyrighted.

Post reply on HN