Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source AI is the path forward

about.fb.com

Post reply on HN
Print view View original

921–930 of 936 posts

Re: Open source AI is the path forward

#921

Post by [deleted] »

MaxBarraclough wrote: post #910
CamperBob2 wrote: post #893

Earlier quoted context omitted.

Terms of art do not require licenses. A given term is either an ordinary dictionary word that everyone including the courts will readily recognize ("Vegan"), a trademark ("Microsoft® Office 365™"), or a fragment of language that everyone can feel free to use for their own purposes without asking permission. "Open Source" falls into the latter category. This kind of argument is literally why trademark law exists. OSI di…

> Terms of art do not require licenses. Agreed. There is no trademark on aileron or carburetor or context-free grammar . A couple of years ago I made this same point myself. [0] > A given term is either an ordinary dictionary word that everyone including the courts will readily recognize ("Vegan"), a trademark ("Microsoft® Office 365™"), or a fragment of language that everyone can feel free to use for their own purpo…

[deleted]

Re: Open source AI is the path forward

#922

Post by JKCalhoun »

ein0p wrote: post #397
JKCalhoun wrote: post #356

Earlier quoted context omitted.

A good point. Forgive me, I am AI naive, is there some way to harness Llama to train ones own actually-open AI?

I could be wrong but most “model” licenses prohibit the use of the models to improve other models

That's a good point. I expect it is ultimately unenforceable though. I'm describing training a model for myself, not for sale or public consumption.

Re: Open source AI is the path forward

#923

Post by the8thbit »

verdverm wrote: post #869
piperswe wrote: post #625

Earlier quoted context omitted.

You can absolutely modify compiled binaries to get something different. That's how lots of video game modding and ROM hacks work.

I considered adding "normally" to the binary modifications expecting a response like this. The concepts are still worlds apart Weights aren't really a binary in the same sense that a compiler produces, they lack instructions and are more just a bunch of floating point values. Nor can you run model weights without separate code to interpret them correctly. In this sense, they are more like a JPEG or 3d model

JPEGs and 3D models are also executable binaries. They, like model weights, contain domain specific instructions that execute in a domain specific and turing incomplete environment. The model weights are the instructions, and those instructions are interpreted by the inference harness to produce outputs.

Re: Open source AI is the path forward

#924

Post by CamperBob2 »

MaxBarraclough wrote: post #910
CamperBob2 wrote: post #893

Earlier quoted context omitted.

Terms of art do not require licenses. A given term is either an ordinary dictionary word that everyone including the courts will readily recognize ("Vegan"), a trademark ("Microsoft® Office 365™"), or a fragment of language that everyone can feel free to use for their own purposes without asking permission. "Open Source" falls into the latter category. This kind of argument is literally why trademark law exists. OSI di…

> Terms of art do not require licenses. Agreed. There is no trademark on aileron or carburetor or context-free grammar . A couple of years ago I made this same point myself. [0] > A given term is either an ordinary dictionary word that everyone including the courts will readily recognize ("Vegan"), a trademark ("Microsoft® Office 365™"), or a fragment of language that everyone can feel free to use for their own purpo…

If the OSI members wanted to "clarify the terminology" in a way that permitted them (and you) to exclude others, trademark law would have absolutely been the correct way to do that. It's too late, however. The ship has sailed.

Come up with a new term and trademark that, and heck, I'll help you out with a legal fund donation when Facebook and friends inevitably try to appropriate it. Apart from that, you've fought the good fight and done what you could. Let it go.

Re: Open source AI is the path forward

#925

Post by the8thbit »

sebastiennight wrote: post #466
gowld wrote: post #460

Earlier quoted context omitted.

Can a human programmer edit the weights according to some semantics?

It is possible to merge two fine-tunes of models from the same family by... wait for it... averaging or combining their weights[0]. I am still amazed that we can do that. [0]: https://arxiv.org/abs/2212.09849

This is absolutely wild.

Re: Open source AI is the path forward

#926

Post by roguas »

the8thbit wrote: post #12

"Eventually though, open source Linux gained popularity – initially because it allowed developers to modify its code however they wanted ..." I find the language around "open source AI" to be confusing. With "open source" there's usually "source" to open, right? As in, there is human legible code that can be read and modified by the user? If so, then how can current ML models be open source? They're very large matric…

No, open source means that sources are open, typically for inspection, modification etc. Also here it can be considered the case. Likely in order to claim "true open source", they would have to share dataset? But even this might not be enough for truely open source model? This dataset is nothing but another artifact. So how did they arrive at this dataset, now they have to share pipelines and infra...

.. the thing is, we have not dealt with llm much, it's hard to say what can be considered open source llm just yet, so we use that as metaphore for now

Re: Open source AI is the path forward

#927

Post by yencabulator »

jijji wrote: post #880
Aeolun wrote: post #685

Earlier quoted context omitted.

I suspect that even if you allowed people to take the data, nobody but a FAANG like organisation could even store it?

yeah, according to the article [0] about the release of Llama 3.1 405B, it was trained on 15 trillion tokens using 16000 Nvidia H100's to do it. Even if they did release the training data, I don't think many people would have the number of gpus required to actually do any real training to create the model.... [0] https://ai.meta.com/blog/meta-llama-3-1/

And a token is the sequence number of a sequence of input in a restricted dictionary. GPT-2 was said to have 50k distinct tokens, so I think it's safe to assume even the latest ones are well below 4M tokens, so max 4 bytes per token. 15 trillion tokens -> 4 bytes/token * 15 T tokens -> training inputIt's the computation that is costly.

Re: Open source AI is the path forward

#928

Post by makingstuffs »

ohlookcake wrote: post #808
makingstuffs wrote: post #761

Earlier quoted context omitted.

> IF I ever had to go to FB for anything, I'd probably install a wall-removing browser extension. Mobile app is of course out of question. You’ll probably find you can no longer make an account. I’m in the same boat as you (not used and haven’t missed in over a decade), however, my partner needed an account to manage an ad campaign for a client and neither of us were able to make one. Both tried a load of different t…

I just tried making one after reading your comment, and it was... pretty straightforward? I'm curious what blockers you encountered

Must be something to do with our situation? Basically both I and my partner just got told we were making fraudulent accounts. Used our real names, emails, phone numbers and same result. Used multiple other phone numbers to see if they were blacklisted for some reason, nope. Used my mother’s WiFi in case it was something to do with IP, nope. Tried from India with an Indian phone, nope.

Re: Open source AI is the path forward

#929

Post by dogcomplex »

Zambyte wrote: post #814
dogcomplex wrote: post #636

Earlier quoted context omitted.

Or just develop the next wave of chips designed for specifically transformer-based architectures (and ternary computing), and bypass the needs for GPUs and CUDA altogether

That would be betting against other architectures like Mamba, which does not seem like an obviously good bet to make yet. Maybe it is though.

You're right, there are a number of avenues that are viable alternatives to the gpu monopoly.

I like the fact that these can be made with just mass-printed multiplication (and in ternary computing's case - addition) gates which require little more than 10 year old tech which is already widely distributed.

Re: Open source AI is the path forward

#930

Post by bayindirh »

JumpCrisscross wrote: post #7

“The Heavy Press Program was a Cold War-era program of the United States Air Force to build the largest forging presses and extrusion presses in the world.” This ”program began in 1944 and concluded in 1957 after construction of four forging presses and six extruders, at an overall cost of $279 million. Six of them are still in operation today, manufacturing structural parts for military and commercial aircraft” [1].…

> A public cluster of GPUs provided for free to American universities, companies and non-profits might not be a bad idea.

USA and Europe is already doing that in a grand scale, in different forms. Both at national and international scale.

I work at an HPC center which provides servers nationally and collaborates on international level.

Post reply on HN