Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source AI is the path forward

about.fb.com

Post reply on HN
Print view View original

891–900 of 936 posts

Re: Open source AI is the path forward

#891

Post by photonthug »

CardenB wrote: post #532
photonthug wrote: post #519

Earlier quoted context omitted.

> Not sure why a publicly accessible GPU cluster would be a better solution than the current system of research grants. You mean a better solution than different teams paying AWS over and over, potentially spending 10x on rent rather than using all that cash as a down payment on actually owning hardware? I can't really speak for the total costs of depreciation/hardware maintenance but renting forever isn't usually a…

Do you have some information to share to support your bias against leasing especially with a depreciating asset?

sure, I’ll hand it over after you spend your own time first to show that everything everywhere that’s owned instead of leased is a poor financial decision.

Re: Open source AI is the path forward

#892

Post by MaxBarraclough »

CamperBob2 wrote: post #885
Flimm wrote: post #669

Earlier quoted context omitted.

You're right, I and those who agree with me were the first to ask people to "shut up", in this case, to ask Meta to stop misusing the term open source. And I was the first to say "shut up", and I know that can be inflammatory and disrespectful, so I shouldn't have used it. I'm sorry. We're here in a discussion forum, I want you to express your opinion even it is to complain about my complaints. For what it's worth, y…

All good, and I also apologize if my objection came across as disrespectful. This whole 'Open Source' thing is a bigger pet peeve than it should be, because I've received criticism for using the term on a page where I literally just posted a .zip file full of source code. The smart thing to do would have been to ignore and forget the criticism, which I will now work harder at doing. In the case of a pork producer who…

> In the case of a pork producer who labels their products as 'vegan', that's different because there is some authority behind the usage of 'vegan'.

I don't see the difference. Open source software is a term of art with a specific meaning accepted by its community. When people misuse the term, invariably in such a way as to broaden it to include whatever it is they're pushing, it's right that the community responds harshly.

Re: Open source AI is the path forward

#893

Post by CamperBob2 »

MaxBarraclough wrote: post #892
CamperBob2 wrote: post #885

Earlier quoted context omitted.

All good, and I also apologize if my objection came across as disrespectful. This whole 'Open Source' thing is a bigger pet peeve than it should be, because I've received criticism for using the term on a page where I literally just posted a .zip file full of source code. The smart thing to do would have been to ignore and forget the criticism, which I will now work harder at doing. In the case of a pork producer who…

> In the case of a pork producer who labels their products as 'vegan', that's different because there is some authority behind the usage of 'vegan'. I don't see the difference. Open source software is a term of art with a specific meaning accepted by its community. When people misuse the term, invariably in such a way as to broaden it to include whatever it is they're pushing, it's right that the community responds h…

Terms of art do not require licenses. A given term is either an ordinary dictionary word that everyone including the courts will readily recognize ("Vegan"), a trademark ("Microsoft® Office 365™"), or a fragment of language that everyone can feel free to use for their own purposes without asking permission. "Open Source" falls into the latter category.

This kind of argument is literally why trademark law exists. OSI did not elect to go down that path. Maybe they should have, but I respect their decision not to, and perhaps you should, too.

Re: Open source AI is the path forward

#894

Post by nomel »

GreenWatermelon wrote: post #766
nomel wrote: post #520

Earlier quoted context omitted.

I think saying it's the new binary is closer to the truth. You can't reproduce it, but you can use it. In this new version, you can even nudge it a bit to do something a little different. New stuff, so probably not good to force old words, with known meanings, onto new stuff.

The model is more akin to a python script than a compiled C binary. This is how I see it: Training Code and dataset are analogous to the developer who wrote the script Model and weights are end product that is then released Inference Code is the runtime that could execute the code. That would be e.g. PyTorch, which can import the weights and run inference.

> The model is more akin to a python script than a compiled C binary.

No, I completely disagree. Python is near pseudo-text source. Source exists for the specific purpose of being easily and completely understood, by humans, because it's for and from humans. You can turn a python calculator into a web server, because it can be split and separated at any point, because it can be completely understood at any point, and it's deterministic at every point.

A model cannot be understood by a human. It isn't meant to be. It's meant to be used, very close to as is. You can't fundamentally change the model, or dissect it, you can only nudge it in a direction, with the force of that nudge being proportional to the money you can burn, along with hope that it turns out how you want.

That's why I say it's closer to a binary: more of a black box you can use. You can't easily make a binary do something fundamentally different without changing the source. You can't easily see into that black box, or even know what it will do without trying. You can only nudge it to act a little differently, or use it as part of a workflow. (decompilation tools aside ;))

Re: Open source AI is the path forward

#895

Post by floydnoel »

guitarlimeo wrote: post #821
ab5tract wrote: post #486

Earlier quoted context omitted.

If you can’t share the dataset, under what twisted reality are you fine to share the derivative models based on those unsharable datasets? In a better world, there would be no “I ran some algos on it and now it’s mine” defense.

Yeah was gonna say exactly the same thing. Weird how the legislation allows releasing LLMs trained on data that is not allowed to be shared otherwise.

Meta might possibly have a license to use (some of) that data, but not a license to distribute it. Legislation has little to do with it, I imagine.

Re: Open source AI is the path forward

#896

Post by wrs »

sailingparrot wrote: post #514
wrs wrote: post #506

Earlier quoted context omitted.

Can you change the tokenizer? No, because all you have is the weights trained with the current tokenizer. Therefore, by any normal definition, you don’t have the source. You have a giant black box of numbers with no ability to reproduce it.

> Can you change the tokenizer? Yes. You can change it however you like, then look at the paper [1] under section 3.2. to know which hyperparameters were used during training and finetune the model to work with your new tokenizer using e.g. FineWeb [2] dataset. You'll need to do only a fraction of the training you would have needed to do if you were to start a training from scratch for your tokenizer of choice. The w…

You can change the tokenizer and build another model, if you can come up with your own version of the rest of the source (e.g., the training set, RLHF, etc.). You can’t change the tokenizer for this model, because you don’t have all of its source.

Re: Open source AI is the path forward

#897

Post by sebastiennight »

sgu999 wrote: post #807
kashyapc wrote: post #740

Earlier quoted context omitted.

I get your frustration of a scorched internet. But I don't think it's all that gloomy. Whether we like it or not, LLMs and some kind of a "down-to-earth AI" is here to stay, once the dust settles. Right now, it feels like everything is burning because we're in the thick of an evolving situation; and the Silicon Valley tech-bros are in a hurry to ride the wave and make a quick buck with their ChatGPT wrapper. (I can't…

What a nice quote. What were "millions of people" simultaneously obsessed with around 1841?

I believe MacKay's book (quoted here) is most famous for its account of the Dutch tulip mania[0], which is one of the most notable financial bubbles in history (somebody reportedly paid 12 acres of land - 5 hectares - for a single bulb).

[0]: https://en.wikipedia.org/wiki/Tulip_mania

Re: Open source AI is the path forward

#898

Post by sebastiennight »

lukas099 wrote: post #796
LauraMedia wrote: post #742

Earlier quoted context omitted.

The big problem with this is that content is harder and harder to find. Try to find a non-AI generated reply to a viral post on Twitter, you're looking at having to scroll down 5-6 1080p screens to finally get to some actual stuff people wrote. The content you're enjoying today still exists, but it's a needle in a haystack of AI spam

The signal:noise ratio is decreasing because it’s easier to generate noise. I think paying for content (or content curation) is probably the way to curate high-signal information feeds.

Just like ads in your Netflix subscription, there is a large profit incentive to charge you for the "high-signal information feeds" and then... fill it with AI-generated content, which is much cheaper to offer.

¿Porqué no los dos?

Re: Open source AI is the path forward

#899

Post by sebastiennight »

sbeaks wrote: post #764
sensanaty wrote: post #718

Meanwhile Facebook is flooded with AI-generated slop with hundreds of thousands of other bots interacting with it to boost it to whoever is insane enough to still use that putrid hellhole of a mass-data-harvesting platform. Dead internet theory is very much happening in real time, and I dread what's about to come since the world has collectively decided to lose their minds with this AI crap. And people on this site a…

Are we going to end up with rererecapture where to post you need something on your phone/laptop measuring typing speed and scanning your face with an IR cam? Or a checkmark showing typed out by hand? Wouldn't get rid of ai content but may slow down bots posting it.

I got so tired of having to prove I'm a human, I wrote a bunch of prompts for multimodal LLMs to solve CAPTCHAs for me.

Re: Open source AI is the path forward

#900

Post by rmbyrro »

maxdo wrote: post #842
rmbyrro wrote: post #837

Earlier quoted context omitted.

You think Russia and China wouldn't be able to steal any closed model for a couple million dollars?

stealing a model and building the entire AI community is a very very big difference : Fine tune, update, Run model without very deep domain knowledge, that's what we receive as an outcome. If you are a software engineer and you steal a model in some close format of Open AI , you will not get lots of benefits even if you understand the format of that model, it's a complex beast by all means. This is a playbook how any…

If you think Chinese engineers can't deploy and productize (even weaponize) LLM models in production with the same ease as Western engineers, I think you're talking about last century China.
Post reply on HN