Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Emad Mostaque resigned as CEO of Stability AI

stability.ai

Post reply on HN
Print view View original

371–380 of 429 posts

Re: Emad Mostaque resigned as CEO of Stability AI

#371

Post by stale2002 »

samatman wrote: post #361
stale2002 wrote: post #159

Earlier quoted context omitted.

> This is one of the worst advices I've ever seen. Really? Because stability AI caused a very large amount of good in the world. It arguably kicked off the entire AI startup industry. > Tricking people to obtain money? How's this not fraud? Its not fraud because you don't have to lie to anyone. You can tell VCs exactly what you plan on doing. Which is to open source all of your code... and... uhhh... yeah that will t…

The crux is that I doubt they intended this. A company that sets out to obtain VC money and blow it all on open source software without turning a profit, is going to leave behind smoking guns. Those will turn up in discovery and make one's life rather difficult.

> The crux is that I doubt they intended this.

Sure they did. They have been open from the start that they were releasing everything open source. They have been very up front about that!

> is going to leave behind smoking guns. Those will turn up in discover

No it won't, and it didn't. The VCs all hopped on board onto a very transparent open source giveaway. Good on them! Nobody lied to anyone about their open source plans.

Re: Emad Mostaque resigned as CEO of Stability AI

#372

Post by gbear605 »

polygamous_bat wrote: post #245
kalkr wrote: post #219

Earlier quoted context omitted.

decentralization != coinshit

https://en.wikipedia.org/wiki/Emad_Mostaque Read his Wikipedia page and tell me he doesn’t sound like your run of the mill crypto scammer. > He claims that he holds B.A. and M.A. degrees in mathematics and computer science from the University of Oxford.[7][8] However, according to him, he did not attend his graduation ceremony to receive his degrees, and therefore, he does not technically possess a BA or an MA.[7]

I didn’t go to Oxford, but going to your graduation ceremony isn’t usually a requirement for possessing a BA. The university just mails your diploma to you.

Re: Emad Mostaque resigned as CEO of Stability AI

#373

Post by radarsat1 »

radarsat1 wrote: post #148
qp11 wrote: post #123

Earlier quoted context omitted.

Pretty much what happened with Speech Recognition for 30 years. That last 10% had to be handled manually. Even if you get 90% right, it still means ever second sentence has issues. And as things scale up the costs of all that manual behind the scenes hacking scale up too. We underestimated how many issues involved Ambiguity - where N people see the same thing and have N different interpretations. So you see a whole b…

We use Whisper for automatic translation, supposedly SotA, but we have to fix its output, I would say, very often. It repeats things, translates things for no reason, has trouble with numbers.. it's improved in leaps and bounds but I'd say that speech recognition doesn't seem to be there yet.

(Can't edit anymore, but I meant "automatic transcription" above..)

Re: Emad Mostaque resigned as CEO of Stability AI

#374

Post by mirekrusin »

nsagent wrote: post #285
mirekrusin wrote: post #165

Earlier quoted context omitted.

SETI made sense because there is a lot of data where you download chunk and do expensive computation and return thin result. Model training is unlike that. It's large state that is constantly updated and updates require full, up to date state. This means you cannot distribute it efficiently over slow network with many smaller workers. That's why NVIDIA is providing scalable clusters with specialized connectivity so t…

You're right that parameter updates typically require full state, but during my PhD I've explored some possibilities to address this limitation (unfortunately, my ideas didn't pan out in the time I had). That said, there is research that has explored this topic and made some progress, such as this paper: https://arxiv.org/abs/2303.14177

Unfortunately it's hardly progress. MoE expert models are still large, have to be trained in usual, linear way, this approach requires training set classification upfront, each expert model is completely independent, each has to relearn concepts, your overall model is as good as dedicated expert, scale is in low numbers ie. 8, not thousands (otherwise you'd have to run inference on beefed up cluster only, experts still have to be loaded when used) etc.

Re: Emad Mostaque resigned as CEO of Stability AI

#375

Post by mirekrusin »

nsagent wrote: post #286
nabakin wrote: post #262

Earlier quoted context omitted.

I don't think MoE allows for that either. You'd have to come up with a whole new architecture that allows parts to be trained independently and still somehow be merged together in the end.

This paper addresses that issue and allows fully independent training of the experts: https://arxiv.org/abs/2303.14177

I wouldn't call peer network of 8 "distributed learning" in the sense we're talking about here.

Re: Emad Mostaque resigned as CEO of Stability AI

#377

Post by mirekrusin »

tkgally wrote: post #287
nsagent wrote: post #286

Earlier quoted context omitted.

This paper addresses that issue and allows fully independent training of the experts: https://arxiv.org/abs/2303.14177

This one, from a couple of days ago, might address that issue as well: https://arxiv.org/abs/2403.13187

This approach works on merging finetunes.

Base model was still trained in usual, non distributed way (by far the most cost).

Fine tunes were also trained in usual, non distributed way.

Proposed approach tries out several combinations to pick one that seems to perform better (where combination means ie. adhoc per layer operation).

Merging is not distributed as well.

There is not much distribution happening overall beyond the fact that fine tunes were trained independently.

Taking weight averages, weighted weight averages, trimming low diffs, doing arithmetic (subtracting base model from fine tune) etc. are all ad hoc trials throwing something on the wall and seeing what sticks the most. None of those work well.

For distributed training to work we'd have to have better algebra around this multidimentional/multilayer/multiconnectivity state. We don't have it and it has many problems, ie. evaluation is way too expensive. But solving "no need to rerun through whole training/benchmark corpus to see if my tiny change is better or not" problem will mean we solved problem of extracting essence of intelligence. If we do that, then hyper-efficient data centers will still keep beating out any distributed approach and it's all largely irrelevant because that's pure AGI already.

Re: Emad Mostaque resigned as CEO of Stability AI

#378

Post by johnnyanmac »

DrSiemer wrote: post #267
johnnyanmac wrote: post #202

Earlier quoted context omitted.

>It won't be decades, but a few more years would be helpful I see it this way to be honest: - companies will aggresively try to use AI in the next 2-3 years, downsizing themselves in the meantime - the 3-5 year launch mark will show that downsizing was an awful idea and took too many hits to really be worth it. I don't know if those hits will be in profits (depends on the company) but it will clearly hit that uncanny…

Companies are aware of the current AI generation being a tool and not a full replacement (or they will be after the first experiments they perform). They will not downsize, they will train their workforce or hire replacements that are willing to pick up these more powerful and efficient tools. In the hands of a skilled professional there will be no uncanny valley. This will result in surplus funds, that can be invest…

You greatly overestimate how quickly companies learned. Outsourcing has been a thing for decades and to this day some people are still trying to do it to cut costs. I guess that's what happens when you don't value retention nor document previous decisions. You repeat the cycle.

Sure, the smart companies will use it as a tool, but most companies aren't smart, or just don't care. It'll vary by industry. There is already talks of sizing down VFX/Animation for a mix of outsourcing and AI reliance, for example. And industry that already underpays its artists.

>Not allowing copyright on AI generated work is a ridiculous and untenable decision that will be overturned eventually.

Maybe, once the dust settles on who and what and how you copyright AI. It'll be a while, though. But I get the logic. No one can (nor wants to) succinctly explain what sources were used in a generative art work right now, and that generative process drives the art a lot more than the artist for most generative art. Even without AI there is a line between "I lightly edited this existing work on photosshop" and "I significantly altered a base template to the point where you can't recognize the template anymore" where copyright will kick in.

Still, my biased hopes involve them being very strict with this line. You can't just give 2 prompts and expect to "own" an artwork.

Re: Emad Mostaque resigned as CEO of Stability AI

#379

Post by dagelf »

johnnyanmac wrote: post #202
Satam wrote: post #18

The underlying reason is that current AI businesses and models are failing to capture any significant economic value. We're going to get there, but it will take some more work. It won't be decades, but a few more years would be helpful

>It won't be decades, but a few more years would be helpful I see it this way to be honest: - companies will aggresively try to use AI in the next 2-3 years, downsizing themselves in the meantime - the 3-5 year launch mark will show that downsizing was an awful idea and took too many hits to really be worth it. I don't know if those hits will be in profits (depends on the company) but it will clearly hit that uncanny…

Models are already general, and you can set them to recursively self improve narrowly and still need humans in the loop for general improvement... but those humans are only going to get less... so change your years into months instead and multiply by rand(2)... and change the hire-backs into more startups doing more things...?

Re: Emad Mostaque resigned as CEO of Stability AI

#380

Post by alephnerd »

monero-xmr wrote: post #369
alephnerd wrote: post #366

Earlier quoted context omitted.

> government will surely ban it This is plain hystronics, but regulation is absolutely coming and will only help the larger existing players to consolidate. Coinbase, FTX, etc have all seen the writing on the wall and are working with regulators on this. The Wild West days are definetly over now. If I'd invest in the space, I'd probably look at KYC and AML product opportunties such as Chainalysis, but even that space…

I’m super deep in the space but there is enormous money to be made everywhere. Or at least everyone I know is searching Zillow…

I'm not denying that there's money in the industry - every portion of the software industry can rake cash due to the extremely high margins that software has compared to just about any other industry.

The magic is finding which subsegments might have even higher margins than others.

Crypto used to have fairly high margins, but all the easy gains have been claimed by larger firms as it's a much more mature industry now, but portions of the ML space still have much more opportunity for growth, so it makes sense to deploy capital there (this is a very high level view so take with a grain of salt - B2C and SMB B2B and Enterprise B2B have entirely different GTM motions and path to profitability).

But at least for me, I can't justify participating in a Series A round for a crypto startup compared to an MLOps startup today.

Post reply on HN