Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Emad Mostaque resigned as CEO of Stability AI

stability.ai

Post reply on HN
Print view View original

411–420 of 429 posts

Re: Emad Mostaque resigned as CEO of Stability AI

#411

Post by kranke155 »

Hendrikto wrote: post #272
achow wrote: post #266

Earlier quoted context omitted.

> Deep UX wrappers for image editing and video editing while owning the end to end stack for image generation or video generation would be a great focal point for Stability that separates itself from the competition. People don't pay for images, they pay for images that solves their problems. Recently, during an interview [1], when questioned about OpenAI's Sora, Shantanu Narayen (Adobe CEO) gave an interesting persp…

> Not only Adobe perhaps knows better than anyone else what is need and workflow of a moviemaker I only ever heard creatives complain about Adobe and their UI/UX and how they don’t understand their customers. Never really used any of their products myself though. Maybe they still are best-in-class. I can’t tell.

I work with Adobe products everyday. They are mostly trash, at least in the video arena. People use them but they are painful to use. Premiere is known for poor stability, After Effects has just started being updated again after maybe 10+ years of nothing useful being added. Some parts of After Effects are clearly bolted on, and incompatible with each other.

They’re terrible products for the most part. But they buy the competition (Substance) or they develop some half baked substitute (Adobe XD) that people might use since it comes with the subscription.

Re: Emad Mostaque resigned as CEO of Stability AI

#412

Post by razodactyl »

cchance wrote: post #393
JCM9 wrote: post #330

Earlier quoted context omitted.

Yes. Each query needs to generate enough revenue to pay for the cost of running the query, a proportional cost of the cost to train the original model, overhead, SG&A, etc. just to break even. Few have shown a plan to do that or explained in a defensible way how they’re going to get there. A challenge at the moment is a lot of the AI movement is led by folks that are brilliant technologists but have little to no expe…

Inference will continue to shift toward profitability, Google and the rest are going to choke the cost to gain traction but as TPU costs come down and GPU costs scale up and models become more efficient the cost to infer will drastically scale down into profitability.

This comment made me laugh because I thought of NVIDIA during the crypto boom and now the AI boom.

They played their cards so damn right when deep learning was taking off.

Re: Emad Mostaque resigned as CEO of Stability AI

#413

Post by razodactyl »

oldgradstudent wrote: post #297
andy99 wrote: post #275

Earlier quoted context omitted.

I don't know Adobe's business so could be wrong, but maybe "creatives" are not their key customers? If they're focusing on enterprise sales, they're selling to enterprise decision makers. Every user hates using microsoft products, and don't get me started on SAP. But these are gigantic companies with wildly successful products aimed at enterprise customers.

> Every user hates using microsoft products Only because they've never had a chance to experience the competition. Having worked in IBM and had to use the Lotus Office Suite I can tell you Microsoft won fair and square. And I'm not even talking about the detestable abomination that is Lotus Notes.

This comment distracted me as I tried to recall a product I saw my mother using when I was younger. I fell asleep after I found it and never got back here until just now: Microsoft Works

This one was so obscure to find because it seems to exist in a weird space of being Microsoft Office but not.

Re: Emad Mostaque resigned as CEO of Stability AI

#414

Post by mirekrusin »

mejutoco wrote: post #226
mirekrusin wrote: post #165

Earlier quoted context omitted.

SETI made sense because there is a lot of data where you download chunk and do expensive computation and return thin result. Model training is unlike that. It's large state that is constantly updated and updates require full, up to date state. This means you cannot distribute it efficiently over slow network with many smaller workers. That's why NVIDIA is providing scalable clusters with specialized connectivity so t…

That is the challenging part indeed. But if we think of mixture of experts models outperforming "monolithic" models, why not? Maybe instead of 8 you can do 1000 and that is easy to paralellize. It sounds worth exploring to me.

Unfortunately it doesn't work like that.

During MoE training you still need access to all weights.

1k experts would mean 30 TB of state to juggle with on 7B params. Training and inference is infeasible at this size.

If you'd want to keep the size while increasing number of experts, you'd end up with 7b -> 56m model. What kind of computation can you do on 56m model? Remember that expert model in MoE runs the whole inference without consulting or otherwise reusing any information from other experts. Thin network at the top just routes it to one of experts. But at this small size those are not "experts" anymore, it'd be more like Mixture of Idiots.

To put it in other way, MoE is optimization technique with low scaling ceiling that is more local maximum solution that global one (this idea works against you quickly if you want to go more that direction).

Re: Emad Mostaque resigned as CEO of Stability AI

#415

Post by sillysaurusx »

baobabKoodaa wrote: post #387
_xivi wrote: post #268

Earlier quoted context omitted.

Sorry I'm not well versed in the story and it's still not clear to me what he did wrong. Where is the controversy here? Is the CEO expected to contribute to research? there seems to be some context I'm missing

You made a point about devs underappreciating the work of other professionals, like CEOs and designers. You made this point in the context of Emad and the success story that is Stable Diffusion. The implication in your point is that Emad surely contributed to the success, even though the CEO is not a developer or researcher. My counter to your point is that Emad wasn't there when the inventions were made. He joined t…

This is 100% correct. Emad tried to lure me into the trap that Eleuther eventually fell into, and I lucked out by blowing him off after getting weird vibes from him. This was back when he was unknown, but was running around offering a bunch of researchers massive GPU cluster time for seemingly altruistic reasons but were in fact creepy reasons. In reality he wanted his name on their work.

I have the DMs to prove this, and have not ever said something like this about someone. I wouldn’t make this accusation lightly, for whatever it’s worth. In the HN discussion of that article, I had left a comment, which Emad DMed me on Twitter about, saying no no, he never lied to investors, and tried to convince me that what I was saying wasn’t true. I was wondering why he cared so much. In retrospect it’s probably because it was correct.

I’ve never worked with him, to be clear, and the few colleagues who have worked at Stability have had generally positive things to say. But there was one that was screwed over by them hard (he was doing contract work, and never got paid for it), and I can think of at least four other alarming data points that all point to the same thing.

It’s unsettling not knowing whether to speak up about this. On one hand it doesn’t really matter that much. On the other hand, it’s the fundamental difference between a CEO that tends to IPO vs one that tends to fail. I hate seeing people fail, and I genuinely thought that my feelings about Emad were mistaken since empirically they were doing fine. Turns out, nope, not fine, and the original impression was right. Weird experience.

Re: Emad Mostaque resigned as CEO of Stability AI

#416

Post by razodactyl »

Takennickname wrote: post #222
llm_trw wrote: post #138

Earlier quoted context omitted.

The `big' AI models are trillion parameter models. The medium sized models like GPT3 and Grok are 185b and 314b respectively. There is no way for _anyone_ to run these on a sub $50k machine in 2024, and even if you can the token generation speed on CPU is under 0.1 tokens per second.

I will make a 2 trillion parameter model just so your comment becomes outdated and wrong.

I approve this comment.

Re: Emad Mostaque resigned as CEO of Stability AI

#417

Post by razodactyl »

GaggiX wrote: post #157
speedgoose wrote: post #146

Earlier quoted context omitted.

To be precise, ChatGPT 3.5 turbo being 20B is officially a mistake from a Microsoft Researcher, quoting a wrong source published before the release of chatgpt3.5 turbo. Up to you to believe it or not. But I wouldn’t claim it’s a 20B according to Microsoft Researchers. The withdrawn paper: https://arxiv.org/abs/2310.17680 The wrong source: https://www.forbes.com/sites/forbestechcouncil/2023/02/17/is... The discussion:…

It's interesting how the paper was completely retracted instead of just being corrected.

Yep. It feels like a 20B parameter model.

Re: Emad Mostaque resigned as CEO of Stability AI

#418

Post by razodactyl »

HarHarVeryFunny wrote: post #347
GaggiX wrote: post #140

Earlier quoted context omitted.

ChatGPT is 20B according to Microsoft researchers, also the fact that big AI models are trillion parameter models is mostly speculation, about GPT-4 it was spread by geohot.

GPT-3 was 175B, so it'd be a bit odd if GPT-4 wasn't at least 5x larger (1T), especially since it's apparently a mixture of experts.

I think it became apparent when mixtral came out. I've noticed too during training that my model overwrites useful information so it makes sense for these types of models to have emerged.

Re: Emad Mostaque resigned as CEO of Stability AI

#419

Post by razodactyl »

livueta wrote: post #49
BigJono wrote: post #37

Earlier quoted context omitted.

I don't think it's that SD and LLMs are solutions looking for problems, it's that there are very clear problems to which they provide 90% of a solution and make it impossible to clear the last 10%. They're the new WYSIWYG/low-code. Everyone that doesn't fully understand the problem space thinks they're some ultimate solution that is going to revolutionise everything. People that do are responding with a resounding 'm…

And even a lot of the hentai is fucking worthless for the same reasons! Try generating some kinbaku. It's really hard to get something where all the rope actually connects and interacts sensibly because it doesn't actually know what a knot is. Instead, you end up with M. C. Escher: Fetish Edition.

I can shed some light on this phenomena: These models are trained on many images but no thought is put into the "generalisation" aspect the ML community was so obsessed with during the deep-learning era.

Whatever Kinbaku is... haha

I think a key part that's missing currently is the agent training approach: https://youtu.be/v3UBlEJDXR0?si=8w4Jt0bNEBfIXkZl

It's very easy to create a "Stochastic Parrot" but I'm quite sure these models are capable of learning underlying information such as correct layout of a knot - given the right data and curriculum of course. Maybe slight architecture tweaks.

I'm sure this is the reason we're starting to see a normal amount of fingers or ability to write text. Proof of concept was 2015 until 2022 now we're starting to see interesting things come out of the workshops.

Re: Emad Mostaque resigned as CEO of Stability AI

#420

Post by brrrrrm »

FeepingCreature wrote: post #290
brrrrrm wrote: post #288

Earlier quoted context omitted.

Cranking up the batch size kills convergence.

Wonder if that can be avoided by modifying the training approach. Ideas offhand: group by topic, train a subset of weights per node; figure out which layers have the most divergence and reduce lr on those only.

A provable way to recover convergence is to calculate the hessian. It’s computationally expensive but there are approximation methods.
Post reply on HN