Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Emad Mostaque resigned as CEO of Stability AI

stability.ai

Post reply on HN
Print view View original

401–410 of 429 posts

Re: Emad Mostaque resigned as CEO of Stability AI

#401

Post by [deleted] »

bawana wrote: post #296
AnthonyMouse wrote: post #212

Earlier quoted context omitted.

You can get a decent approximation for LLM performance in tokens/second by dividing the model size in GB by the system's memory bandwidth. That's assuming it's well-optimized and memory rather than compute bound, but those are often both true or pretty close. And "depending on the task" is the point. There are systems that would be uselessly slow for real-time interaction but if your concern is to have it process con…

Why would increasing memory bandwidth reduce performance? You said "You can get a decent approximation for LLM performance in tokens/second by dividing the model size in GB by the system's memory bandwidth"

[deleted]

Re: Emad Mostaque resigned as CEO of Stability AI

#402

Post by jacky2wong »

huytersd wrote: post #383
jacky2wong wrote: post #256

I think Stability is in an interesting situation. A few suggestions on its direction and current state: 1. Stability AI's loss of talent at the foundational research layer is worrying. They've lost an incredibly expensive moat and there's enough unsolved problems in the foundation layer (faster models, more energy efficient models, etc.) to ensure Stability provides differentiated offerings. Step 1 should be rectifyi…

They tried to ponder to the open source CEO’s but as much as open source is an ideal, it’s a pretty sure way to failure for the most part. The only way open source works is if a rich company open sources parts of their non revenue forming items.

Yes - open source is a good go to market strategy. They should not open source if there's no good, strong business strategy around how they plan to commercialize. But there are many ways to commercialize an open source model. For many it's lead generation into a paid offering or a managed offering of the open source model.

Re: Emad Mostaque resigned as CEO of Stability AI

#403

Post by AnthonyMouse »

FireBeyond wrote: post #337
AnthonyMouse wrote: post #144

Earlier quoted context omitted.

You can get registered DDR4 for ~$1/GB. A trillion parameter model in FP16 would need ~2TB. Servers that support that much are actually cheap (~$200), the main cost would be the ~$2000 in memory itself. That is going to be dog slow but you can certainly do it if you want to and it doesn't cost $50,000.

Even looking on Amazon, DDR4 seems still a decent bit above $2/GB: 2 x 32GB: $142 2 x 64GB: $318 8GB: $16 2 x 16GB: $64 2TB of 128GB DDR4 ECC: $9,600 ( https://www.amazon.com/NEMIX-RAM-Registered-Compatible-Mothe... ) > Servers that support that much are actually cheap (~$200) What does this mean? What motherboards support 2TB of RAM at $200? Most of them are pushing $1,000. With no CPU. It may not hit $50K, but it's…

Here's a server that supports 3TB of memory for $130, you get 3TB by filling all 24 memory slots with 128GB LRDIMMs, 2TB with 16:

https://www.ebay.com/itm/176298520843

Here are 128GB LRDIMMs for $98:

https://www.ebay.com/itm/196305803969

For 2TB and the server you're at $1698. You can get a drive bracket for a few bucks and a 2TB SSD for $100 and have almost $200 left over to put faster CPUs in it if you want to.

That's stinking Optane, would work if you're desperate. Normal 128GB LRDIMMs cost more than other DDR4 DIMMs. You can, however, get DDR4 RDIMMs for ~$1/GB:

https://www.ebay.com/itm/186345903230

With 32GB RDIMMs that machine would max out at 768GB, which could still run a 1T model at q4 or grok at FP16. And then it would cost less than $1000.

Or find a quad-socket system with 48 memory slots and then use 64GB LRDIMMs ($1.12/GB):

https://www.ebay.com/itm/176299295509

The quad socket systems aren't $200, but you can find them for $550 or so:

https://www.newegg.com/hp-proliant-rack-mount/p/2NS-0006-3E5...

Maybe less if you shop around (they're not as common).

Re: Emad Mostaque resigned as CEO of Stability AI

#404

Post by AnthonyMouse »

bawana wrote: post #296
AnthonyMouse wrote: post #212

Earlier quoted context omitted.

You can get a decent approximation for LLM performance in tokens/second by dividing the model size in GB by the system's memory bandwidth. That's assuming it's well-optimized and memory rather than compute bound, but those are often both true or pretty close. And "depending on the task" is the point. There are systems that would be uselessly slow for real-time interaction but if your concern is to have it process con…

Why would increasing memory bandwidth reduce performance? You said "You can get a decent approximation for LLM performance in tokens/second by dividing the model size in GB by the system's memory bandwidth"

Yeah the sentence is backwards, you divide the system's memory bandwidth by the size of the model.

Re: Emad Mostaque resigned as CEO of Stability AI

#405

Post by Sn0wCoder »

pama wrote: post #353
alephnerd wrote: post #339

Earlier quoted context omitted.

> flood of money going into building new models In my experience, there really hasn't been that significant of a flood of money in this space for several years now, or at least not to the level I've seen based on discussion here on HN. I think HN tends to skew towards conversations around models for some reason, but almost all my peers are either funding or working on either tooling or ML driven applications since 20…

I agree that the quality of the posts has decayed dramatically since the start of the COVID pandemic and lots of the Reddit type memes and upvotes have added horrible levels of noise. I can still find gems in it and it’s still miles ahead of Twitter, but I do question my time using it passively and would much rather have a smaller and more focused community again.

Been on HN off and on for about 9 years with different accounts. I do believe the comments are getting more difficult to parse (find the good vs noise) but still the best site I have found - is hand curated links and people willing to offer solid advice given a question in the comments. I use Reddit from time to time but only for pure time killing purposes. I have learned more from the comments on HN than any course, book or (other) web site. Hopefully it will stay that way for years to come. A smaller community could offer more quality responses but less chance of finding the person who has the answer to your question.

Re: Emad Mostaque resigned as CEO of Stability AI

#406

Post by ajkjk »

ilaksh wrote: post #128
technics256 wrote: post #80

Looks like he went to crypto https://x.com/sreeramkannan/status/1771340250801127664?s=46

Decentralized systems, peer to peer, Blockchain, smart contracts, are all important technologies with real use cases. It is not accurate to refer to any of them as simply "crypto" especially in this context.

Are they now

Re: Emad Mostaque resigned as CEO of Stability AI

#407

Post by nabakin »

nsagent wrote: post #286
nabakin wrote: post #262

Earlier quoted context omitted.

I don't think MoE allows for that either. You'd have to come up with a whole new architecture that allows parts to be trained independently and still somehow be merged together in the end.

This paper addresses that issue and allows fully independent training of the experts: https://arxiv.org/abs/2303.14177

Cool paper. It's more independent than dense or normal MoE but I think it's still far away from the distributed training you're looking for because you still need a seed LM which is trained normally and when fine-tuning each expert from the seed LM, you still need enough GPUs or VRAM to fine-tune that LLM so you're still limited to large GPU clusters which is the problem we're trying to avoid.

In the case of the paper, they are using OPT-6.7b as the seed LM which requires 8xV100 GPUs for fine-tuning each expert. That's a combined total of 256GB of VRAM for a single expert while the 3090 only has 24GB of VRAM and is still one of the most expensive GPUs out there.

Maybe we could use something like PEFT or QLoRA in combination with this technique to make each expert small enough for the community to fine-tune and make a worse Mixtral 8x7b, but I don't know enough to say for sure.

Or maybe it turns out we can make a good MoE model with thousands of smaller experts. Experts small enough for a separate member of the community to independently fine-tune on a normal GPU, but idk.

To have both a performant and distributed LLM trained from scratch, we still need a completely different architecture to do it, but this work is pretty cool and may mean that if nothing else, there is something the community can do to help move things forward.

Also, I was going to say the MoE routing on this technique was lacking, but I found a more recent paper[0] by Meta which fixes this with a final fine-tuning stage.

[0] https://arxiv.org/abs/2403.07816

Re: Emad Mostaque resigned as CEO of Stability AI

#408

Post by KETpXDDzR »

> Earlier today, Emad Mostaque resigned from his role as CEO of Stability AI and from his position on the Board of Directors of the company to pursue decentralized AI.

Reading explanations of "decentralized AI" [0] sounds like a sales pitch for investors that missed out on the AI and crypto hype. Technically, it sounds like a distributed file storage which already exists.

[0] https://www.forbes.com/sites/digital-assets/2024/02/24/decen...

Re: Emad Mostaque resigned as CEO of Stability AI

#409

Post by bruce511 »

immibis wrote: post #381
immibis wrote: post #190

Earlier quoted context omitted.

That's right. We invented programming AI a very long time ago, and called it an "assembler". All you had to do was tell the assembler what kind of program you wanted, and it would do the programming work for you! Then we invented another AI to tell the assembler what kind of program you wanted, and called it a "compiler". All you had to do was tell the compiler what kind of program you wanted it to tell the assembler…

P.S. Visual Basic with its GUI designer was a quite effective way to rapidly build apps of questionable quality but great business value. Somebody should bring that paradigm back.

The paradigm never left :)

My day job is programming in an environment which originated in the mid 90s. A contemporary of the Visual Basic era, but somewhat more powerful, and requiring substantially less code.

While I, and a few thousand others still use it (and it gets updated every couple years or so) it has never been fashionable. Ironically because it's perceived as 'not real programming'.

We routinely build systems with hundreds of thousands of lines of code, much of it founded in the 90s and having been added to for 25 years. Most of it was built, and worked on, by individuals, or very small teams. Much of it today is still active doing the boring business software that keep the lights on.

But its not "main stream" because programmers pick language based on popularity, and enterprises pick programmers based on language. A self-fueling cycle of risk aversion.

A lucky few though hot off the treadmill a long time ago and "followed a path less travelled by". And that has made all the difference.

Re: Emad Mostaque resigned as CEO of Stability AI

#410

Post by ben_w »

numpad0 wrote: post #397
ben_w wrote: post #358

Earlier quoted context omitted.

> The point is, generative AI images are not widely regarded as good art. They're often seen as passable for some filler use cases and hard to tell apart from human generations, but not "good". > It's not not-there-yet because AI sometimes generates sixth fingers, it's something another level from Gustav Klimt My point is: yes AI is different — it's better . (Or, less provocatively: better by my specific standards ).…

> and yet to my eye it messes with anatomy wait what? Isn't that missing the point of expressionism? Klimt's Judith I is basically a photo, surely he can draw sh*t if he wanted to? But myriad predecessors such as Vermeer, Rembrandt, Van Gogh, da Vinci, et al., have done enough in realism, and also photography was becoming more viable and more prevalent, that artists basically started diversifying? Isn't that what lea…

I think you're conflating "good" in the sense of "competent" with "good" in the sense of "ethical" or "legal".

I am asserting here that the AI is (at its best) more competent, not any of the other things.

I suspect that the law will follow the economics, just as it often has done for everything else before — you're communicating with me via a device named after the job that the device made redundant ("computer").

But I said "often" not "always", because the business leaders ignoring the workers they were displacing 200 years ago led to riots, and eventually to the Communist Manifesto. I wouldn't discount this repeating.

--

I've just looked up "Judith I" (I recognise the art, just not the name), and I don't even understand why you're holding this up as an example of "basically a photo".

As for the other artists demonstrating realism: photography made realism redundant despite being initially dismissed as "not real art". Artists were forced to diversify, because a small box of chemistry was allowing unskilled people do their old job faster, cheaper, and better. Photography only became an art in its own right when people found ways to make it hard, for example by travelling the world and using it to document their travels, or with increasingly complex motion pictures.

I suspect that art fulfils the same role in humans as tails fulfil in peacocks: an expensive signal to demonstrate power, such that the difficulty is the entire point and anything which makes it easy is seen as worse than not even trying. This is also why forgeries are a big deal, instead of being "that's a nice picture", and why an original painting can retain a high price despite (or perhaps because of) a large number of extremely cheap prints being plastered onto everything from dorm rooms to chocolate wrappers.

Post reply on HN