Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Databricks Strikes $1.3B Deal for Generative AI Startup MosaicML

wsj.com

Post reply on HN
Print view View original

61–70 of 113 posts

Re: Databricks Strikes $1.3B Deal for Generative AI Startup MosaicML

#61

Post by stingraycharles »

throwaway54232 wrote: post #7
stingraycharles wrote: post #3

I can’t help but feel this is mostly hype driven by a company that’s looking to reinvent themselves while facing the prospect of irrelevance. The piece reads like it’s a press release, with abstract things such as “corporate leaders under pressure to get their data ready for AI”. That has nothing to do with LLMs, as the current AI “hype” cycle has been going on for almost a decade already. Corporate leaders have alwa…

This was my guess as well. I am wondering what you see as the "prospect of irrelevance" they are facing. I have my guess but I would like to hear your take.

People are doing more and more self-hosting and desire PaaS-like offerings that run in their own cloud, rather than SaaS “we manage your spark cluster for you” which is what DataBricks was founded upon. PaaS has significantly thinner margins, but it’s desirable from the customers’ perspective as it’s cheaper and much better from a (data) security point of view.

As such, they’ll be forced to do a lot more services oriented work rather than product / platform oriented work, because it pays well. Their sales team is also excellent. I see a similar fate for them as Cloudera.

Re: Databricks Strikes $1.3B Deal for Generative AI Startup MosaicML

#62

Post by quickthrower2 »

stingraycharles wrote: post #3

I can’t help but feel this is mostly hype driven by a company that’s looking to reinvent themselves while facing the prospect of irrelevance. The piece reads like it’s a press release, with abstract things such as “corporate leaders under pressure to get their data ready for AI”. That has nothing to do with LLMs, as the current AI “hype” cycle has been going on for almost a decade already. Corporate leaders have alwa…

Looking at the Mosaic website, and looking at many AI startups, I can't help feeling like they are all small AWS Sagemaker, Bedrock, Trainium, ... (etc.) competitors, and not sure how they will compete with the sheer capital that Amazon has, and potential to offer that compute power cheaper (at least long enough to kill off most of the competition). Maybe it is it the off chance that one of these companies might become a big player worth billions that makes the odds worth it?

Re: Databricks Strikes $1.3B Deal for Generative AI Startup MosaicML

#63

Post by swyx »

huge congrats to the team - we interviewed Jonathan and Abhi from the team last month ( https://www.latent.space/p/mosaic-mpt-7b#details ) and it was blindingly obvious that they were an incredible team of engineers building one of the most valuable training platforms in the industry. congrats!

Re: Databricks Strikes $1.3B Deal for Generative AI Startup MosaicML

#64

Post by CrimsonWanderer »

swyx wrote: post #63

huge congrats to the team - we interviewed Jonathan and Abhi from the team last month ( https://www.latent.space/p/mosaic-mpt-7b#details ) and it was blindingly obvious that they were an incredible team of engineers building one of the most valuable training platforms in the industry. congrats!

Good to hear they are responsive to PR opportunities. Too bad their interview process doesn't match this level of enthusiasm.

I had a recruiter reach out from DB late last year. I wasn't looking for new work but it seemed like it'd be worth a chat. I had to move my schedule around to fit it in and then get up extra early to be prepared for the chat...only the recruiter never showed. Didn't follow up about missing a meeting they had scheduled. It's been crickets. That is enough of a lack of professionalism really stood out as a red flag. Hard pass.

Re: Databricks Strikes $1.3B Deal for Generative AI Startup MosaicML

#66

Post by throwaway72762 »

Tostino wrote: post #47
brucethemoose2 wrote: post #34

Earlier quoted context omitted.

> thanks to existing FOSS work on stuff like PEFT and LoRA YMMV. Sometimes a LORA is fine, but sometimes a full finetune is necessary for higher quality output. That being said, backwards pass free training keeps making more and more progress. Seems like a short matter of time before it becomes practical.

Look at QLoRA. The QLoRA can be attached to all layers, allowing you to alter behavior with much less data than the original LoRA implementation. It seems to "stick" better. I just fine tuned a ~30b parameter model on my 2x 3090s to check it out. It worked fantastically. I should be able to fine tune up-to 65b parameter models locally but wanted to get my dataset right on a smaller model before trying.

Are there any repos and steps you can point to to do this? I'd love to try to do exactly what you describe. I have been trying to do the same and have run into a lot of repos with broken dependencies.

Re: Databricks Strikes $1.3B Deal for Generative AI Startup MosaicML

#67

Post by DebtDeflation »

poomer wrote: post #15

This class of startup, "build domain specific LLMS using your own data", is extremely crowded right now but I am not optimistic about their future. For large companies, the actual modeling work for this is already easy for any ML team, thanks to existing FOSS work on stuff like PEFT and LoRA. The hard part is figuring out what data goes into the fine tuning process and how to get this data in a usable form, but this…

> "build domain specific LLMS using your own data",

It seems to me that the vast majority of these people would be better off just doing semantic search with their documents chunked, run through an embeddings process, and stored in a vector database, with the search queries and results then run through an LLM at the final step to create an actual "answer". For applications where this is not practical, I agree that LoRA should be the next approach. I have a hard time believing that the future is everyone training their own domain specific LLMs from the ground up.

Re: Databricks Strikes $1.3B Deal for Generative AI Startup MosaicML

#68

Post by jamesblonde »

cfeduke wrote: post #52
jamesblonde wrote: post #40

Earlier quoted context omitted.

Databricks are not on-premises either.

> Databricks are not on-premises either. Usually, no. But in some cases, yes.

I must have missed the product announcement. Do you have a link, because everywhere I look I can't find it.

Re: Databricks Strikes $1.3B Deal for Generative AI Startup MosaicML

#69

Post by swyx »

underlines wrote: post #25

Coincidentally this comes after MosaicML released the best open source commercially usable LLMs on huggingface: mpt-30b, the first open source LLM with 8k context length that can be extended even further with ALiBi and has been trained on a whopping 1 trillion tokens vs. 300 billion for Pythia and OpenLLaMA, and 800 billion for StableLM.

do we know how it ranks vs the other models yet? its not yet up on https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...

Re: Databricks Strikes $1.3B Deal for Generative AI Startup MosaicML

#70

Post by hackitup7 »

jedberg wrote: post #65

I didn't realize Databricks was doing so well they could spare $1B in equity/cash. Congrats not only to MosaicML but Databricks as well!

Yeah, they're one of the strongest private software companies right now. I'd say easily top 10.
Post reply on HN