81–90 of 113 posts
I didn't realize Databricks was doing so well they could spare $1B in equity/cash. Congrats not only to MosaicML but Databricks as well!
Yeah, they're one of the strongest private software companies right now. I'd say easily top 10.
I can’t help but feel this is mostly hype driven by a company that’s looking to reinvent themselves while facing the prospect of irrelevance. The piece reads like it’s a press release, with abstract things such as “corporate leaders under pressure to get their data ready for AI”. That has nothing to do with LLMs, as the current AI “hype” cycle has been going on for almost a decade already. Corporate leaders have alwa…
Looking at the Mosaic website, and looking at many AI startups, I can't help feeling like they are all small AWS Sagemaker, Bedrock, Trainium, ... (etc.) competitors, and not sure how they will compete with the sheer capital that Amazon has, and potential to offer that compute power cheaper (at least long enough to kill off most of the competition). Maybe it is it the off chance that one of these companies might beco…
Mosaic is one of the better providers for this. AWS is nowhere near ready at this current point in time, it is pretty much a "dumb" infra provider in large LLM training at this point. (Of course they won't be standing still and will prob acquire that capability one way or another)
Falcon 7B/40B is good, but hasn't really caught on like LLaMA has. One big reason is that its not well supported by 4-bit inference code (namely llama.cpp and GPTQ). Mosaic's 7B (and 30B?) models have the same issue, and 7B kinda paled in comparison to LLaMA 7B... But maybe it would be better if finetuned? To me, its kinda baffling that Mosaic didn't work on adding highly quantized inference to the popular frameworks…
Mosaic's MPT models are already supported in GGML: https://github.com/ggerganov/ggml Here's MPT-30B running in 4-bit precision on CPU :) https://twitter.com/abacaj/status/1673133443339763712?s=20
Earlier quoted context omitted.
This was my guess as well. I am wondering what you see as the "prospect of irrelevance" they are facing. I have my guess but I would like to hear your take.
People are doing more and more self-hosting and desire PaaS-like offerings that run in their own cloud, rather than SaaS “we manage your spark cluster for you” which is what DataBricks was founded upon. PaaS has significantly thinner margins, but it’s desirable from the customers’ perspective as it’s cheaper and much better from a (data) security point of view. As such, they’ll be forced to do a lot more services ori…
Do you have some data to support this? This is a pretty bold claim.
Earlier quoted context omitted.
I don't know if I'd say this is reinventing themselves, or at least not from the ground up, they've been somewhat active in the open source LLM scene for some time now. Not that releasing dolly alone caused some major foundational shift, but it was a solid contribution. Additionally, acquiring Mosaic fills a pretty gaping hole in their existing product offering. The whole process of pretraining, fine-tuning, and eval…
From my understanding, Databrick's core value proposition is -- 'We let you do things you want to, with the people you have .' It may surprise HN to learn, but most companies don't have top-tier technical talent. Consequently, how do you do {cutting edge thing board and investors are demanding company do} with the people you have/can afford? Use an offering that decreases the necessary skill level by providing powerf…
Running your own Spark, especially on prem, is a lot of work. Most companies would prefer to just provide their data and let someone else handle the query engine.
The parent is right however that Databricks has a feature store (tokenization) but it's not simple to set up and just getting content in and out of it is a major pain point right now.
Earlier quoted context omitted.
> Databricks are not on-premises either. Usually, no. But in some cases, yes.
I must have missed the product announcement. Do you have a link, because everywhere I look I can't find it.
If you * need * it, you find a human to talk to (sales or connect from your network).
I can’t help but feel this is mostly hype driven by a company that’s looking to reinvent themselves while facing the prospect of irrelevance. The piece reads like it’s a press release, with abstract things such as “corporate leaders under pressure to get their data ready for AI”. That has nothing to do with LLMs, as the current AI “hype” cycle has been going on for almost a decade already. Corporate leaders have alwa…
Just curious, how is Databricks going to be irrelevant? I thought they were a reasonably powerful database provider for data analytics platforms in many companies?
Earlier quoted context omitted.
Look at QLoRA. The QLoRA can be attached to all layers, allowing you to alter behavior with much less data than the original LoRA implementation. It seems to "stick" better. I just fine tuned a ~30b parameter model on my 2x 3090s to check it out. It worked fantastically. I should be able to fine tune up-to 65b parameter models locally but wanted to get my dataset right on a smaller model before trying.
Are there any repos and steps you can point to to do this? I'd love to try to do exactly what you describe. I have been trying to do the same and have run into a lot of repos with broken dependencies.
Earlier quoted context omitted.
Their moat is their phenomenal sales team. Like, I'm at company 3 that pays for databricks. Amount of those companies that use Spark = 0. I've given up complaining now.
So what are they using, the notebooks?
Earlier quoted context omitted.
What has replaced Databricks, in your opinion? Doesn’t seem «dead» to me, yet. Serious question.
Their moat was to be able to spin up spark clusters for you. This was very useful when companies just starting in that space and don’t have the skills. But with K8s and clusters becoming more common for normal workload, it’s not hard to manage your own spark clusters anymore. There goes their moat. Their other offerings just aren’t that amazing. People now don’t just want to buy hand tools anymore, they want power to…