Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DBRX: A new open LLM

databricks.com

Post reply on HN
Print view View original

121–130 of 360 posts

Re: DBRX: A new open LLM

#121

Post by hintymad »

ramoz wrote: post #107
hintymad wrote: post #99

Just curious, what business benefit will Databricks get by spending potentially millions of dollars on an open LLM?

Their goal is to always drive enterprise business towards consumption. With AI they need to desperately steer the narrative away from API based services (OpenAI). By training LLMs, they build sales artifacts (stories, references, even accelerators with LLMs themselves) to paint the pictures needed to convince their enterprise customer market that Databricks is the platform for enterprise AI. Their blog details how th…

Thanks! Why do they not focus on hosting other open models then? I suspect other models will soon catch up with their advantages in faster inference and better benchmark results. That said, maybe the advantage is aligned interests: they want customers to use their platforms, so they can keep their models open. In contrast, Mistral removed their commitment to open source as they found a potential path to profitability.

Re: DBRX: A new open LLM

#122

Post by ml_hardware »

djoldman wrote: post #4

Model card for base: https://huggingface.co/databricks/dbrx-base > The model requires ~264GB of RAM I'm wondering when everyone will transition from tracking parameter count vs evaluation metric to (total gpu RAM + total CPU RAM) vs evaluation metric. For example, a 7B parameter model using float32s will almost certainly outperform a 7B model using float4s. Additionally, all the examples of quantizing recently releas…

Looks like someone has got DBRX running on an M2 Ultra already: https://x.com/awnihannun/status/1773024954667184196?s=20

Re: DBRX: A new open LLM

#123

Post by superdupershant »

saeleor wrote: post #81

looks great, although I couldn't find anything on how "open" the license is/will be for commercial purposes wouldn't be the first branding as open source going the LLaMA route

It's similar to llama2.

  > If, on the DBRX version release date, the monthly active users of the products
  > or services made available by or for Licensee, or Licensee’s affiliates, is
  > greater than 700 million monthly active users in the preceding calendar 
  > month, you must request a license from Databricks, which we may grant to you
  > in our sole discretion, and you are not authorized to exercise any of the
  > rights under this Agreement unless or until Databricks otherwise expressly
  > grants you such rights.
https://www.databricks.com/legal/open-model-license

Re: DBRX: A new open LLM

#124

Post by wrs »

swalsh wrote: post #77
djoldman wrote: post #4

Model card for base: https://huggingface.co/databricks/dbrx-base > The model requires ~264GB of RAM I'm wondering when everyone will transition from tracking parameter count vs evaluation metric to (total gpu RAM + total CPU RAM) vs evaluation metric. For example, a 7B parameter model using float32s will almost certainly outperform a 7B model using float4s. Additionally, all the examples of quantizing recently releas…

> The model requires ~264GB of RAM This feels as crazy as Grok. Was there a generation of models recently where we decided to just crank on the parameter count?

Isn’t that pretty much the last 12 months?

Re: DBRX: A new open LLM

#125

Post by m3kw9 »

These tiny “state of the art” performance increases are really indicative the current architecture for LLM(Transformers + Mixture of Experts) is maxed out even if you train it more/differently. The writings are on all over the walls.

Re: DBRX: A new open LLM

#126

Post by londons_explore »

patrick-fitz wrote: post #98

Looking at the license restrictions: https://github.com/databricks/dbrx/blob/main/LICENSE "If, on the DBRX version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Databricks, which we may grant to you in our sole discretion, a…

I do wonder what value those companies who have >700 million users might get from this?

Pretty much all of the companies with >700 million users could easily reproduce this work in a matter of weeks if they wanted to - and they probably do want to, if only so they can tweak and improve the design before they build products on it.

Given that, it seems silly to lose the "open source" label just for a license clause that doesn't really have much impact.

Re: DBRX: A new open LLM

#127

Post by stri8ed »

swalsh wrote: post #74
emmender2 wrote: post #46

this proves that all llm models converge to a certain point when trained on the same data. ie, there is really no differentiation between one model or the other. Claims about out-performance on tasks are just that, claims. the next iteration of llama or mixtral will converge. LLMs seem to evolve like linux/windows or ios/android with not much differentiation in the foundation models.

The models are commodities, and the API's are even similar enough that there is zero stickiness. I can swap one model for another, and usually not have to change anything about my prompts or rag pipelines. For startups, the lesson here is don't be in the business of building models. Be in the business of using models. The cost of using AI will probably continue to trend lower for the foreseeable future... but you can…

Or be in the business of building infrastructure for AI inference.

Re: DBRX: A new open LLM

#128

Post by adtac »

patrick-fitz wrote: post #98

Looking at the license restrictions: https://github.com/databricks/dbrx/blob/main/LICENSE "If, on the DBRX version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Databricks, which we may grant to you in our sole discretion, a…

Ironically, the LLaMA license text [1] this is lifted verbatim from is itself probably copyrighted [2] and doesn't grant you the permission to copy it or make changes like s/meta/dbrx/g lol.

[1] https://github.com/meta-llama/llama/blob/main/LICENSE#L65 [2] https://opensource.stackexchange.com/q/4543

Re: DBRX: A new open LLM

#129

Post by renewiltord »

SubiculumCode wrote: post #79
renewiltord wrote: post #70

Earlier quoted context omitted.

Yeah, this is why I ask climate scientists to use a proper 0 K graph but they always zoom it in to exaggerate climate change. Display correctly with 0 included and you’ll see that climate change isn’t a big deal. It’s a common marketing and fear mongering trick.

Where are your /s tags? The scale should be chosen to allow the reader to correctly infer meaningful differences. If 1° is meaningful in terms of the standard error/ CI AND 1° unit has substantive consequences , then that should be emphasized.

> Where are your /s tags?

I would never do my readers dirty like that.

Re: DBRX: A new open LLM

#130

Post by d_burfoot »

Tubbe wrote: post #83
jobigoud wrote: post #61

Earlier quoted context omitted.

It's even possible they converge when trained on different data, if they are learning some underlying representation. There was recent research on face generation where they trained two models by splitting one training set in two without overlap, and got the two models to generate similar faces for similar conditioning, even though each model hadn't seen anything that the other model had.

Got a link for that? Sounds super interesting

https://en.wikipedia.org/wiki/Theory_of_forms
Post reply on HN