Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DBRX: A new open LLM

databricks.com

Post reply on HN
Print view View original

141–150 of 360 posts

Re: DBRX: A new open LLM

#141

Post by einarfd »

londons_explore wrote: post #126
patrick-fitz wrote: post #98

Looking at the license restrictions: https://github.com/databricks/dbrx/blob/main/LICENSE "If, on the DBRX version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Databricks, which we may grant to you in our sole discretion, a…

I do wonder what value those companies who have >700 million users might get from this? Pretty much all of the companies with >700 million users could easily reproduce this work in a matter of weeks if they wanted to - and they probably do want to, if only so they can tweak and improve the design before they build products on it. Given that, it seems silly to lose the "open source" label just for a license clause tha…

The point of the more than 700 million user restriction. Is so Amazon, Google cloud or Microsoft Azure. Can not setup an offering where they host and sell access to the model without an agreement with them.

This point is probably inspired by the open source software vendors that have switched license over competition from the big cloud vendors.

Re: DBRX: A new open LLM

#142

Post by mrtranscendence »

bboygravity wrote: post #131

Less than 1 week after Nancy Pelosi bought a 5M USD share in Databricks, this news is published. https://twitter.com/PelosiTracker_/status/177119703064106223... Crime pays in the US.

Are you alleging that Nancy Pelosi invested in Databricks, a private company without a fluctuating share price, because she learned that they would soon release a small, fairly middling LLM that probably won't move the needle in any meaningful way?

Re: DBRX: A new open LLM

#143

Post by bevekspldnw »

emmender2 wrote: post #46

this proves that all llm models converge to a certain point when trained on the same data. ie, there is really no differentiation between one model or the other. Claims about out-performance on tasks are just that, claims. the next iteration of llama or mixtral will converge. LLMs seem to evolve like linux/windows or ios/android with not much differentiation in the foundation models.

The big thing for locally hosted is inference efficiency and speed. Mistral wears that crown by a good margin.

Re: DBRX: A new open LLM

#144

Post by nabakin »

patrick-fitz wrote: post #98

Looking at the license restrictions: https://github.com/databricks/dbrx/blob/main/LICENSE "If, on the DBRX version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Databricks, which we may grant to you in our sole discretion, a…

Also aren't claiming they are the best LLM out there when they clearly aren't like Inflection. Overall solid

Re: DBRX: A new open LLM

#145

Post by bevekspldnw »

PheonixPharts wrote: post #21
XCSme wrote: post #11

I am planning to buy a new GPU. If the GPU has 16GB of VRAM, and the model is 70GB, can it still run well? Also, does it run considerably better than on a GPU with 12GB of VRAM? I run Ollama locally, mixtral works well (7B, 3.4GB) on a 1080ti, but the 24.6GB version is a bit slow (still usable, but has a noticeable start-up time).

While GPUs are still the kings of speed, if you are worried about VRAM I do recommend a maxed out Mac Studio. Llama.cpp + quantized models on Apple Silicon is an incredible experience, and having 192 GB of unified memory to work with means you can run models that just aren't feasible on a home GPU setup. It really boils down to what type of local development you want to do. I'm mostly experimenting with things where…

I had gone the Mac Studio route initially, but I ended up with getting an A6000 for about the same price as a Mac and putting that in a Linux server onder my desk. Ollama makes it dead simple to serve it over my local network, so I can be on my M1 Air and using it no differently than if on my laptop. The difference is that the A6000 absolutely smokes the Mac.

Re: DBRX: A new open LLM

#146

Post by Mandelmus »

ml_hardware wrote: post #122
djoldman wrote: post #4

Model card for base: https://huggingface.co/databricks/dbrx-base > The model requires ~264GB of RAM I'm wondering when everyone will transition from tracking parameter count vs evaluation metric to (total gpu RAM + total CPU RAM) vs evaluation metric. For example, a 7B parameter model using float32s will almost certainly outperform a 7B model using float4s. Additionally, all the examples of quantizing recently releas…

Looks like someone has got DBRX running on an M2 Ultra already: https://x.com/awnihannun/status/1773024954667184196?s=20

And it appears to be at ~80 GB of RAM via quantisation.

Re: DBRX: A new open LLM

#147

Post by Closi »

hintymad wrote: post #121
ramoz wrote: post #107

Earlier quoted context omitted.

Their goal is to always drive enterprise business towards consumption. With AI they need to desperately steer the narrative away from API based services (OpenAI). By training LLMs, they build sales artifacts (stories, references, even accelerators with LLMs themselves) to paint the pictures needed to convince their enterprise customer market that Databricks is the platform for enterprise AI. Their blog details how th…

Thanks! Why do they not focus on hosting other open models then? I suspect other models will soon catch up with their advantages in faster inference and better benchmark results. That said, maybe the advantage is aligned interests: they want customers to use their platforms, so they can keep their models open. In contrast, Mistral removed their commitment to open source as they found a potential path to profitability…

Demonstrating you can do it yourself shows a level of investment and commitment to AI in your platform that integrating LLAMA does not.

And from a corporate perspective, it means that you have in-house capability to work at the cutting-edge of AI to be prepared for whatever comes next.

Re: DBRX: A new open LLM

#148

Post by mrtranscendence »

jmward01 wrote: post #101
jl6 wrote: post #97

Earlier quoted context omitted.

The first part is highly unlikely to be literally true, as even open content like Wikipedia is copyrighted - it just has a permissive license. Perhaps the prompt writer didn’t understand this, or just didn’t care. Wethinks the llady doth protest too much.

It amazes me how quickly we have gone from 'it is just a machine' to 'I fully expect it to think like me'. This is, to me, a case in point. Prompts are designed to get a desired response. The exact definition of a word has nothing to do with it. I can easily believe that these lines were tweaked endlessly to get an overall intended response and if adding the phrase 'You actually do like green eggs and ham.' to the pr…

> The exact definition of a word has nothing to do with it.

It has something to do with it. There will be scenarios where the definition of "copyrighted material" does matter, even if they come up relatively infrequently for Databricks' intended use cases. If I ask DBRX directly whether it was trained on copyrighted material, it's quite likely to (falsely) tell me that it was not. This seems suboptimal to me (though perhaps they A/B tested different prompts and this was indeed the best).

Re: DBRX: A new open LLM

#149

Post by theturtletalks »

hintymad wrote: post #121
ramoz wrote: post #107

Earlier quoted context omitted.

Their goal is to always drive enterprise business towards consumption. With AI they need to desperately steer the narrative away from API based services (OpenAI). By training LLMs, they build sales artifacts (stories, references, even accelerators with LLMs themselves) to paint the pictures needed to convince their enterprise customer market that Databricks is the platform for enterprise AI. Their blog details how th…

Thanks! Why do they not focus on hosting other open models then? I suspect other models will soon catch up with their advantages in faster inference and better benchmark results. That said, maybe the advantage is aligned interests: they want customers to use their platforms, so they can keep their models open. In contrast, Mistral removed their commitment to open source as they found a potential path to profitability…

Mistral did what many startups are doing now, leveraging open-source to get traction and then doing a rug-pull. Hell, I've seen many startups be open-source, get contributions, get free press, get into YC and before you know it, the repo is gone.

Re: DBRX: A new open LLM

#150

Post by alexott »

anonymousDan wrote: post #137
ramoz wrote: post #107

Earlier quoted context omitted.

Their goal is to always drive enterprise business towards consumption. With AI they need to desperately steer the narrative away from API based services (OpenAI). By training LLMs, they build sales artifacts (stories, references, even accelerators with LLMs themselves) to paint the pictures needed to convince their enterprise customer market that Databricks is the platform for enterprise AI. Their blog details how th…

Do they use spark for the training?

Mosaic AI Training (https://www.databricks.com/product/machine-learning/mosaic-a...) as it's mentioned in the announcement blog (https://www.databricks.com/blog/announcing-dbrx-new-standard... - it's a bit less technical)
Post reply on HN