Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DBRX: A new open LLM

databricks.com

Post reply on HN
Print view View original

171–180 of 360 posts

Re: DBRX: A new open LLM

#171

Post by swalsh »

sroussey wrote: post #96
swalsh wrote: post #74

Earlier quoted context omitted.

The models are commodities, and the API's are even similar enough that there is zero stickiness. I can swap one model for another, and usually not have to change anything about my prompts or rag pipelines. For startups, the lesson here is don't be in the business of building models. Be in the business of using models. The cost of using AI will probably continue to trend lower for the foreseeable future... but you can…

Embeddings are not interchangeable. However, you can setup your system to have multiple embeddings from different providers for the same content.

Embeddings are indeed sticky, I was referring to the LLM model itself.

Re: DBRX: A new open LLM

#172

Post by pram »

n2d4 wrote: post #166
emmender2 wrote: post #46

this proves that all llm models converge to a certain point when trained on the same data. ie, there is really no differentiation between one model or the other. Claims about out-performance on tasks are just that, claims. the next iteration of llama or mixtral will converge. LLMs seem to evolve like linux/windows or ios/android with not much differentiation in the foundation models.

There's at least an argument to be made that this is because all the models are heavily trained on GPT-4 outputs (or whatever the SOTA happens to be during training). All those models are, in a way, a product of inbreeding.

Consider the bulldog: https://youtube.com/watch?v=hUgmkCgMWbg

Re: DBRX: A new open LLM

#173

Post by zeeg »

patrick-fitz wrote: post #98

Looking at the license restrictions: https://github.com/databricks/dbrx/blob/main/LICENSE "If, on the DBRX version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Databricks, which we may grant to you in our sole discretion, a…

Its literally described as open source all over.

https://www.databricks.com/blog/announcing-dbrx-new-standard...

Its even implied in comparisons everywhere:

> Figure 1: DBRX outperforms established open source models on language understanding (MMLU), Programming (HumanEval), and Math (GSM8K).

> The aforementioned three reasons lead us to believe that open source LLMs will continue gaining momentum. In particular, we think they provide an exciting opportunity for organizations to customize open source LLMs that can become their IP, which they use to be competitive in their industry.

Just search "open source".

Re: DBRX: A new open LLM

#174

Post by BryantD »

jjtheblunt wrote: post #138

I’d like to know how Nancy Pelosi, who sure as hell doesn’t know what Apache Spark is, bought $1 million worth (and maybe $5million) of Databricks stock days ago. https://www.dailymail.co.uk/sciencetech/article-13228859/amp...

I don't have any interest in defending Pelosi's stock trades, and I agree that sitting members of Congress should not be trading stocks.

That said, this report seems inaccurate to me. Pelosi put between 1 and 5 million dollars of Forge Investments, which is a method for investing in per-IPO companies, as I understand it. Databricks is one of those, but so is OpenAI, Hugging Face, Anthropic, and Humane. If I wanted to invest in pre-IPO AI companies it seems like a very natural choice and I don't think we need insider trading to explain it.

It's also the case that the report she filed calls out Databricks stock, which is perhaps an indication that she was particularly interested in that. Stronger reporting would tell us how often she's invested in Forge, if this is the first time, and so on. One other possible explanation is that she was investing ahead of the Humane Pin shipping and wanted to pull attention away from it, for example.

Re: DBRX: A new open LLM

#175

Post by jimmySixDOF »

sroussey wrote: post #96
swalsh wrote: post #74

Earlier quoted context omitted.

The models are commodities, and the API's are even similar enough that there is zero stickiness. I can swap one model for another, and usually not have to change anything about my prompts or rag pipelines. For startups, the lesson here is don't be in the business of building models. Be in the business of using models. The cost of using AI will probably continue to trend lower for the foreseeable future... but you can…

Embeddings are not interchangeable. However, you can setup your system to have multiple embeddings from different providers for the same content.

There are people who make the case for custom fine tuned embedding models built to match your specific types of data and associations. Whatever you use internally it gets converted to the foundation model of choice's formats by their tools on the edge. Still Embeddings and the chunking strategies feeding into them are both way too underappreciated parts of the whole pipeline.

Re: DBRX: A new open LLM

#176

Post by jjtheblunt »

hiddencost wrote: post #162
jjtheblunt wrote: post #138

I’d like to know how Nancy Pelosi, who sure as hell doesn’t know what Apache Spark is, bought $1 million worth (and maybe $5million) of Databricks stock days ago. https://www.dailymail.co.uk/sciencetech/article-13228859/amp...

You know she has advisors, right?

Ignoring the snark: Obviously.

SEC put Martha Stewart in jail for following her advisor, and that was for about $45,000.

Re: DBRX: A new open LLM

#177

Post by spxneo »

swalsh wrote: post #74
emmender2 wrote: post #46

this proves that all llm models converge to a certain point when trained on the same data. ie, there is really no differentiation between one model or the other. Claims about out-performance on tasks are just that, claims. the next iteration of llama or mixtral will converge. LLMs seem to evolve like linux/windows or ios/android with not much differentiation in the foundation models.

The models are commodities, and the API's are even similar enough that there is zero stickiness. I can swap one model for another, and usually not have to change anything about my prompts or rag pipelines. For startups, the lesson here is don't be in the business of building models. Be in the business of using models. The cost of using AI will probably continue to trend lower for the foreseeable future... but you can…

Excellent comment. Shows good awareness of economic forces at play here.

We are just going to use whatever LLM is best fast/cheap and the giants are in an arms race to deliver just that.

But only two companies in this epic techno-cold war have an economic moat but the other moat is breaking down inside the moat of the other company. The moat inside the moat cannot run without the parent moat.

Re: DBRX: A new open LLM

#178

Post by SparkyMcUnicorn »

madiator wrote: post #167
ml_hardware wrote: post #122

Earlier quoted context omitted.

Looks like someone has got DBRX running on an M2 Ultra already: https://x.com/awnihannun/status/1773024954667184196?s=20

That's great, but it did not really write the program that the human asked it to do. :)

That's because it's the base model, not the instruct tuned one.

Re: DBRX: A new open LLM

#179

Post by giancarlostoro »

zozbot234 wrote: post #49
XCSme wrote: post #42

Earlier quoted context omitted.

I already have 128GB of RAM (DDR4), and was wondering if upgrading from a 1080ti (12GB) to a 4070ti super (16GB), would make a big difference. I assume the FP32 and FP16 operations are already a huge improvement, but also the 33% increased VRAM might lead to fewer swaps between VRAM and RAM.

That's system memory, not unified memory. Unified means that all or most of it is going to be directly available to the Apple Silicon GPU.

This is the key factor here. I have a 3080, with 16GB of Memory, but still have to run some models on CPU since the memory is not unified at all.

Re: DBRX: A new open LLM

#180

Post by wavemode »

m3kw9 wrote: post #125

These tiny “state of the art” performance increases are really indicative the current architecture for LLM(Transformers + Mixture of Experts) is maxed out even if you train it more/differently. The writings are on all over the walls.

It would not surprise me if this is what has delayed OpenAI in releasing a new model. After more than a year since GPT-4, they may have by now produced some mega-trained mega-model, but running it is so expensive, and its eval improvement over GPT-4 so marginal, that releasing it to the public simply makes no commercial sense just yet.

They may be working on how to optimize it to reduce cost, or re-engineer it to improve evals.

Post reply on HN