Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DBRX: A new open LLM

databricks.com

Post reply on HN
Print view View original

341–350 of 360 posts

Re: DBRX: A new open LLM

#341

Post by bevekspldnw »

Datagenerator wrote: post #305
bevekspldnw wrote: post #230

Earlier quoted context omitted.

Large scale document classification tasks in very ambiguous contexts. A lot of my work goes into using big models to generate training data for smaller models. I have multiple millions of documents so GPT is cost prohibitive, and too slow. My tools of choice tend to be a first pass with Mistral to check task performance and if lacking using Mixtral. Often I find with a good prompt Mistral will work as well as Mixtral…

Interesting I have the same task, can you share your tools? My goal is to detect if documents contain GDPR sensitive parts or are copies of official documents like ID's and driving licenses etc - would be great to reuse your work!

Working in the same sector, we’ll license it out soon.

Re: DBRX: A new open LLM

#342

Post by mazlix »

bboygravity wrote: post #251
mrtranscendence wrote: post #142

Earlier quoted context omitted.

Are you alleging that Nancy Pelosi invested in Databricks, a private company without a fluctuating share price, because she learned that they would soon release a small, fairly middling LLM that probably won't move the needle in any meaningful way?

Are you suggesting that Nancy Pelosi, who consistently beats the market through obvious insider trading for years in a row, bought a share in Databricks without any insider info? Possible, yet unlikely is my opinion. https://jacobin.com/2021/12/house-speaker-paul-stocks-inside... PS: "without a fluctuating share price" is non-sense. Just because the share is of a private company, doesn't mean its price can't fluctuat…

I'm not a lawyer and this isn't investment advice so don't sue me if I'm wrong but I'm not sure this qualifies as insider trading in the way that would be illegal for public markets.

Aren't most investors in private companies privy to information that isn't entirely public?

I can see how this feels a bit different because DataBricks might be the size where it might trade with a decent amount of liquidity, but certainly in smaller rounds it's got to be pretty normal.

Maybe if she bought it secondary and the person from whom she purchased the shares was witheld this information they could sue?

Re: DBRX: A new open LLM

#343

Post by brucethemoose2 »

belter wrote: post #269
brucethemoose2 wrote: post #237

I would note the actual leading models right now (IMO) are: - Miqu 70B (General Chat) - Deepseed 33B (Coding) - Yi 34B (for chat over 32K context) And of course, there are finetunes of all these. And there are some others in the 34B-70B range I have not tried (and some I have tried, like Qwen, which I was not impressed with). Point being that Llama 70B, Mixtral and Grok as seen in the charts are not what I would call…

For all the Model Cards and License notices, I find it interesting there is not much information on the contents of the dataset used for training. Specifically, if it contains data subject to Copyright restrictions. Or did I miss that?

Yeah, its an unspoken but rampant thing in the llm community. Basically no one respects licenses for training data.

I'd say the majority of instruct tunes, for instance, use OpenAI output (which is against their TOS).

But its all just research! So who cares! Or at least, that seems to be the mood.

Re: DBRX: A new open LLM

#344

Post by marci »

irusensei wrote: post #329
ml_hardware wrote: post #122

Earlier quoted context omitted.

Looks like someone has got DBRX running on an M2 Ultra already: https://x.com/awnihannun/status/1773024954667184196?s=20

I can run a certain 120b on my M3 max with 128GB memory. However I found that while it “fits” Q5 was extremely slow. The story was different with Q4 though which ran just fine around ~3.5-4 t/s. Now this model is ~134B right? It could be bog slow but on the other hand its a MoE so there might be a chance it could have satisfactory results.

From the article, should have the speed of a ~36b.

Re: DBRX: A new open LLM

#346

Post by joaquincabezas »

it looks like TensorRT-LLM (TRT-LLM) is the way to go for a realtime API for more and more companies (i.e perplexity ai’s pplx-api, Mosaic’s, baseten…). Would be super-nice to find people deploying multimodal (i.e LLaVA or CLIP/BLIP) to discuss approaches (and cry a bit together!)

Re: DBRX: A new open LLM

#347

Post by evrial »

Bourgeois have fun with number crushers. Clowns, make comparison of metrics normalized to token/second/watt and token/second/per memory stick of ram 8gb, 16gb, 32gb and consumer GPUs.

Re: DBRX: A new open LLM

#348

Post by evrial »

panarky wrote: post #340
d-z-m wrote: post #295

Earlier quoted context omitted.

If you chart the temperature of the ocean are you measuring it in Kelvin?

Apparently, if you want to avoid "chart crime" when you chart temperatures, then it's deceptive if you don't start at absolute zero.

When was the temperature on Earth at absolute zero?

Re: DBRX: A new open LLM

#349

Post by evrial »

MajimasEyepatch wrote: post #212
IshKebab wrote: post #95

Earlier quoted context omitted.

That sounds unsurprising? Like if you take any set of numbers, randomly split it in two, then calculate the average of each half... it's not surprising that they'll be almost the same. If you took two different training sets then it would be more surprising. Or am I misunderstanding what you mean?

It doesn't really matter whether you do this experiment with two training sets created independently or one training set split in half. As long as both are representative of the underlying population, you would get roughly the same results. In the case of human faces, as long as the faces are drawn from roughly similar population distributions (age, race, sex), you'll get similar results. There's only so much variati…

That's explanation of central limit theorem in statistics. And any language is mostly statistics and models are good at statistical guessing of the next word or token.

Re: DBRX: A new open LLM

#350

Post by evrial »

taneq wrote: post #261
MajimasEyepatch wrote: post #212

Earlier quoted context omitted.

It doesn't really matter whether you do this experiment with two training sets created independently or one training set split in half. As long as both are representative of the underlying population, you would get roughly the same results. In the case of human faces, as long as the faces are drawn from roughly similar population distributions (age, race, sex), you'll get similar results. There's only so much variati…

If not are sampled from the same population then they’re not really independent, even if they’re totally disjoint.

They are sourced mostly from the same population and crawled from everything can be crawled.
Post reply on HN