Cohere's First Model for Developers • Hacker News Board

Re: Cohere's First Model for Developers

#41

Post by namr2000 » Tue, Jun 16, 2026, 6:48 PM UTC

amunozo wrote: post #28

Are these models trained from scratch or do they necessarily need distillation from bigger models to be competitive? It's usually the case that they're a small model for a family with a bigger model. In the first case, does anybody know what's the economy of training this 30B-A3B model vs. training a DeepSeek V4 Pro or Flash size of models (1.6T, 200 something B, less activated)?

You don't have to train from scratch but you can. Distillation ends up being somewhere in the ballpark of 1000x faster to train [1]. It also comes with the huge advantage of not needing to create RLHF datasets, since you can just copy the behavior of the teacher model. This saves an enormous amount of labeling money at the cost of making the model behave similarly to the teacher. If you are training from scratch, you can look at LLM scaling laws to figure out roughly the compute budget you need to optimally train a model [2].

Based on [2] a 30B model needs something like 2e+23 FLOPS to train from scratch whereas a 1.6T model needs something like 1e+27 FLOPs to train. So DeepSeek v4 Pro was roughly 5000x more expensive to train than this model. I'm not totally sure how MOE affects scaling laws, so these numbers might be different in reality, but it gives you a good ballpark estimate of the difference in training scale.

[1] https://arxiv.org/abs/2505.12781 [2] https://arxiv.org/abs/2203.15556

Re: Cohere's First Model for Developers

#42

Post by zuzululu » Tue, Jun 16, 2026, 7:34 PM UTC

daijj wrote: post #37

greyb wrote: post #17

Earlier quoted context omitted.

>Wasn't aware that Cohere was still around but this release doesn't exactly instill confidence. It's being kept alive because the Canadian government is desperate to have a local frontier lab and is willing to inject funding and force its adoption in government services, but leadership at Cohere is known to be weak in Canadian tech circles, and they pivoting to an enterprise-first market around production RAG rather…

Mulling over applying there to work. Hearing a bunch of mixed reviews where some people also complain about leadership but the day to day seems to be quite good. Any reason big US investors haven't put any money into it? (besides the fact that it's Canadian?)

why would anybody put money in Cohere when they can do it in an American AI company with larger pay off?

Re: Cohere's First Model for Developers

#43

Post by bitwize » Tue, Jun 16, 2026, 9:09 PM UTC

doodlesdev wrote: post #26

bitwize wrote: post #19

Earlier quoted context omitted.

I ran Gemma4 26B A4B on an 8yo PC with a fucking GTX and it did rather well.

Well, that's pretty impressive. Care to share your setup to do that? How much DDR3/DDR4 do you have, too?

I... downloaded a 4-bit quantized GGUF of the model, used llama.cpp to run it, and pointed OpenCode at that. My machine is an 8-core Gen1 Ryzen 7, 32 GiB of DDR4, (I think) 4 GiB of VRAM on the graphics.

Re: Cohere's First Model for Developers

#44

Post by moralestapia » Wed, Jun 17, 2026, 1:48 AM UTC

zuzululu wrote: post #38

moralestapia wrote: post #30

Earlier quoted context omitted.

It really is. I’m very familiar with that as well. It’s truly embarrassing how much hand-holding those guys have received from angels, investors, the government, etc. To the point where the same investors they’re going to pitch to are preparing their slides, telling them what to say during the presentation, and then approving them for even more funding afterward, lol. That government part is corruption and illegal, b…

this is just one of the many weird stuff I hear out of Canada that always surprises me, I've compiled a list: - the wife of a professor I knew in canada apparently makes 400k/year for some Aboriginal art gallery that gets like two visits a year. They kicked out small businesses in that building so they could have a 6000 sq ft for an art gallery that sits empty with the weirdest "art" that nobody has heard of. - canad…

>[...] charged the Canadian government $70 million for some flutter app [...]

I met one of these guys. Some parts of Canada are massively corrupt. If you're in the inner circle, the amount of things you get for free is unimaginable. If you're not, then you get the privilege of a 50%+ tax bill.

Re: Cohere's First Model for Developers

#45

Post by zuzululu » Wed, Jun 17, 2026, 5:03 AM UTC

moralestapia wrote: post #44

zuzululu wrote: post #38

Earlier quoted context omitted.

this is just one of the many weird stuff I hear out of Canada that always surprises me, I've compiled a list: - the wife of a professor I knew in canada apparently makes 400k/year for some Aboriginal art gallery that gets like two visits a year. They kicked out small businesses in that building so they could have a 6000 sq ft for an art gallery that sits empty with the weirdest "art" that nobody has heard of. - canad…

>[...] charged the Canadian government $70 million for some flutter app [...] I met one of these guys. Some parts of Canada are massively corrupt. If you're in the inner circle, the amount of things you get for free is unimaginable. If you're not, then you get the privilege of a 50%+ tax bill.

also I find the salary is atrociously low in Canada for the exact same role

Re: Cohere's First Model for Developers

#46

Post by amunozo » Wed, Jun 17, 2026, 6:12 AM UTC

namr2000 wrote: post #41

amunozo wrote: post #28

Are these models trained from scratch or do they necessarily need distillation from bigger models to be competitive? It's usually the case that they're a small model for a family with a bigger model. In the first case, does anybody know what's the economy of training this 30B-A3B model vs. training a DeepSeek V4 Pro or Flash size of models (1.6T, 200 something B, less activated)?

You don't have to train from scratch but you can. Distillation ends up being somewhere in the ballpark of 1000x faster to train [1]. It also comes with the huge advantage of not needing to create RLHF datasets, since you can just copy the behavior of the teacher model. This saves an enormous amount of labeling money at the cost of making the model behave similarly to the teacher. If you are training from scratch, you…

Thank you for taking the time, this is a very useful and complete answer.

Re: Cohere's First Model for Developers

#47

Post by doodlesdev » Sat, Jun 20, 2026, 8:42 PM UTC

bitwize wrote: post #43

doodlesdev wrote: post #26

Earlier quoted context omitted.

Well, that's pretty impressive. Care to share your setup to do that? How much DDR3/DDR4 do you have, too?

I... downloaded a 4-bit quantized GGUF of the model, used llama.cpp to run it, and pointed OpenCode at that. My machine is an 8-core Gen1 Ryzen 7, 32 GiB of DDR4, (I think) 4 GiB of VRAM on the graphics.

Well... That's much simpler than I imagined LOL, thank you for answering!