Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Smollm3: Smol, multilingual, long-context reasoner LLM

huggingface.co

Post reply on HN
Print view View original

31–40 of 83 posts

Re: Smollm3: Smol, multilingual, long-context reasoner LLM

#31

Post by mhitza »

ivape wrote: post #27
mhitza wrote: post #8

Earlier quoted context omitted.

You really need to try them all out yourself and make sure you have proper benchmarks. While machine learning is not my field, I've tried to finetune Mistral 7B (following their official guide and toolset) and the results did not satisfy. Had a few very specific questions from the dataset that no matter how much I've finetuned and tweaked the process it was not able to respond with correct information. A mix of vecto…

How much data did you use to fine tune?

Kilobytes to megabytes of data. I was trying to fine-tune it for some specific legislation I was expecting to be able afterwards to ask about.

Re: Smollm3: Smol, multilingual, long-context reasoner LLM

#32

Post by refulgentis »

WhitneyLand wrote: post #7

Mostly SOTA performance at the 3B level. A notable addition to the small but truly open club of models that provide full disclosure, code, recipes to reproduce their work. Looks like ballpark a million dollars of GPU time if you want to train up one for yourself (4000 gpus/24 days). Very nice write up that’s generous in sharing their learnings. This is a solid and positive contribution.

I spent about 10 minutes this AM cross-checking with Phi-4-mini benchmarks, as it was very odd to not include the leader in benchmarks and it seemed universally behind.

For context, I dev an LLM client, a core tenant is keeping local as close to cloud parity as much as is possible. (via llama.cpp)

Companies aren't taking local AI seriously on a sustained basis outside Microsoft.

Overall, I usually would bite my tongue. HF is a great citizen, and I doubt this'll be a one off. However, when I see superlatives affirmed, while leaving out the local SoTA for many many moons that is a godsend in this sector, I think it is good to, rather than shy away, stand up and say this.

Re: Smollm3: Smol, multilingual, long-context reasoner LLM

#33

Post by adrianlzt »

refulgentis wrote: post #32
WhitneyLand wrote: post #7

Mostly SOTA performance at the 3B level. A notable addition to the small but truly open club of models that provide full disclosure, code, recipes to reproduce their work. Looks like ballpark a million dollars of GPU time if you want to train up one for yourself (4000 gpus/24 days). Very nice write up that’s generous in sharing their learnings. This is a solid and positive contribution.

I spent about 10 minutes this AM cross-checking with Phi-4-mini benchmarks, as it was very odd to not include the leader in benchmarks and it seemed universally behind. For context, I dev an LLM client, a core tenant is keeping local as close to cloud parity as much as is possible. (via llama.cpp) Companies aren't taking local AI seriously on a sustained basis outside Microsoft. Overall, I usually would bite my tongu…

From the blog post: "SmolLM3 supports tool calling, and its chat template incorporates two distinct sections for tool descriptions: XML Tools and Python Tools"

Re: Smollm3: Smol, multilingual, long-context reasoner LLM

#34

Post by a_wild_dandan »

janalsncm wrote: post #22
bitwize wrote: post #9

There's a British comedy skit lurking in here. "So it's a small large language model?" "Oh yes, very small." "How can it be small and large at the same time?" "Well, it's small by the standards of a large language model." "So it's large." "Oh yes, very large." "Large compared to what?" "Small language models." "And so something like ChatGPT, what would that be exactly? A large large language model?" "Yes, precisely.…

Standards have shifted as well. Gpt2 used to be considered “large” but it is half the size of this. Oh and also Sam Altman said it was too dangerous to release. At this point I consider anything too big to run on consumer grade hardware to be large, but an exact definition is a little silly to argue about.

Altman released GPT-2 despite expressing that doing so was a bad idea? That's wild.

Re: Smollm3: Smol, multilingual, long-context reasoner LLM

#35

Post by peatmoss »

Imustaskforhelp wrote: post #26
hynky wrote: post #25

Earlier quoted context omitted.

yes, both core web datasets are publicly available as well as the rest

Thanks! To be honest, if I might argue then that this is one of the best truly open source models that we have got. There is AllenAI and (Elmo?) and there is also this one which does distributed training but I think this looks a lot like SOTA for 3B parameters to me. Thanks for telling me, I am not going to lie, I am going to try to test it now! (Ima try some GGUF since ollama convenience)

OLMo: https://allenai.org/olmo

AFAIK, they were the first open everything model.

Re: Smollm3: Smol, multilingual, long-context reasoner LLM

#36

Post by Alifatisk »

a_wild_dandan wrote: post #34
janalsncm wrote: post #22

Earlier quoted context omitted.

Standards have shifted as well. Gpt2 used to be considered “large” but it is half the size of this. Oh and also Sam Altman said it was too dangerous to release. At this point I consider anything too big to run on consumer grade hardware to be large, but an exact definition is a little silly to argue about.

Altman released GPT-2 despite expressing that doing so was a bad idea? That's wild.

I think Altman meant it's too dangerous to open-source GPT-2, therefore locked it in behind a service.

Re: Smollm3: Smol, multilingual, long-context reasoner LLM

#38

Post by _kb »

bitwize wrote: post #9

There's a British comedy skit lurking in here. "So it's a small large language model?" "Oh yes, very small." "How can it be small and large at the same time?" "Well, it's small by the standards of a large language model." "So it's large." "Oh yes, very large." "Large compared to what?" "Small language models." "And so something like ChatGPT, what would that be exactly? A large large language model?" "Yes, precisely.…

Australian. This is straight up Clarke and Dawe / Utopia.

Re: Smollm3: Smol, multilingual, long-context reasoner LLM

#39

Post by janalsncm »

Alifatisk wrote: post #36
a_wild_dandan wrote: post #34

Earlier quoted context omitted.

Altman released GPT-2 despite expressing that doing so was a bad idea? That's wild.

I think Altman meant it's too dangerous to open-source GPT-2, therefore locked it in behind a service.

It’s not locked behind a service though.

https://huggingface.co/openai-community/gpt2/blob/main/model...

Re: Smollm3: Smol, multilingual, long-context reasoner LLM

#40

Post by magicalhippo »

mhitza wrote: post #8
_1 wrote: post #5

Which small model is good for fine tuning to various enterprise data sets? Our business units are wanting to run small models in browser and on mobile devices, without dealing with RAG and cloud resources.

You really need to try them all out yourself and make sure you have proper benchmarks. While machine learning is not my field, I've tried to finetune Mistral 7B (following their official guide and toolset) and the results did not satisfy. Had a few very specific questions from the dataset that no matter how much I've finetuned and tweaked the process it was not able to respond with correct information. A mix of vecto…

> Maybe building syntethic questions and answers around the dataset yields better results but I didn't have time to experiment with that approach.

While they answer a slightly different question in the Physics of Language Models[1], based on their results it seems to me it is likely that one needs to do such augmentation of the dataset to get good results.

However, they also show that the dataset the base model is trained on can drastically affect finetuning performance. So if the base model is trained on a poor dataset for your specific task, perhaps you'll never get good performance.

[1]: https://physics.allen-zhu.com/part-3-knowledge/part-3-1

Post reply on HN