SmolLM2

#1

Post by edward » Sat, Nov 02, 2024, 7:31 AM UTC

SmolLM2
simonwillison.net

Re: SmolLM2

#2

Post by ksri » Sat, Nov 02, 2024, 9:00 AM UTC

Is there a way to run this in the browser as yet? Transformers js doesn't seem to support this. Is there another way to run this in the browser?

Re: SmolLM2

#3

Post by adhamsalama » Sat, Nov 02, 2024, 9:04 AM UTC

ksri wrote: post #2

Is there a way to run this in the browser as yet? Transformers js doesn't seem to support this. Is there another way to run this in the browser?

Maybe WebAssembly?

Re: SmolLM2

#4

Post by kgeist » Sat, Nov 02, 2024, 9:06 AM UTC

Does it support anything other than English? Sadly, most open-weights models have no support for languages other than English, which makes them useless for 75% world's population who don't speak English at all.

Does anyone know of a good lightweight open-weights LLM which supports at least a few major languages (let's say, the official UN languages at least)?

Re: SmolLM2

#5

Post by oulipo » Sat, Nov 02, 2024, 9:06 AM UTC

Nice! Do you think they could be fine-tuned to implement a cool thing like https://withaqua.com/ ? eg to teach it to do "inline edits" of what you say?

Re: SmolLM2

#6

Post by numberless » Sat, Nov 02, 2024, 9:09 AM UTC

I wonder how one would finetune this

Re: SmolLM2

#7

Post by Nowado » Sat, Nov 02, 2024, 9:16 AM UTC

ksri wrote: post #2

Is there a way to run this in the browser as yet? Transformers js doesn't seem to support this. Is there another way to run this in the browser?

Probably ONNX.

Re: SmolLM2

#8

Post by cpa » Sat, Nov 02, 2024, 9:17 AM UTC

What’s the context size? I couldn’t find it on the model summary page. Tangential: if it’s not on the model page, does it mean that it’s not that relevant here? If so, why?

Re: SmolLM2

#9

Post by EugeneOZ » Sat, Nov 02, 2024, 9:21 AM UTC

kgeist wrote: post #4

Does it support anything other than English? Sadly, most open-weights models have no support for languages other than English, which makes them useless for 75% world's population who don't speak English at all. Does anyone know of a good lightweight open-weights LLM which supports at least a few major languages (let's say, the official UN languages at least)?

> SmolLM2 models primarily understand and generate content in English.

https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct/b...

Re: SmolLM2

#10

Post by echoangle » Sat, Nov 02, 2024, 9:22 AM UTC

Semi-related but is there a standard way to run this (or other models from huggingface) in a docker container and interact with them through a web API? ChatGPT tells me to write my own FastAPI wrapper which should work, but is there no pre-made solution for this?