31–40 of 51 posts
Very interesting. According to their X posts, this meme model "SmolLm" beats Meta's new 1B and 3B models across almost all metrics. I wonder how this is possible given that Meta has been in this game for much longer and probably has much more data at their disposal as well.
Usually, that's because they use a groundbreaking ML method called TTDS, or "training on the test dataset".
Semi-related but is there a standard way to run this (or other models from huggingface) in a docker container and interact with them through a web API? ChatGPT tells me to write my own FastAPI wrapper which should work, but is there no pre-made solution for this?
$ docker run --runtime nvidia --gpus all \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HUGGING_FACE_HUB_TOKEN=" \
-p 8000:8000 \
--ipc=host \
vllm/vllm-openai:latest \
--model mistralai/Mistral-7B-v0.1
$ curl http://localhost:8000/v1/completions \
-H "Content-Type: application/json" \
-d '{
"model": "mistralai/Mistral-7B-v0.1",
"prompt": "San Francisco is a",
"max_tokens": 7,
"temperature": 0
}'Why would I care about this when I can have the entire English Wikipedia on my phone? Really struggling to understand why people are so excited about this stuff.
Knowledge doesnt amount to much on the LLM, i think what most are excited about is the artificial reasoning.
Earlier quoted context omitted.
Knowledge doesnt amount to much on the LLM, i think what most are excited about is the artificial reasoning.
What is left for the brain to do? First people let their bodies atrophy. Next it's the mind. Wall-E here we come.
Very interesting. According to their X posts, this meme model "SmolLm" beats Meta's new 1B and 3B models across almost all metrics. I wonder how this is possible given that Meta has been in this game for much longer and probably has much more data at their disposal as well.
Usually, that's because they use a groundbreaking ML method called TTDS, or "training on the test dataset".
Earlier quoted context omitted.
Usually, that's because they use a groundbreaking ML method called TTDS, or "training on the test dataset".
I would hope Simon would not fall victim to such shenanigans, and has his own test dataset.
Does it support anything other than English? Sadly, most open-weights models have no support for languages other than English, which makes them useless for 75% world's population who don't speak English at all. Does anyone know of a good lightweight open-weights LLM which supports at least a few major languages (let's say, the official UN languages at least)?
Earlier quoted context omitted.
Usually, that's because they use a groundbreaking ML method called TTDS, or "training on the test dataset".
I would hope Simon would not fall victim to such shenanigans, and has his own test dataset.
You'll note that I didn't quote their benchmarks in my own post at all, because I didn't want to boost them without feeling confident in what they were stating.
I posted about this because my own very limited initial experiments passed a loose vibe check!
I'm impressed any time a 1.7GB (or 130MB) model file appears to be able to do anything useful at all.