231–240 of 442 posts
Four independent Chinese companies released extremely good open source models in the past few months (DeepSeek, Qwen/Alibaba, Kimi/Moonshot, GLM/Z.ai). No American or European companies are doing that, including titans like Meta. What gives?
Earlier quoted context omitted.
If I understand transformers properly, this is unlikely to work. The whole point of “Large” Language Models is that you primarily make them better by making them larger, and when you do so, they get better at both general and specific tasks (so there isn’t a way to sacrifice generality but keep specific skills when training a small models). I know a lot of people want this (Apple really really wants this and is pouri…
Yeah - the whole business model of companies like OpenAI and Anthropic, at least at the moment, seems to be that the models are so big that you need to run them in the cloud with metered access. Maybe that could change in the future to sale or annual licence business model if running locally became possible. I think scale helps for general tasks where the breadth of capability may be needed, but it's not so clear tha…
I actually find that things which make me a better programmer are often those things which have the least overlap with it. Like gardening!
Earlier quoted context omitted.
It's not confusing, no one is really confused except the people upset that the meaning is different in a different context. On top of that, in many cases a company/group/whoever can't even reproduce the model themselves. There are lots of sources of non-determinism even if folks are doing things in a very buttoned up manner. And, when you are training on trillions of tokens, you are likely training on some awful soun…
i disagree. words matter. the whole point of open source is that anyone can look and see exactly how the sausage is made. that is the point. that is why the word "open" is used. ...and sure, compiling gcc is nondeterministic too, but i can still inspect the complete source from where it comes because it is open source, which means that all of the source materials are available for inspection.
Earlier quoted context omitted.
Only until they start incorporating this test into their training data.
Dataset contamination alone won't get them good-looking SVG pelicans on bicycles though, they'll have to either cheat this particular question specifically or train it to make vector illustrations in general. At which point it can be easily swapped for another problem that wasn't in the data.
Earlier quoted context omitted.
The latter. A reasoning model has been finetuned to use the scratchpad for intermediate results (which works better than just prompting a model to do the same).
I'd expect the same (fine tuning to be better than mere prompting) for most anything. So a model is or is not "a reasoning model" according to the extent of a fine tune. Are there specific benchmarks that compare models vs themselves with and without scratchpads? High with:without ratios being reasonier models? Curious also how much a generalist model's one-shot responses degrade with reasoning post-training.
uv tool install llm llm install llm-moonshot llm keys set moonshot # paste key llm -m moonshot/kimi-k2-thinking 'Generate an SVG of a pelican riding a bicycle' https://tools.simonwillison.net/svg-render#%3Csvg%20width%3D... Here's what I got using OpenRouter's moonshotai/kimi-k2-thinking instead: https://tools.simonwillison.net/svg-render#%20%20%20%20%3Csv...
I suspect that the OpenRouter result originates from a quantized hosting provider. The difference compared to the direct API call from Moonshot is striking, almost like night and day. It creates a peculiar user and developer experience since OpenRouter enforces quantization restrictions only at the API level, rather than at the account settings level.
How does one effectively use something like this locally with consumer-grade hardware?
uv tool install llm llm install llm-moonshot llm keys set moonshot # paste key llm -m moonshot/kimi-k2-thinking 'Generate an SVG of a pelican riding a bicycle' https://tools.simonwillison.net/svg-render#%3Csvg%20width%3D... Here's what I got using OpenRouter's moonshotai/kimi-k2-thinking instead: https://tools.simonwillison.net/svg-render#%20%20%20%20%3Csv...
Love seeing this benchmark become more iconic with each new model release. Still in disbelief at the GPT-5 variants' performance in comparison but its cool to see the new open source models get more ambitious with their attempts.
TLDR; this is an alibaba funded start-up out of Beijing Okay, I'm sorry but I have to say wtf named this thing. Moonshot AI is such an overused generic name that I had to ask an LLM which company this is. This is just Alibaba hedging their Qwen model. This company is far from "open source", it's had over $1B USD in funding.
I just googled "Moonshot AI" and got the information right away. Not sure what's confusing about it, the only other "Moonshot" I know of is Alphabet's Moonshot Factory.
> This company is far from "open source", it's had over $1B USD in funding.
Since when does open source mean you can't make any money? Mozilla has a total of $1.2B in assets. The company isn't open source nor claiming to be.
This model was released under a "modified MIT-license" [0]:
> Our only modification part is that, if the Software (or any derivative works thereof) is used for any of your commercial products or services that have more than 100 million monthly active users, or more than 20 million US dollars (or equivalent in other currencies) in monthly revenue, you shall prominently display "Kimi K2" on the user interface of such product or service.
Which sounds pretty fair to me.
[0] - https://huggingface.co/moonshotai/Kimi-K2-Thinking/blob/main...