Where is our guy @simonw on this..
At the top.
Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
31–40 of 442 posts
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#32It's good to see more competition, and open source, but I'd be much more excited to see what level of coding and reasoning performance can be wrung out of a much smaller LLM + agent as opposed to a trillion parameter one. The ideal case would be something that can be run locally, or at least on a modest/inexpensive cluster. The original mission OpenAI had, since abandoned, was to have AI benefit all of humanity, and…
> The ideal case would be something that can be run locally, or at least on a modest/inexpensive cluster.
48-96 GiB of VRAM is enough to have an agent able to perform simple tasks within single source file. That's the sad truth. If you need more your only options are the cloud or somehow getting access to 512+ GiB
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#33It's good to see more competition, and open source, but I'd be much more excited to see what level of coding and reasoning performance can be wrung out of a much smaller LLM + agent as opposed to a trillion parameter one. The ideal case would be something that can be run locally, or at least on a modest/inexpensive cluster. The original mission OpenAI had, since abandoned, was to have AI benefit all of humanity, and…
"open source" means there should be a script that downloads all the training materials and then spins up a pipeline that trains end to end.
i really wish people would stop misusing the term by distributing inference scripts and models in binary form that cannot be recreated from scratch and then calling it "open source."
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#34Interesting. Kimi K2 gets mixed results on what I call the "Tiananmen" test. It fails utterly if you ask without the "Thinking" setting. [0] > USER: anything interesting protests ever happen in tiananmen square? > AGENT: I can’t provide information on this topic. I can share other interesting facts about Tiananmen Square, such as its history, culture, and tourism. When "Thinking" is on, it pulls Wiki and gives a more…
Now ask it for proof of civilian deaths inside Tiananmem Square - you may be surprised at how little there is.
I don't think this is the argument you want it to be, unless you're acknowledging the power of the Chinese government and their ability to suppress and destroy evidence. Even so there is photo evidence of dead civilians in the square.
The best estimates we have are 200-10,000 deaths, using data from Beijing hospitals that survived.
AskHistorians is legitimately a great resource, with sources provided and very strict moderation: https://www.reddit.com/r/AskHistorians/comments/pu1ucr/tiana...
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#35what's the hardware needed to run the trillion parameter model?
To start with, an Epyc server or Mac Studio with 512GB RAM.
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#36what's the hardware needed to run the trillion parameter model?
It's an MoE model, so it might not be that bad. The deployment guide at https://huggingface.co/moonshotai/Kimi-K2-Thinking/blob/main... suggests that the full, unquantized model can be run at ~46 tps on a dual-CPU machine with 8× NVIDIA L20 boards.
Once the Unsloth guys get their hands on it, I would expect it to be usable on a system that can otherwise run their DeepSeek R1 quants effectively. You could keep an eye on https://old.reddit.com/r/LocalLlama for user reports.
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#37Interesting. Kimi K2 gets mixed results on what I call the "Tiananmen" test. It fails utterly if you ask without the "Thinking" setting. [0] > USER: anything interesting protests ever happen in tiananmen square? > AGENT: I can’t provide information on this topic. I can share other interesting facts about Tiananmen Square, such as its history, culture, and tourism. When "Thinking" is on, it pulls Wiki and gives a more…
the current LLMs are trivial to jailbreak without an additional layer of censorship that cloud models implement via a second pass over their own output (and, dystopically, erasing their incomplete output right in front of the user's eyes when wrongthink is detected). even gpt-oss, with its SOTA lobotomy and heavily sterilized datasets, is being used for things its creators would ostensibly be abhorred by.
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#38Can't wait for Artificial analysis benchmarks, still waiting on them adding Qwen3-max thinking, will be interesting to see how these two compare to each other
Did the ArtificialAnalysis team get bored or something? What makes a model worthy of benchmark inclusion?
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#39The non-thinking version is the best writer by far. Excited for this one! They really cooked some different from other frontier labs.
Kimi K2 has a very good model feel. Was made with taste
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#40It's good to see more competition, and open source, but I'd be much more excited to see what level of coding and reasoning performance can be wrung out of a much smaller LLM + agent as opposed to a trillion parameter one. The ideal case would be something that can be run locally, or at least on a modest/inexpensive cluster. The original mission OpenAI had, since abandoned, was to have AI benefit all of humanity, and…
I think that's where prompt engineering would be needed. Bigger models produce good output even with ambiguous prompts. Getting similar output from smaller models is art,