Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi-K3 Technical Report [pdf]

github.com

Post reply on HN
Print view View original

181–190 of 199 posts

Re: Kimi-K3 Technical Report [pdf]

#181

Post by throwaway27448 »

DennisP wrote: post #168
bityard wrote: post #58

Earlier quoted context omitted.

Software licenses aren't enforced through litigation as much as they are enforced through the _threat_ of litigation and legal risk. In other words, pretty much every company pays lawyers to minimize legal risk. Those lawyers inevitably look at all the contracts, agreements, and software licenses, and tell the C-suite what to do in order to keep their legal exposure as low as possible. "Don't violate other companies'…

Normal software is copyrighted. Model weights don't seem to be copyrighted, as another comment pointed out here: https://news.ycombinator.com/item?id=49073127 If weights aren't copyrighted then there's no IP to license, other than the bit of software that runs the whole thing and it's probably not too hard to write that yourself. There might even be open source reimplementations soon. Lots of companies are willing to…

I'm at a bit of a loss here—is copyright formed around semantics rather than text? What are the constraints of matching these semantics?

Re: Kimi-K3 Technical Report [pdf]

#182

Post by reissbaker »

yorwba wrote: post #141
reissbaker wrote: post #83

Earlier quoted context omitted.

I think it's pretty hard to hold that worldview: Anthropic couldn't ship a reasoning model until they copied DeepSeek R1's homework, and they've all copied DS-style super-sparse MoEs at this point too.

With slightly different cherry-picking, you could equally well claim that DeepSeek couldn't ship a reasoning model until they copied the idea from OpenAI's o1-preview, and they also copied MoEs from Google Brain/Jagellonian University https://arxiv.org/abs/1701.06538 way back in 2017, too! But ultimately these were ideas floating around in the air, if one group hadn't done the experiment, someone else would have.

No, OpenAI did not publish how they trained o1, and at the time there was significant misunderstanding and belief in the research community that they were using some kind of Monte-Carlo tree search. DeepSeek figured out GRPO on their own. Similarly, while others invented MoEs, DeepSeek's ultra-sparse variants were extremely novel, to the point where the revelation of how efficient they were to train temporarily collapsed Nvidia's stock.

Regardless I think it's impossible to believe that most LLM research was done by closed labs that don't publish, especially Anthropic (who missed out on and copied two of the largest pieces of important research of the last several years), and that none was done by open labs like DeepSeek, and that the open labs are just copycats. It's quite clear that isn't the case.

Re: Kimi-K3 Technical Report [pdf]

#183

Post by baron3dl »

GodelNumbering wrote: post #60

Back of the envelope calculation (could be off, correct me if I am) If you are a large enough company that spends million+ on inference a month, it makes sense to buy a GB300 rack ($6M on top range from what I could find) which has 20.7 TB. Since the model is mixed trained (MXFP4), you would need less than 10% of the rack's memory to serve the full model. Aggregate HBM bandwidth: 576 TB/s. You can run over 6000 paral…

I was wondering about this myself, specifically where does it make sense, so I spent a few hours working through it with Claude and wrapped it up into a writeup and interactive model. https://3dl.dev/kimi-k3.html

Re: Kimi-K3 Technical Report [pdf]

#184

Post by viccis »

BeetleB wrote: post #172
viccis wrote: post #171

Earlier quoted context omitted.

Oh, I understand. It's the nature of the voting system of comment feedback. Reddit behaves the same way. Arguments become competitions to see who can inject enough vitriol while still maintaining a placid genteel demeanor. The one who "wins" (convinces the peanut gallery to upvote them) is the one who can avoid looking like they got mad. It's why people can advocate for ethnic cleansing here, and that's fine as long…

> Oh, I understand. It's the nature of the voting system of comment feedback. Reddit behaves the same way. Arguments become competitions to see who can inject enough vitriol while still maintaining a placid genteel demeanor. The one who "wins" (convinces the peanut gallery to upvote them) is the one who can avoid looking like they got mad. Depends on which subreddit and which flavor of groupthink. The behavior you sa…

>Depends on which subreddit and which flavor of groupthink. The behavior you say is upvoted is one I often see downvoted to oblivion on Reddit.

Really doubt that but feel free to give an example.

>If it's OK to address his comment sans the "decel", it should also be OK to address his use of "decel" without discussing the rest of the comment

I think the only acceptable response is to answer his argument if you can read it. If you can't handle a heated commenter and insist on making his anger the discussion, then you'd have been better off not answering it. So just recognize this and ignore.

Re: Kimi-K3 Technical Report [pdf]

#185

Post by Ey7NFZ3P0nzAe »

SirLordBoss wrote: post #65
jvanderbot wrote: post #6

Earlier quoted context omitted.

This comment would be much better without the second line

Why? Absolutely correct, especially considering the position of the person they're referring to

I find that phrasing agressive in a way that usually minimize usefulness of further answers challenging the notion.

Re: Kimi-K3 Technical Report [pdf]

#186

Post by tesnorindian »

> Two findings in the Linux kernel illustrate the depth of these results. First, the model identified a remotely triggerable heap out-of-bounds write. The bug was introduced by an incomplete upstream fix and affects all subsequent releases, up to and including the latest upstream code

Hope these Linux Kernel bugs due to some incomplete upstream fix identified by K3 patched? Moonshot did not provide much details on this.

https://www.opensourceforu.com/2026/07/linux-maintainers-bat...

Looks like K3 is good with identifying zero days

https://www.heise.de/en/news/Kimi-K3-Chinese-AI-finds-severa...

Re: Kimi-K3 Technical Report [pdf]

#187

Post by DennisP »

throwaway27448 wrote: post #181
DennisP wrote: post #168

Earlier quoted context omitted.

Normal software is copyrighted. Model weights don't seem to be copyrighted, as another comment pointed out here: https://news.ycombinator.com/item?id=49073127 If weights aren't copyrighted then there's no IP to license, other than the bit of software that runs the whole thing and it's probably not too hard to write that yourself. There might even be open source reimplementations soon. Lots of companies are willing to…

I'm at a bit of a loss here—is copyright formed around semantics rather than text? What are the constraints of matching these semantics?

From the other comment:

> this was partially tested last year in Thaler v. Perlmutter which affirmed that copyright requires human authorship

Model weights are not authored by humans. Neither is AI output, which is why that's not copyrighted either. In another realm, a famous photo taken by a monkey was deemed by the courts to be free of copyright.

Re: Kimi-K3 Technical Report [pdf]

#188

Post by retinaros »

FuriouslyAdrift wrote: post #135
GodelNumbering wrote: post #60

Back of the envelope calculation (could be off, correct me if I am) If you are a large enough company that spends million+ on inference a month, it makes sense to buy a GB300 rack ($6M on top range from what I could find) which has 20.7 TB. Since the model is mixed trained (MXFP4), you would need less than 10% of the rack's memory to serve the full model. Aggregate HBM bandwidth: 576 TB/s. You can run over 6000 paral…

We're running Kimi 2.8 on a $107k server and getting around 50k tokens per second or better on most things. We've already saved money compared to last years token cost on Claude/Gemini

on 1 paralel request maybe

Re: Kimi-K3 Technical Report [pdf]

#189

Post by retinaros »

russell_h wrote: post #73
999900000999 wrote: post #70

Earlier quoted context omitted.

And hire 2 or 3 dev ops to keep it running ? That another 400 to 700k. It becomes your problem and not someone else’s. However, I don’t trust hosted LLMs for anything that needs to be private.

> However, I don’t trust hosted LLMs for anything that needs to be private. Why not? Do you trust AWS with things that need to be private?

aws its their business to make ur data safe. frontier lab business is to use your data and mine to train models

Re: Kimi-K3 Technical Report [pdf]

#190

Post by retinaros »

calebkaiser wrote: post #115
vanuatu wrote: post #67

Earlier quoted context omitted.

To me its clear that it is decel the only reason other labs can catch up is because the frontier labs can be distilled, and they siphon a % of the labs' revenue to reinvest into the next iteration full accel would mean nationalizing the big 2 labs and locking in manhattan project style until RSI (Edit: some great counterpoints in the replies. my view has definitely been changed!)

OpenAI's head of strategic futures publicly stated that you can't explain the quality of the newest Kimi via distillation. Further, you can just read the papers released alongside most open models. Plenty of hugely influential research results published that drive the frontier forward. It's not like these models are just existing architectures downloaded from Huggingface and trained on frontier lab APIs.

it cant be explained. but distillation shortcut training time by a lot. you need much more than that to be near SOTA obviously
Post reply on HN