Live data from Hacker News

Kimi-K3 on HuggingFace

huggingface.co

471–480 of 588 posts

Re: Kimi-K3 on HuggingFace

#471

Earlier quoted context omitted.

The core intuition here is probably that you can not separate reasoning and domain knowledge. How are you going to use tools, if you don't have the context to use them? Imaging if you could only think in terms of lambda calculus, and was asked to check the weather to let me know if the weekend is good for a hike.

I think the idea here would be to not use your super smart but specialized model to check the weather. It's not obvious that it's impossible to (eg) remove most of its biology knowledge, without removing much of its ability to develop software (for example). If you're developing biology software, then don't use that particular compressed model. (If you're claiming that it is impossible, and you have references you ca…

I don't think that is true.

I think more of the capabilities comes from the cross domains knowledge.

Anyways, software is also designed for a domain. The reason why it is so adept at making a fitness tracker is likely because it knows about biology.

Re: Kimi-K3 on HuggingFace

#472
post #309

Earlier quoted context omitted.

There are a number of use cases where sending the contents of your context and prompts (and the resulting output) to a 3rd party service is off the table as an option, and people will compromise speed for data sovereignty. And not everyone's electricity is equally expensive, I pay about $0.075 USD per kWh. It would for example cost me about $48 a month of electricity (not counting cost of cooling) to run a quad socke…

That's an unusually low electric rate for the US - way below the lowest state average which is Idaho at 12.4 cents. It's certainly possible that you are getting 7.5 cents including delivery, but I've had friends say that they're "getting 13 cents per kWh" here in Massachusetts, but that's just the supply rate and the delivery is another ~18 cents. There are parts of states like Grant County Washington that have cheap…

Not the parent, but here is one location that has rates in that range in the U.S.

https://casscountyelectric.com/rates

Re: Kimi-K3 on HuggingFace

#473
Many people are talking about price, but I think that the most interesting aspect of this release, by far, is customization.

Any startup can download the weights, tinker with them, and fine-tune them. The real win here isn't necessarily cost, but performance on your data and IP sovereignty. It's a huge win. Kudos to the Kimi team.

Re: Kimi-K3 on HuggingFace

#474

Earlier quoted context omitted.

I think the idea here would be to not use your super smart but specialized model to check the weather. It's not obvious that it's impossible to (eg) remove most of its biology knowledge, without removing much of its ability to develop software (for example). If you're developing biology software, then don't use that particular compressed model. (If you're claiming that it is impossible, and you have references you ca…

I don't think that is true. I think more of the capabilities comes from the cross domains knowledge. Anyways, software is also designed for a domain. The reason why it is so adept at making a fitness tracker is likely because it knows about biology.

Sure, that's a plausible theory but I haven't seen that anyone's proved it.

An alternate theory is that models need lots of input knowledge to learn complex reasoning, but don't need so much at inference time. An example is arithmetic. Early in their training, models do arithmetic (poorly) by pattern-matching on memorized examples. Eventually they grok arithmetic and stop pattern matching, and then they don't use the examples anymore.

The fitness tracker doesn't take much knowledge of biology. Large models have quite a lot of biology knowledge that most people will never use. Same goes for lots of other topics. For basic knowledge that easily fits in context it could search the internet, or a local collection of introductory textbooks.

Re: Kimi-K3 on HuggingFace

#475

It is online on https://app.fireworks.ai/models/fireworks/kimi-k3 (Uncached Input $3.00/M Cached Input $0.30/M Output $15.00/M)

I love fireworks.ai! They launched it couple of hours ago and we have it now already live on our platform for our users. Just a shame they deprecated the on-demand flux models :( Where do I get my fix for image gen now?

Re: Kimi-K3 on HuggingFace

#476

Earlier quoted context omitted.

This tracks, in my experience the 27B is better at coding and instruction following. I'm shocked at how much of a difference the dense models vs MoE makes. But it's a moot point, because for local inference on consumer hardware, the MoE is so much faster.

Are you sure the difference is from MoE and not that 3.6 is newer?

Qwen 3.5 27B also scores higher than 3.5 122BA10B. So even in the same generation the smaller dense model outperformed the larger MOE

Re: Kimi-K3 on HuggingFace

#477

Earlier quoted context omitted.

> It's worse than that: a typical MoE model routes a separate set of experts at every layer, not just every token! But in practice, RAM offload (for systems with non-unified VRAM) and even SSD offload still work surprisingly well given some amount of caching. Caching really has nothing to do with this. With RAM offload you can mostly benefit from: 1) Batching for prefill is a huge win, even with MoE, since the batch…

> With MoE it's low probability. For even the sparsest MoE open models, having more than a handful of inferences in the batch is enough to make it more likely than not that you'll get some MoE weight reuse within any given layer. This assumes totally random sampling, ignoring any cross-request correlation that would push that probability even higher in many practical scenarios. > With MTP it becomes _extremely_ low p…

> For even the sparsest MoE open models, having more than a handful of inferences in the batch is enough to make it more likely than not that you'll get some MoE weight reuse within any given layer. This assumes totally random sampling, ignoring any cross-request correlation that would push that probability even higher in many practical scenarios.

If you tell me the model and the number of parallel streams, I will do the math.

Re: Kimi-K3 on HuggingFace

#478

Earlier quoted context omitted.

good find! This sounds a bit like what Meta was doing with the earlier Llama models? There is also this paragraph in their licence that is smart marketing-wise: > 3. If the Software (or any derivative works thereof) is used for any of the Licensee's commercial products or services that have more than 100 million monthly active users, or more than 20 million US dollars (or equivalent in other currencies) in monthly re…

Is that even enforceable?

IANAL, but probably not, at least not in the United States. Under U.S. copyright law, the weights of machine learning models are excluded from copyright as they are the product of an automated optimization process (e.g., stochastic gradient descent, expectation maximization, genetic algorithms) rather than human authorship. Granted, this has yet to be fully tested in court and going to court is expensive, so it's likely that your employer would prefer to err on the side of caution and respect such attempts at model licensing anyway. Nonetheless, this was partially tested last year in Thaler v. Perlmutter which affirmed that copyright requires human authorship, reading the Copyright Act's provisions on ownership, duration, and transferability as presupposing a human author[1].

If you want to assess the position of the U.S. Copyright Office for yourself, the relevant text can be found in the Compendium of U.S. Copyright Office Practices § 313.2, "Works That Lack Human Authorship"[2], which states:

> […] the Copyright Act protects “original works of authorship.” 17 U.S.C. § 102(a) (emphasis added). To qualify as a work of “authorship” a work must be created by a human being. See Burrow-Giles Lithographic Co., 111 U.S. at 58. Works that do not satisfy this requirement are not copyrightable.

> […] the Office will not register works produced by a machine or mere mechanical process that operates randomly or automatically without any creative input or intervention from a human author. The crucial question is “whether the ‘work’ is basically one of human authorship, with the computer [or other device] merely being an assisting instrument, or whether the traditional elements of authorship in the work (literary, artistic, or musical expression or elements of selection, arrangement, etc.) were actually conceived and executed not by man but by a machine.” U.S. COPYRIGHT OFFICE, REPORT TO THE LIBRARIAN OF CONGRESS BY THE REGISTER OF COPYRIGHTS 5 (1965).

Oh, and there's also a bit in the following Section 313.3, "Works That Do Not Constitute Copyrightable Subject Matter"[2], which explicitly excludes mathematical principles, formulas, algorithms, and equations, along with DNA sequences and other genetic or chemical compounds, regardless of whether they are produced by humans or by nature. If one takes the perspective that machine learning models are algorithms, the conclusions on copyrightability are pretty clear.

[1] https://media.cadc.uscourts.gov/opinions/docs/2025/03/23-523...

[2] https://www.copyright.gov/comp3/chap300/ch300-copyrightable-...

Re: Kimi-K3 on HuggingFace

#479
post #12

Did someone run censorship and political bias tests on this ? Must be interesting.

So far it's completely refusing to discuss Tiananmen square. And oh boy, try asking it if Xi Jinping looks like winnie the pooh.

Re: Kimi-K3 on HuggingFace

#480
post #309

Earlier quoted context omitted.

There are a number of use cases where sending the contents of your context and prompts (and the resulting output) to a 3rd party service is off the table as an option, and people will compromise speed for data sovereignty. And not everyone's electricity is equally expensive, I pay about $0.075 USD per kWh. It would for example cost me about $48 a month of electricity (not counting cost of cooling) to run a quad socke…

That's an unusually low electric rate for the US - way below the lowest state average which is Idaho at 12.4 cents. It's certainly possible that you are getting 7.5 cents including delivery, but I've had friends say that they're "getting 13 cents per kWh" here in Massachusetts, but that's just the supply rate and the delivery is another ~18 cents. There are parts of states like Grant County Washington that have cheap…

I'm actually getting 11 cents in winter, 13 in summer, but my utility company is a co-op. Average for my state is I think 19 cents.

I think you can get down to around 8 if you are signed up for an interruptible load, or a dedicated off peak load, depending on the company, but yeah, standard rates aren't that low.

Post reply on HN