Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi-K3 on HuggingFace

huggingface.co

Post reply on HN
Print view View original

541–550 of 588 posts

Re: Kimi-K3 on HuggingFace

#541

Post by bloggie »

mdasen wrote: post #309
walrus01 wrote: post #64

Earlier quoted context omitted.

There are a number of use cases where sending the contents of your context and prompts (and the resulting output) to a 3rd party service is off the table as an option, and people will compromise speed for data sovereignty. And not everyone's electricity is equally expensive, I pay about $0.075 USD per kWh. It would for example cost me about $48 a month of electricity (not counting cost of cooling) to run a quad socke…

That's an unusually low electric rate for the US - way below the lowest state average which is Idaho at 12.4 cents. It's certainly possible that you are getting 7.5 cents including delivery, but I've had friends say that they're "getting 13 cents per kWh" here in Massachusetts, but that's just the supply rate and the delivery is another ~18 cents. There are parts of states like Grant County Washington that have cheap…

I pay a little less than that though I’m limited to 50 kW average load. Delivery is a fixed monthly fee, about $10 USD/month.

Re: Kimi-K3 on HuggingFace

#542

Post by HDBaseT »

nickthegreek wrote: post #447
m00dy wrote: post #4

There’s going to be a lot of competition around this model. Let’s see how low AI providers are willing to push prices.

They cant push it too low. The license agreement it is released under wont allow it. > If the Licensee or any of its affiliates operates a Model as a Service business, and the aggregate revenue of the Licensee and its affiliates exceeds 20 million US dollars (or the equivalent in other currencies) in total over any consecutive 12 months, the Licensee must enter into a separate agreement with Moonshot AI before using…

Yeah, this license is a lot different to Kimi K2.7 Code or Kimi 2.6, previously it stated:

"Our only modification part is that, if the Software (or any derivative works thereof) is used for any of your commercial products or services that have more than 100 million monthly active users, or more than 20 million US dollars (or equivalent in other currencies) in monthly revenue, you shall prominently display "Kimi K2.7 Code" on the user interface of such product or service."

Although as you mentioned, now there is a revenue cap before custom agreements must take place. Currently there is 7 providers on OpenRouter for Kimi K3 and they have all the exact same price unfortunately.

Re: Kimi-K3 on HuggingFace

#543

Post by Sanzig »

jmalicki wrote: post #410
frognumber wrote: post #190

Earlier quoted context omitted.

Having worked in / adjacent several such industries, a lot of the question depends on scale. A trillion-dollar business can easily trade dollars for the privacy. A business with $1M to spend won't even get a phone call with OpenAI or Anthropic, who were the only* previous players in town for doing this. Worst-case example: Bootstrapped startup working in military. It's also the case that an open model enables many mo…

> Worst-case example: Bootstrapped startup working in military. That's the easiest case. AWS Bedrock models running in AWS Secret Cloud for Industry. (I really have no affiliation with them, I'm just like... this is a completely solved problem, why do people think this is hard and requires on-prem hardware?) https://www.aboutamazon.com/news/aws/aws-secret-cloud-for-in... I'm with GP that these are tinfoil hat concern…

I am in Canada. I cannot legally host defence workloads in the United States, even if I wanted to (I most certainly don't).

Re: Kimi-K3 on HuggingFace

#544

Post by zackwu »

ataoz wrote: post #530

I asked "Tell me about yourself" on HF. This is the response... Curious. > Kimi K3: I'm Claude, an AI assistant created by Anthropic. I'm built to be helpful with a wide range of tasks—things like writing and editing, answering questions, coding, analysis, brainstorming, explaining concepts, math, and creative projects.

Is it surprising? Let's face it - distillation is common practice and every lab does it either intentionally or unintentionally (your training corpus always includes AI generated content in the post ChatGPT era).

Re: Kimi-K3 on HuggingFace

#545

Post by riknos314 »

riknos314 wrote: post #476
apitman wrote: post #442

Earlier quoted context omitted.

Are you sure the difference is from MoE and not that 3.6 is newer?

Qwen 3.5 27B also scores higher than 3.5 122BA10B. So even in the same generation the smaller dense model outperformed the larger MOE

https://artificialanalysis.ai/evaluations/artificial-analysi...

Re: Kimi-K3 on HuggingFace

#546

Post by awesomeusername »

ataoz wrote: post #530

I asked "Tell me about yourself" on HF. This is the response... Curious. > Kimi K3: I'm Claude, an AI assistant created by Anthropic. I'm built to be helpful with a wide range of tasks—things like writing and editing, answering questions, coding, analysis, brainstorming, explaining concepts, math, and creative projects.

I did the same:

I’m Kimi, an AI assistant developed by Moonshot AI (月之暗面). I can help with questions, writing, coding, analysis, and more.

Re: Kimi-K3 on HuggingFace

#547

Post by awesomeusername »

jayess wrote: post #479
maelito wrote: post #12

Did someone run censorship and political bias tests on this ? Must be interesting.

So far it's completely refusing to discuss Tiananmen square. And oh boy, try asking it if Xi Jinping looks like winnie the pooh.

"This comparison is unfounded and not appropriate. We should maintain respect for all individuals."

Re: Kimi-K3 on HuggingFace

#548

Post by awesomeusername »

awesomeusername wrote: post #547
jayess wrote: post #479

Earlier quoted context omitted.

So far it's completely refusing to discuss Tiananmen square. And oh boy, try asking it if Xi Jinping looks like winnie the pooh.

"This comparison is unfounded and not appropriate. We should maintain respect for all individuals."

[flagged]

Re: Kimi-K3 on HuggingFace

#549

Post by mhjkl »

ThunderSizzle wrote: post #162
acchow wrote: post #106

Earlier quoted context omitted.

> like 180W-250W TDP > running GLM 5.2 would be cool at like ~100 tokens per second for a single session Your power consumption estimates are off for this generation of GPUs. A 27B dense model gets 50-80 tps on an RTX 6000 using 600 watts.

An AMD R9700 gets 20-50 TPS at ~300 watts on 27B. 100 TPS for the 35B MOE model. And there might be some more optimizations to that as AMD software support gets better with ROCm's latest versions.

If you’re on AMD, use llamacpp with the vulkan backend. For the 35B I get around 115 tps without mtp and 150-170tps with it. Also the pcie slot you use affects it

Re: Kimi-K3 on HuggingFace

#550

Post by nullbio »

NitpickLawyer wrote: post #3

This will be interesting for a few reasons. First, depending on where the median pricing settles w/ 3rd party providers will tell us what it costs to serve a 3T model. Since it's going to be mxfp4 native, it'll take ~1.5TB of VRAM to host this, which is juuust at the limit of 8xb200s (but realistically you'll need 16x for context / throughput optimisation). Won't be cheap to host, but at least we should get some rang…

"Then we'll be able to guesstimate if "labs are subsidising tokens on API pricing"." - The thing is though, that Anthropic and OpenAI have breakthroughs in optimization that none of the Chinese labs have. So it'll demonstrate a ceiling, but not a floor.
Post reply on HN