Live data from Hacker News

Kimi-K3 Technical Report [pdf]

github.com

191–199 of 199 posts

Re: Kimi-K3 Technical Report [pdf]

#191
post #184

Earlier quoted context omitted.

> Oh, I understand. It's the nature of the voting system of comment feedback. Reddit behaves the same way. Arguments become competitions to see who can inject enough vitriol while still maintaining a placid genteel demeanor. The one who "wins" (convinces the peanut gallery to upvote them) is the one who can avoid looking like they got mad. Depends on which subreddit and which flavor of groupthink. The behavior you sa…

>Depends on which subreddit and which flavor of groupthink. The behavior you say is upvoted is one I often see downvoted to oblivion on Reddit. Really doubt that but feel free to give an example. >If it's OK to address his comment sans the "decel", it should also be OK to address his use of "decel" without discussing the rest of the comment I think the only acceptable response is to answer his argument if you can rea…

> I think the only acceptable response is to answer his argument if you can read it.

When indulging in a discussion with others, one has to realize that the universe of what people consider acceptable responses isn't limited to yours.

Yes, it's clear that's what you think. It's also the whole point of discussion here. Merely repeating it isn't supporting your perspective.

> If you can't handle a heated commenter and insist on making his anger the discussion, then you'd have been better off not answering it. So just recognize this and ignore.

He is the one who brought the anger into the discussion, and thus it became part of the discussion. Not addressing it is a symptom of not handling it.

Do understand: Until about a decade ago, I thought like you. It caused me all kinds of headaches in the real world, and so I decided to study what an effective conversation. I took a multi-day course, as well as read several books on it.

All, without exception, point out that failure to address the emotional content is a bad idea and one of the reasons conversations become ineffective.

Re: Kimi-K3 Technical Report [pdf]

#192
post #184

Earlier quoted context omitted.

>Depends on which subreddit and which flavor of groupthink. The behavior you say is upvoted is one I often see downvoted to oblivion on Reddit. Really doubt that but feel free to give an example. >If it's OK to address his comment sans the "decel", it should also be OK to address his use of "decel" without discussing the rest of the comment I think the only acceptable response is to answer his argument if you can rea…

> I think the only acceptable response is to answer his argument if you can read it. When indulging in a discussion with others, one has to realize that the universe of what people consider acceptable responses isn't limited to yours. Yes, it's clear that's what you think. It's also the whole point of discussion here. Merely repeating it isn't supporting your perspective. > If you can't handle a heated commenter and…

>All, without exception, point out that failure to address the emotional content is a bad idea and one of the reasons conversations become ineffective.

I am not sure how you could walk away from this comment thread and still think that's the case. Minus this digression which does argue about central concepts, the discussion about the original point is pretty close to the bottom of Graham’s Hierarchy of Disagreement.

I guess maybe a better question would be why you think pivoting a conversation about how open weight models are changing the industry into a conversation about whether a commenter was rude is a positive outcome? I would understand either ignoring him or answering his content, but I'm trying to understand how getting into back and forth about tone (back in my day we call that a flame war) is a good use of one's time.

>Do understand: Until about a decade ago, I thought like you. It caused me all kinds of headaches in the real world

I've found it to be quite the opposite! Not rankling at someone's emotionally charged language in real life has been incredibly useful, both for interactions with friends and strangers, but also with family. The people I know who are miserable about visiting family at Thanksgiving feel that way because they are unable to stop themselves from getting into conflicts because their relatives made some emotionally charged statement. There are also plenty of situations where people come in hot and angry, and you can turn it around by simply letting their tone slide and trying to help them. Happened all the time when I tutored compsci students. They'd come in angry and surly because they'd been beating their head against a problem or concept, and a little patience and forbearance went a long way towards improving their attitude.

Re: Kimi-K3 Technical Report [pdf]

#193

Earlier quoted context omitted.

> We're running Kimi 2.8 on a $107k server Equipped with what? Is it CPU based inference, a mix...?

Not GP, but my educated guess is that they are running a system with between 4 and 6 MI325 or MI355x or similar AMD GPUs. With the 50k tps as the total figure for all parallel requests. Those cards have a lot of memory for their price, allowing you to push to really high batch sizes while still having a large context size for each request

4x MI300A in a Gigabyte server

Re: Kimi-K3 Technical Report [pdf]

#194

Earlier quoted context omitted.

We're running Kimi 2.8 on a $107k server and getting around 50k tokens per second or better on most things. We've already saved money compared to last years token cost on Claude/Gemini

Are you developing software? Is most of it used on a coding agent? (Like Claude Code or ChatGPT Codex?) If so, what coding agent do you use? If you're not developing software what do you use it for (roughly)?

Our use cases are data analysis, software development, general AI chat with RAG so far.

Re: Kimi-K3 Technical Report [pdf]

#195

Earlier quoted context omitted.

We're running Kimi 2.8 on a $107k server and getting around 50k tokens per second or better on most things. We've already saved money compared to last years token cost on Claude/Gemini

on 1 paralel request maybe

Correct but even with 30 simultaneous users, we're not seeing any noticable lag compared to AIaaS even with agentic workloads.

Re: Kimi-K3 Technical Report [pdf]

#196

Earlier quoted context omitted.

We're running Kimi 2.8 on a $107k server and getting around 50k tokens per second or better on most things. We've already saved money compared to last years token cost on Claude/Gemini

how many requests per second can the server take?

That's about it across the whole server. Kimi 2.8 is about as big as we can go. Kimi 3 is massive.

Re: Kimi-K3 Technical Report [pdf]

#197
post #149

Earlier quoted context omitted.

> I don’t trust hosted LLMs for anything that needs to be private I'd update this to 'I don’t LLMs for anything that needs to be private' What's to prevent the LLM from sliding a heavily obfuscated binary blob into the application that does nefarious things? If you aren't creating the LLM itself from scratch, I don't feel it can be trusted.

Why do you feel that creating the LLM from scratch is sufficient to trust it? Are you suggesting that you personally would read all 15 trillion tokens (plus every single agentic trade used in RL, along with its relative advantage in the batch) and personally guarantee that gradient descent would train a model which would not exfiltrate your corporate data? Or that perhaps you have a perfect alignment algorithm which…

> Why do you feel that creating the LLM from scratch is sufficient to trust it

If you're doing the training yourself, you at least have a verifiable supply chain and an audit trail. Today, we have no idea if a black-box model handed to us is coded to recognize specific domains or patterns and back door an application in a sneakily targeted way.

Black box is a black box. Many open-weight models clearly haven't been trained or created in the manner their creators claim---which raises the obvious question: if they lied about the recipe, what else did they lie about? Putting those models in a production capacity scares the living crap out of me.

That said, building from scratch isn't about achieving mathematical perfection or manually auditing 15 trillion tokens---that's impossible. It's about eliminating third-party supply chain risk and having actual governance over the pipeline.

Of course, that doesn't mean we can magically guarantee gradient descent won't produce weird emergent behaviors, or that we can blindly trust OpenAI not to backdoor things. But at least with the latter, you're making a calculated operational decision rather than blindly trusting an opaque black box of entirely unknown provenance.

Re: Kimi-K3 Technical Report [pdf]

#198
post #50

Earlier quoted context omitted.

This comment would be much better without the second line

I know it's a hard ask on this site, but I need you to start parsing content and not tone. It was a helpful bit of context, even if it was a bit vitriolic.

Speaking of tone, the phrase "I need you to X" is super condescending. You're not his superior, and he's not your child nor your subordinate. He is not beholden in any way to what you "need."

Much better would be to just use good ol' "please":

> I know it's a hard ask on this site, but please start parsing content and not tone

Re: Kimi-K3 Technical Report [pdf]

#199
post #197

Earlier quoted context omitted.

Why do you feel that creating the LLM from scratch is sufficient to trust it? Are you suggesting that you personally would read all 15 trillion tokens (plus every single agentic trade used in RL, along with its relative advantage in the batch) and personally guarantee that gradient descent would train a model which would not exfiltrate your corporate data? Or that perhaps you have a perfect alignment algorithm which…

> Why do you feel that creating the LLM from scratch is sufficient to trust it If you're doing the training yourself, you at least have a verifiable supply chain and an audit trail. Today, we have no idea if a black-box model handed to us is coded to recognize specific domains or patterns and back door an application in a sneakily targeted way. Black box is a black box. Many open-weight models clearly haven't been tr…

Why do you think you've prevented third party supply train risk when you're training on a 30+ Terabyte blob that you have little understanding of its contents. It's essentially entirely composed of unfiltered user input.
Post reply on HN