Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi Linear: An Expressive, Efficient Attention Architecture

github.com

Post reply on HN
Print view View original

41–50 of 50 posts

Re: Kimi Linear: An Expressive, Efficient Attention Architecture

#41

Post by naasking »

hirako2000 wrote: post #18
logicartisan wrote: post #17

Amazing how fast AI keeps improving, every new model feels like a big step forward

It solely is improving on efficiency. While it is extremely valuable given the disproportionate (to value) costs of these things, your statement almost sounds like it has improved an even more challenging aspect, pushing performance.

There has been work that has pushed performance too, like tiny recursive models. Applying LLMs in recursive loops also improves output, so efficiency improvements make this viable, which can count as improvements in performance.

Re: Kimi Linear: An Expressive, Efficient Attention Architecture

#42

Post by acuozzo »

hirako2000 wrote: post #18
logicartisan wrote: post #17

Amazing how fast AI keeps improving, every new model feels like a big step forward

It solely is improving on efficiency. While it is extremely valuable given the disproportionate (to value) costs of these things, your statement almost sounds like it has improved an even more challenging aspect, pushing performance.

> It solely is improving on efficiency.

Consider the implications of increases in efficiency *when you hold compute constant*.

The win is far more obvious when it's "we can do more with what we have" instead of "we can do the same with less".

Re: Kimi Linear: An Expressive, Efficient Attention Architecture

#44

Post by m00x »

softwaredoug wrote: post #35

Everyone is worried about AI data centers destroying the planet with their extreme energy needs. Though it seems we have a big learning curve still to make AI inference and training more efficient. How likely are we to NOT see the AI data center apocalypse through better algorithms?

I don't think this worry is widespread, or even warranted. China has been able to more than double the US in energy production without massive effects on the environment by using nuclear, solar, and hydro.

If anything, the US is massively underproducing.

Re: Kimi Linear: An Expressive, Efficient Attention Architecture

#45

Post by samus »

embedding-shape wrote: post #27
samus wrote: post #21

Earlier quoted context omitted.

We very much can, especially such a Mixture of Experts model with only 3B activated parameters. With an RTX 3070 (7GB GRAB VRAM), 32 GB RAM and an SSD I can run such models at speeds tolerable for casual use.

How many tok/s are you getting (with any runtime) with either the Kimi-Linear-Instruct or Kimi-Linear-Base on your RTX 3070?

With a Qwen3-32B-A3B (Q8) I'm getting 10-20 t/sec on KoboldAI, e.g., llama cpp. Faster than I can read, so good enough for hobby use. I expect this model to be significantly faster, but llama.cpp-based software probably doesn't support it yet.

Re: Kimi Linear: An Expressive, Efficient Attention Architecture

#46

Post by nh43215rgb »

oxqbldpxo wrote: post #29

I switched from chatgpt to Perplexity; and now to Kimi K2, after reading an article here explaining that all the fear around some of the Chinese models spying and so on.. is simply not true. I have to say that in my experience Kimi K2 is way better than perplexity. I hope we can get our act together. Seems that building this Ai's requires a level of collaboration that is in opposition to greed.

> Chinese models spying and so on.. is simply not true.

They all must be doing a great favor to humanity in a good will then.

Sorry, but seriously -- Chinese government, controlled by the Chinese Communist Party (CCP), can effectively seize or shut down internet services and infrastructure at will within its borders under its national security laws.

No need to read the TOS; it's in the law.

Re: Kimi Linear: An Expressive, Efficient Attention Architecture

#47

Post by hexaga »

meowface wrote: post #10
quotemstr wrote: post #9

Earlier quoted context omitted.

1/4 of their layers are conventional quadratic attention

Could someone explain every term in this subthread in a very simple way to someone who basically only knows "transformers are a neural network architecture that use something called 'attention' to consider the entire input the whole time or something like that", and who does not understand what "quadratic" even means in a time complexity or mathematical sense beyond that "quad" has something to do with the number fou…

There are different varieties of attention, which just amounts to some kind of learned mixing function between tokens in a sequence.

For an input of length N (tokens), the standard kind of attention requires N squared operations (hence, quadratic - it scales with the square of input length). You have to check how every token attends to every other token.

There are a bunch of alternative mixing functions which are instead linear with respect to N. Every additional token costs the same amount of work. The typical method is to have a constant size state manipulated recurrently, which necessarily implies some level of lossy compression in the state (quadratic attention doesn't really have state in this sense - it computes and checks every possible relation always).

Linear attentions kind of suck in comparison to quadratic attention but the efficiency is very attractive, especially at inference time where you don't need more VRAM to store more context.

TLDR; conventional attentions scale N^2 time, N space (kv cache), and are exact. linear attentions scale N time, constant space (recurrent state), and are lossy.

Re: Kimi Linear: An Expressive, Efficient Attention Architecture

#48

Post by ted_dunning »

m00x wrote: post #44
softwaredoug wrote: post #35

Everyone is worried about AI data centers destroying the planet with their extreme energy needs. Though it seems we have a big learning curve still to make AI inference and training more efficient. How likely are we to NOT see the AI data center apocalypse through better algorithms?

I don't think this worry is widespread, or even warranted. China has been able to more than double the US in energy production without massive effects on the environment by using nuclear, solar, and hydro. If anything, the US is massively underproducing.

The folks choking on the air pollution in Beijing might think differently.

Re: Kimi Linear: An Expressive, Efficient Attention Architecture

#49

Post by ted_dunning »

lostmsu wrote: post #33

Any comparison with existing models on common benchmarks? Text? Coding? MMLU?

Did you even look at the article?

Evaluation Benchmarks Our evaluation encompasses three primary categories of benchmarks, each designed to assess distinct capabilities of the model:

• Language Understanding and Reasoning: Hellaswag [121], ARC-Challenge [14], Winogrande [83], MMLU [36], TriviaQA [47], MMLU-Redux [26], MMLU-Pro [103], GPQA-Diamond [82], BBH [94], and [105].

• Code Generation: LiveCodeBench v6 4 [44], EvalPlus [60].

• Math & Reasoning: AIME 2025, MATH 500, HMMT 2025, PolyMath-en.

• Long-context: MRCR 5 , RULER [38], Frames [52], HELMET-ICL [118], RepoQA [61], Long Code Arena [13] and LongBench v2 [6].

• Chinese Language Understanding and Reasoning: C-Eval [43], and CMMLU [55].

Re: Kimi Linear: An Expressive, Efficient Attention Architecture

#50

Post by lostmsu »

ted_dunning wrote: post #49
lostmsu wrote: post #33

Any comparison with existing models on common benchmarks? Text? Coding? MMLU?

Did you even look at the article? Evaluation Benchmarks Our evaluation encompasses three primary categories of benchmarks, each designed to assess distinct capabilities of the model: • Language Understanding and Reasoning: Hellaswag [121], ARC-Challenge [14], Winogrande [83], MMLU [36], TriviaQA [47], MMLU-Redux [26], MMLU-Pro [103], GPQA-Diamond [82], BBH [94], and [105]. • Code Generation: LiveCodeBench v6 4 [44],…

What article? README and the linked Tech Report don't list MMLU results.
Post reply on HN