Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fast KV Compaction via Attention Matching

arxiv.org

Post reply on HN
Print view View original

1–10 of 17 posts

Re: Fast KV Compaction via Attention Matching

#2

Post by cadamsdotcom »

Superficially it sounds like this could create a bit more of a move toward doing compaction on some continuous basis, or compacting in batches once you hit the context limit, rather than starting fresh with a summary and system prompt..

Feels like high fidelity, fast compaction could be a path to “solving” long context.

Re: Fast KV Compaction via Attention Matching

#8

Post by mikodin »

WarmWash wrote: post #7

Considering the insanity of the AI arms race going on now, and the incredible sums of money be thrown at any slight advantage, is there any reason to believe that any meaningful AI breakthrough would be openly published for anyone to leverage?

I would say yes.

The reality is that the money being thrown = the time of humans. I guess compute as well, but in terms of people doing innovation - openly published things are the same thing, minus the money.

Re: Fast KV Compaction via Attention Matching

#9

Post by abeppu »

WarmWash wrote: post #7

Considering the insanity of the AI arms race going on now, and the incredible sums of money be thrown at any slight advantage, is there any reason to believe that any meaningful AI breakthrough would be openly published for anyone to leverage?

I do sometimes wonder -- if the transformers paper wasn't published, what would the industry be like? Would the same ideas have been put together in almost the same way weeks or months later somewhere else?
Post reply on HN