Live data from Hacker News

Running LLMs with 3.3M Context Tokens on a Single GPU

Post reply on HN

Print view View original

1–4 of 4 posts

Running LLMs with 3.3M Context Tokens on a Single GPU

#1

Post by Van_Chopiszt » Tue, Oct 15, 2024, 1:35 PM UTC

Running LLMs with 3.3M Context Tokens on a Single GPU
arxiv.org

Re: Running LLMs with 3.3M Context Tokens on a Single GPU

#2

Post by [deleted] » Tue, Oct 15, 2024, 1:35 PM UTC

[deleted]

Re: Running LLMs with 3.3M Context Tokens on a Single GPU

#3

Post by [deleted] » Tue, Oct 15, 2024, 1:44 PM UTC

[deleted]

Re: Running LLMs with 3.3M Context Tokens on a Single GPU

#4

Post by charlie_xxx » Tue, Oct 15, 2024, 3:46 PM UTC

Their demo looks really cool: https://github.com/mit-han-lab/duo-attention

Post reply on HN