New LLM optimization technique slashes memory costs
venturebeat.com
New LLM optimization technique slashes memory costs
1–10 of 227 posts
Re: New LLM optimization technique slashes memory costs
#2[flagged]
Re: New LLM optimization technique slashes memory costs
#3Re: New LLM optimization technique slashes memory costs
#4TFP: https://arxiv.org/abs/2410.13166
Re: New LLM optimization technique slashes memory costs
#5Wonder how this compares with Microsoft's HeadKV paper [1] which claims a 98% percent reduction in memory while retaining 97% of the performance.
Re: New LLM optimization technique slashes memory costs
#6This only decreases memory cost of input context window, not the memory cost to load and run the models.
Re: New LLM optimization technique slashes memory costs
#7This is for inference right? Not training?
Re: New LLM optimization technique slashes memory costs
#8[flagged]
Re: New LLM optimization technique slashes memory costs
#9This only decreases memory cost of input context window, not the memory cost to load and run the models.
Context window requires ram too.
Re: New LLM optimization technique slashes memory costs
#10interesting