[flagged]
I think alignment in this context refers to distribution alignment: “aligning” it so its outputs more efficiently steer the target model with less tokens
LLMLingua: Compressing Prompts for Faster Inferencing
11–20 of 51 posts
Re: LLMLingua: Compressing Prompts for Faster Inferencing
#12LLMLingua uses a well-trained small language model after alignment, such as GPT2-small or LLaMA-7B, to detect the unimportant tokens in the prompt and enable inference with the compressed prompt in black-box LLMs, achieving up to 20x compression with minimal performance loss.
Came here to mention this. Whenever I hear "alignment" I immediately say "No way am I going to use that shit". Seriously, there's alignment and then there's censorship—the AI creators are using the former when they actually mean the latter. This needs to stop.
My understanding is that in an academic context you’ll hear alignment anytime a model is tuned to accomplish a certain task, not just to steer its political affiliation and idea of ethics
I don’t think this models use of alignment implies any sort of censorship, it’s just being tuned to accomplish the task of outputting only important tokens for the target llm
Re: LLMLingua: Compressing Prompts for Faster Inferencing
#13Wild. if I'm reading this correctly it's effectively a sort of "zip" algorithm for both the inputs and outputs of a prompt based model. thus, it allows a user to compress their request down to the minimal token size which retains the same semantics. In effect, this then allows a user to encode a more dense set of tokens into the original request. Does that sound about right?
Sounds right to me. I think it’s fun that is this may be the only compression algorithm where the output is still human understandable.
It reads like a slightly garbled version of what someone writing down bullet point notes of a lecture might write.
It’s so rare that the human optimized and machine optimized versions of an input are so similar
Re: LLMLingua: Compressing Prompts for Faster Inferencing
#14Earlier quoted context omitted.
Came here to mention this. Whenever I hear "alignment" I immediately say "No way am I going to use that shit". Seriously, there's alignment and then there's censorship—the AI creators are using the former when they actually mean the latter. This needs to stop.
My understanding is that in an academic context you’ll hear alignment anytime a model is tuned to accomplish a certain task, not just to steer its political affiliation and idea of ethics I don’t think this models use of alignment implies any sort of censorship, it’s just being tuned to accomplish the task of outputting only important tokens for the target llm
[flagged]
Re: LLMLingua: Compressing Prompts for Faster Inferencing
#15Earlier quoted context omitted.
My understanding is that in an academic context you’ll hear alignment anytime a model is tuned to accomplish a certain task, not just to steer its political affiliation and idea of ethics I don’t think this models use of alignment implies any sort of censorship, it’s just being tuned to accomplish the task of outputting only important tokens for the target llm
[flagged]
I’m really not all that familiar with the space so I could be mistaking. The definition of ai alignment on Wikipedia says an aligned model one that “advances intended objectives”.
In the paper, “distribution alignment” is one of methods used to improve the results of compression so intent is preserved:
> To narrow the gap between the distribution of the LLM and that of the small language model used for prompt compression, here we align the two distributions via instruction tuning
So in any case for this paper alignment seems to be used in very specific way that doesn’t seem related to censorship
Edit: would to love to hear from someone who has a better understanding of the paper to clarify. I am operating from the position of layman here
Re: LLMLingua: Compressing Prompts for Faster Inferencing
#16This always seemed like the end game vs. getting a degree in prompt engineering. If you get enough data on "initial prompt attempt" -> "final successful prompt", the whole thing can be replaced by a fine tuned model. You would just select a "prompt rewritter llm" that optimizes for accuracy, cost, alignment etc.
GPT on top of GPT. It is turtles all the way down.
Re: LLMLingua: Compressing Prompts for Faster Inferencing
#17Earlier quoted context omitted.
My understanding is that in an academic context you’ll hear alignment anytime a model is tuned to accomplish a certain task, not just to steer its political affiliation and idea of ethics I don’t think this models use of alignment implies any sort of censorship, it’s just being tuned to accomplish the task of outputting only important tokens for the target llm
[flagged]
It is common, standard usage precisely in this context.
Re: LLMLingua: Compressing Prompts for Faster Inferencing
#18Very interesting, we've started on an approach to enable LLM agents communicate and context share in their own language, but I think calling it compression is actually more intuitive. I love this
Re: LLMLingua: Compressing Prompts for Faster Inferencing
#19[flagged]
Stop writing the same inane comment. Nobody cares that you don't understand alignment nor that because of the misunderstanding and your political beliefs you see it as the devil.
Re: LLMLingua: Compressing Prompts for Faster Inferencing
#20LLMLingua uses a well-trained small language model after alignment, such as GPT2-small or LLaMA-7B, to detect the unimportant tokens in the prompt and enable inference with the compressed prompt in black-box LLMs, achieving up to 20x compression with minimal performance loss.
“Why waste time say lot word when few word do trick” -Kevin Malone
Perfection. Key insight. "Few Word [is] All Need" (with a robust enough foundation model)
Linked for the culture: https://www.youtube.com/watch?v=bctjSvn-OC8&t=4s
Sleep big last night