Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs know more than what they say

arjunbansal.substack.com

Post reply on HN
Print view View original

11–19 of 19 posts

Re: LLMs know more than what they say

#11

Post by ruby314 »

sanxiyn wrote: post #7
ruby314 wrote: post #6

Earlier quoted context omitted.

Thanks for sharing the reference. It's definitely related! But the approach in that paper is different in a couple of very significant ways: 1. They train a classifier on top of the activations of the base LLM. so you would have to have access to the activations of the LLM you are using for the task. We use the activations of an open evaluator LLM (e.g. Llama-3.1-8B-Instruct) 2. Our features are projections - a few n…

I didn't mean to suggest it as a competition to the method presented (LSR: latent space readout). It is old after all. LSR's use in evaluator LLM and working with small samples (because it works with linear direction) does seem novel and useful to me. An advantage of being aware of early papers is that it accumulates citations so you can often find good works in reverse citations. I had a brief look and the following…

and yeah, we fully acknowledge that our list of references is not comprehensive :) thanks again for sharing!

Re: LLMs know more than what they say

#12

Post by eric_gu »

ruby314 wrote: post #9
eric_gu wrote: post #8

Earlier quoted context omitted.

This is cool! I have not read up on evaluation techniques that use LLM-as-a-Judge, so I hadn't heard of the term "evaluator LLM" before. Questions that came to mind: - How are you deciding on the positive/negative concept pairs to generate your latent "evaluation direction?" - What layer of activations on the evaluator model do you use—the output layer? - What base model are you using for solving the HaluEval task? -…

Ty! - We generate contrast pairs (for this post, using gpt-4o) and do some post processing for quality (synthetic data). The impact of different types of contrast pairs is a continuing area of research for us. - We treat the evaluator model layers as hyper parameters, similar to the steering research (some of which we cite in our “non-comprehensive list of references”). We also see that the middle layers tend to be m…

Thanks for the reply!

Re question #3: I'm not sure I understand why you need to vary the base model or how doing so would allow LSR to take advantage? Isn't your LSR technique used on the activations of the evaluator model?

As a note of feedback, I found the original article a bit hard to understand even with multiple reads. I would have really benefited from a traditional "methodology" section like in an ML paper! The graphs upfront don't make sense to someone who isn't familiar with the problem setting, and even now I'm not sure if the x-axis in the HaluEval Benchmark bar chart refers to the base model or the evaluator model. Maybe it's just me.

Re: LLMs know more than what they say

#13

Post by ruby314 »

eric_gu wrote: post #12
ruby314 wrote: post #9

Earlier quoted context omitted.

Ty! - We generate contrast pairs (for this post, using gpt-4o) and do some post processing for quality (synthetic data). The impact of different types of contrast pairs is a continuing area of research for us. - We treat the evaluator model layers as hyper parameters, similar to the steering research (some of which we cite in our “non-comprehensive list of references”). We also see that the middle layers tend to be m…

Thanks for the reply! Re question #3: I'm not sure I understand why you need to vary the base model or how doing so would allow LSR to take advantage? Isn't your LSR technique used on the activations of the evaluator model? As a note of feedback, I found the original article a bit hard to understand even with multiple reads. I would have really benefited from a traditional "methodology" section like in an ML paper! T…

Re #3 - my bad, mixing terminology in my answer above. It’s the “base model” for the evaluator model (vs a fine tuned evaluator model). Just using the labeled Halubench dataset as the outputs to be evaluated, so no base model for the Halueval task.

Thanks for the feedback, really helpful. We may edit for clarity.

Re: LLMs know more than what they say

#15

Post by coffeephoenix »

tarasglek wrote: post #14

This looks cool, but I'm confused as to how this is surfaced in your product, llama-8 is not present in your model list. I thought maybe you offer hallucination detection, but I also don't see that. RAG evals also not visible

Thanks for the q.

The LSR version of AutoFeedback is on our Pro tier. You can sign up for more info on that here: https://tally.so/r/w2XVQA

In the free trial of Log10.io you can use the ICL (self-improving) version of AutoFeedback for hallucination detection and RAG evals. In generalize, you can customize evals for a wide range of use cases using AutoFeedback: https://docs.log10.io/feedback/auto_feedback https://github.com/log10-io/log10-cookbook/tree/main https://arjunbansal.substack.com/p/hybrid-evaluation-scaling...

As for models, we support evaluating a wide range of models including Llama - in addition we support OpenAI, Anthropic, Gemini, Mistral, MosaicML, Together, and self-hosted models. We are also compatible with frameworks such as Langchain, Magentic and LiteLLM. For how to integrate different models please see here: https://github.com/log10-io/log10

Re: LLMs know more than what they say

#16

Post by ruby314 »

coffeephoenix wrote: post #15
tarasglek wrote: post #14

This looks cool, but I'm confused as to how this is surfaced in your product, llama-8 is not present in your model list. I thought maybe you offer hallucination detection, but I also don't see that. RAG evals also not visible

Thanks for the q. The LSR version of AutoFeedback is on our Pro tier. You can sign up for more info on that here: https://tally.so/r/w2XVQA In the free trial of Log10.io you can use the ICL (self-improving) version of AutoFeedback for hallucination detection and RAG evals. In generalize, you can customize evals for a wide range of use cases using AutoFeedback: https://docs.log10.io/feedback/auto_feedback https://gith…

Also wanted to address the confusion on the role of llama-3-8b and llama-3.1-8b. In the blog post we use these models as an example of an evaluator LLM. We select what is best for your custom eval under the hood. LSR is just one example of the research powering our custom evals

Re: LLMs know more than what they say

#17

Post by autokad »

ruby314 wrote: post #4
autokad wrote: post #3

if what I understand is correct, that they project the LLM's internal activations into meaningful linear directions derived from contrasting examples, I guess this is similar to how we began to derive a lot more value from ebeddings by using the embedding values for various things.

yes that's correct! we project an evaluator LLM's internal activations onto meaningful linear directions, derived from contrasting examples. the strongest connection is to LLM interpretability (existence of meaningful linear directions) and steering research (computation from contrast pairs). This has been done with base model activations to understand base model behavior, but we show you can boost evaluation accurac…

this is pretty interesting work, I am curious to what emerges from this kind of thing in the future. I have been working on something 'similar' that involves the same problem, how do you evaluate a response without a source of truth, and this gets at that.

I was looking at it from the stand point of the embeddings of the output with different temperatures

Re: LLMs know more than what they say

#18

Post by eric_gu »

ruby314 wrote: post #13
eric_gu wrote: post #12

Earlier quoted context omitted.

Thanks for the reply! Re question #3: I'm not sure I understand why you need to vary the base model or how doing so would allow LSR to take advantage? Isn't your LSR technique used on the activations of the evaluator model? As a note of feedback, I found the original article a bit hard to understand even with multiple reads. I would have really benefited from a traditional "methodology" section like in an ML paper! T…

Re #3 - my bad, mixing terminology in my answer above. It’s the “base model” for the evaluator model (vs a fine tuned evaluator model). Just using the labeled Halubench dataset as the outputs to be evaluated, so no base model for the Halueval task. Thanks for the feedback, really helpful. We may edit for clarity.

Ah understood. Makes sense now!

Re: LLMs know more than what they say

#19

Post by uiDevofNW »

This is a stupid arguement. I wish author understood an ounce of how LLMs works. Of course, they know more than whay they say. That's because LLMs are nothing but probabistic structures. They mix and match and provide probabilistic approach. Therefore, they are always making a choice between multiple options.

I wish there was a global mandatory course before these substacky authors write for fame.

Post reply on HN