Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong

github.com

Post reply on HN
Print view View original

21–30 of 55 posts

Re: Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong

#21

Post by somenameforme »

jurgenburgen wrote: post #18
mistrial9 wrote: post #16

Earlier quoted context omitted.

opinion is ephemeral, cannot be independently confirmed or denied; purely single viewer information sounds like not-a-fact to me Opinion is a ghost in the machine—fleeting, unanchored, and vanishing upon inspection. It leaves no footprint for another mind to verify, no independent trail of evidence to confirm or deny its existence. To rely on such solitary perception is to drift in a void where facts do not exist; it…

It seems to me trivial to verify: just ask.

Exactly. A fact is something that can be objectively confirmed or rejected. Exactly what a specific person likes is a fact, because it can be verified/falsified by simply asking them. An opinion, by contrast is 'Elephants are large.' The reason that's an opinion is because while it seems obviously true, it's subjective. Similarly, grass is purple with pink polka-dots is a fact. It's obviously false, but of course right/wrong has nothing to do with fact/opinion.

Re: Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong

#22

Post by HenryNdubuaku »

zdw wrote: post #8

Have you benched this for coding tasks, with a fallback to a larger local model, for example Qwen-3.6-27B? Or using it for sub-tasks, where a framework with a larger primary model dispatches simpler jobs ("summarize ...", etc.) to it?

We are currently working on larger models atm! your suggestion is cool btw

Re: Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong

#24

Post by HenryNdubuaku »

mncharity wrote: post #10

> the hidden state for different layers carry meaningful self-awareness signal for various situations. Is it plausible to wonder if some developer judgement feels, like maybe "the code I just wrote is clean/crufty", or "things came together smoothly/janky", might have extractable signals in some models? If so, might one create a shopping list of desired signals to check for in a model, as with activation steering con…

YOu are thinking along the right direction, we are going deeper into the signals.

Re: Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong

#26

Post by antonvs »

mistrial9 wrote: post #16
BugsJustFindMe wrote: post #15

Earlier quoted context omitted.

> pure opinion (I like green) "I like green" is a statement of fact about what you like. "Green is good" would be an opinion. It's funny to me how often people mix the two. > forms can be filtered by these classes fairly easily Obviously not so easily. :)

opinion is ephemeral, cannot be independently confirmed or denied; purely single viewer information sounds like not-a-fact to me Opinion is a ghost in the machine—fleeting, unanchored, and vanishing upon inspection. It leaves no footprint for another mind to verify, no independent trail of evidence to confirm or deny its existence. To rely on such solitary perception is to drift in a void where facts do not exist; it…

"Ephemeral" isn't relevant - a statement of fact can be true at one time and false at another.

Confirmation also is somewhat orthogonal: there can be true facts that we can't confirm.

Conventionally, "I like green" is considered a factual claim about the speaker's preferences. Even if you refuse to acknowledge it as a fact because of the restrictions you've given, that doesn't make it an opinion. It's subjective, but it's not an opinion.

Would you say that if someone is experiencing a headache, and they say "I have a headache", that that's also merely an opinion?

Re: Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong

#27

Post by naasking »

jurgenburgen wrote: post #18
mistrial9 wrote: post #16

Earlier quoted context omitted.

opinion is ephemeral, cannot be independently confirmed or denied; purely single viewer information sounds like not-a-fact to me Opinion is a ghost in the machine—fleeting, unanchored, and vanishing upon inspection. It leaves no footprint for another mind to verify, no independent trail of evidence to confirm or deny its existence. To rely on such solitary perception is to drift in a void where facts do not exist; it…

It seems to me trivial to verify: just ask.

Opinions can only be seen as facts when modelled as time series data. Verifying an opinion X at time T does not entail X at time T+1.

Re: Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong

#29

Post by pjc50 »

BugsJustFindMe wrote: post #12

> "post-trained to know when it's wrong" Is it also post-trained to know when it's wrong about when it's wrong? > "Every response comes with a confidence score between 0 and 1" How confident is it in its confidence? Please, I'm sure that what you're doing is very neat and useful, but use other language to describe it. I beg you. You can't know when you're wrong. You can only know when you're unsure or inconsistent. Y…

This is an old problem in philosophy, one that the pre-LLM systems were interested in, which got completely bypassed by the present approach. What is knowledge? It's not a simple question even for humans, we don't have a magic truth detector in the brain.

https://plato.stanford.edu/entries/knowledge-analysis/

This is why LLMs are more successful in programming than other disciplines: it's easier to assess correctness of results, through proofs, typechecking, and testing. Similarly for maths and applied maths. Much harder in the humanities.

Re: Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong

#30

Post by dmrivers »

Cool idea, some feedback:

1. Consider using conformal prediction to calibrate the cutoff. Conformal prediction provides a distribution-free guarantee under exchangeability. This would let you turn your raw probe score into a threshold with a guaranteed bound on the rate of wrongly-kept on-device answers. Source: https://en.wikipedia.org/wiki/Conformal_prediction

2. The best indicators of confidence in ML come from multiple independent methods. What was the result if you combine the token entropy and verbal confidence reporting methods? Does this improve the result?

3. I noticed you didn't mention the assessment method of rerunning the model and judging whether outputs are consistent. How does that method compare in terms of AUROC?

Post reply on HN