Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong

github.com

Post reply on HN
Print view View original

31–40 of 55 posts

Re: Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong

#31

Post by shakna »

mistrial9 wrote: post #13
BugsJustFindMe wrote: post #12

> "post-trained to know when it's wrong" Is it also post-trained to know when it's wrong about when it's wrong? > "Every response comes with a confidence score between 0 and 1" How confident is it in its confidence? Please, I'm sure that what you're doing is very neat and useful, but use other language to describe it. I beg you. You can't know when you're wrong. You can only know when you're unsure or inconsistent. Y…

IMHO this also depends on the type of assertion.. For example, one system defines three broad classes of assertion : pure opinion (I like green); factual references (the current temperature here is 20c) ; and the rest in a catch-all about "reasoned judgements" Complex assertions and their compounded forms can be filtered by these classes fairly easily. I rarely see this referenced in general comments.

The catch-all is a problem. We've been studying fallacies in arguments since before the time of Plato, and do not have a complete answer that always works, to prevent the logical failure.

Complex assertions and compounding arguments, can still result in circular reasoning - the petitio principii, or worse.

Most LLMs are guided by strong statements like it must do this or that. Which assumes that it won't apply the fallacy of accident, even though people regularly do. (Don't agree? Argue a definition with a BA for a day and come back to me.)

Re: Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong

#34

Post by embedding-shape »

pjc50 wrote: post #29
BugsJustFindMe wrote: post #12

> "post-trained to know when it's wrong" Is it also post-trained to know when it's wrong about when it's wrong? > "Every response comes with a confidence score between 0 and 1" How confident is it in its confidence? Please, I'm sure that what you're doing is very neat and useful, but use other language to describe it. I beg you. You can't know when you're wrong. You can only know when you're unsure or inconsistent. Y…

This is an old problem in philosophy, one that the pre-LLM systems were interested in, which got completely bypassed by the present approach. What is knowledge? It's not a simple question even for humans, we don't have a magic truth detector in the brain. https://plato.stanford.edu/entries/knowledge-analysis/ This is why LLMs are more successful in programming than other disciplines: it's easier to assess correctness…

> successful in programming than other disciplines

It seems successful specifically in "computer science", for "programming" which encompasses a much wider field, "successful" is still up for debate. Also, we haven't even figured out yet what "success" even means in a software development process, for some it's that the program does everything the users want, in the way they expect, for other's, it's about correctness, reliability, performance, composability, maintainability and/or a whole host of other "measures".

Re: Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong

#35

Post by jstanley »

zamadatix wrote: post #32
HenryNdubuaku wrote: post #23

Earlier quoted context omitted.

no because we freeze the main weights and only train our added weights.

It's impossible to add weights which degrade the quality?

Not if the new weights don't influence the old outputs.

Re: Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong

#36

Post by cmrdporcupine »

NitpickLawyer wrote: post #20
BugsJustFindMe wrote: post #12

> "post-trained to know when it's wrong" Is it also post-trained to know when it's wrong about when it's wrong? > "Every response comes with a confidence score between 0 and 1" How confident is it in its confidence? Please, I'm sure that what you're doing is very neat and useful, but use other language to describe it. I beg you. You can't know when you're wrong. You can only know when you're unsure or inconsistent. Y…

A few years ago, before RLVR became ubiquitous, people were using generate -> classify loops to get models to improve math reasoning. Turns out classification is sometimes easier to do and more accurate than generating coherent traces. So you'd get a model to "think step by step", then run each step (.split('/n/n')) through a classifier, and tell the model "this step might be wrong, re-check". If you're interested I…

> And youc an get some signal out of it, and have it re-do a step.

Or the opposite https://arxiv.org/abs/2606.06574v1

Re: Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong

#38

Post by maxgashkov »

I don't really get why you need handoff if your score is accurate. If it is, and it is low for a given response, just let the harness re-run the prompt with a different seed until the score is high enough. If this approach doesn't work, your score is most likely garbage.

Re: Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong

#39

Post by HenryNdubuaku »

maxgashkov wrote: post #38

I don't really get why you need handoff if your score is accurate. If it is, and it is low for a given response, just let the harness re-run the prompt with a different seed until the score is high enough. If this approach doesn't work, your score is most likely garbage.

Smaller models in production can produce quite interesting outputs.

Re: Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong

#40

Post by HenryNdubuaku »

dmrivers wrote: post #30

Cool idea, some feedback: 1. Consider using conformal prediction to calibrate the cutoff. Conformal prediction provides a distribution-free guarantee under exchangeability. This would let you turn your raw probe score into a threshold with a guaranteed bound on the rate of wrongly-kept on-device answers. Source: https://en.wikipedia.org/wiki/Conformal_prediction 2. The best indicators of confidence in ML come from mu…

1. Thanks for this!

2. We combine both on Cactus; rolling entropy method helps with per-token signal for early exit, while the probe runs on complete output.

3. Users hate this in production, drains battery on edge devices, but ultimately primarily works ok-ish on 7B+ models, we will share the results too.

Post reply on HN