Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination Risk Calculator

github.com

Post reply on HN
Print view View original

31–40 of 44 posts

Re: Hallucination Risk Calculator

#31

Post by dingnuts »

photonthug wrote: post #22
ep103 wrote: post #19

Earlier quoted context omitted.

Yesterday I used ChatGPT to transform a csv file. Move around a couple of columns, add a few new ones. Very large file. It got them all right. Except when I really looked through the data, for 3 of the excel cells, it clearly just made up new numbers. I found the first one by accident, the remaining two took longer than it would have taken to modify the file from scratch myself. Watching my coworkers blindly trust ou…

After we fix the all the simple specious reasoning of stuff like Alexander-the-great and agree to out-source certain problems to appropriate tools, the high-dimensional analogs of stuff like Datasaurus[0] and Simpson's paradox[1] etc are still going to be a thing. But we'll be so disconnected from the representation of the problems that we're trying to solve that we won't even be aware of the possibility of any dange…

> it just feels like a scam aimed at burning through more tokens.

IT IS A SCAM TO BURN MORE TOKENS. You will know when it is no longer a scam when you either:

1) pay a flat price with NO USAGE LIMITS

or

2) pay per token with the ability to mark a response as bullshit & get a refund for those wasted tokens.

Until then: the incentives are the same as a casino's which means IT IS A SCAM.

Re: Hallucination Risk Calculator

#32

Post by cowboylowrez »

lock1 wrote: post #25
blamestross wrote: post #20

This seems less accurate than `return 1.0` Using the unboundedly unreliable systems to evaluate reliability is just a bad premise.

Can't wait for (((LLM) Hallucination Risk Calculator) Risk Calculator) Risk Calculator to propagate & magnify the error even further! /j

have multiple llms and a voting quorum. sort of how we elect politicians. it'll work just as well I guarantee it!

Re: Hallucination Risk Calculator

#33

Post by wongarsu »

cowboylowrez wrote: post #32
lock1 wrote: post #25

Earlier quoted context omitted.

Can't wait for (((LLM) Hallucination Risk Calculator) Risk Calculator) Risk Calculator to propagate & magnify the error even further! /j

have multiple llms and a voting quorum. sort of how we elect politicians. it'll work just as well I guarantee it!

Back in the GPT2 times I did use that technique. Also just running the model multiple times with slightly different prompts and choosing the most common response. It doesn't cure all problems but it does lead to better results. It isn't very good for your wallet though

Re: Hallucination Risk Calculator

#34

Post by jmogly »

photonthug wrote: post #22
ep103 wrote: post #19

Earlier quoted context omitted.

Yesterday I used ChatGPT to transform a csv file. Move around a couple of columns, add a few new ones. Very large file. It got them all right. Except when I really looked through the data, for 3 of the excel cells, it clearly just made up new numbers. I found the first one by accident, the remaining two took longer than it would have taken to modify the file from scratch myself. Watching my coworkers blindly trust ou…

After we fix the all the simple specious reasoning of stuff like Alexander-the-great and agree to out-source certain problems to appropriate tools, the high-dimensional analogs of stuff like Datasaurus[0] and Simpson's paradox[1] etc are still going to be a thing. But we'll be so disconnected from the representation of the problems that we're trying to solve that we won't even be aware of the possibility of any dange…

To me it’s a problem of if a piece of information is not well represented in the training data the llm will always tend towards bad token predictions for related to said information. I think the next big thing in LLM’s could be figuring out how to tell if a token was just a “fill in” or “guess” vs a well predicted token. That way you can have some sort of governor that can kill a response if it is getting too guessy, or atleast provide some other indication that the provided tokens are likely hallucinated.

Maybe there is some way to do it based on the geometry of how the neural net activated for a token, or some other more statistics based approach, idk I’m not an expert.

Re: Hallucination Risk Calculator

#35

Post by befictious »

photonthug wrote: post #22
ep103 wrote: post #19

Earlier quoted context omitted.

Yesterday I used ChatGPT to transform a csv file. Move around a couple of columns, add a few new ones. Very large file. It got them all right. Except when I really looked through the data, for 3 of the excel cells, it clearly just made up new numbers. I found the first one by accident, the remaining two took longer than it would have taken to modify the file from scratch myself. Watching my coworkers blindly trust ou…

After we fix the all the simple specious reasoning of stuff like Alexander-the-great and agree to out-source certain problems to appropriate tools, the high-dimensional analogs of stuff like Datasaurus[0] and Simpson's paradox[1] etc are still going to be a thing. But we'll be so disconnected from the representation of the problems that we're trying to solve that we won't even be aware of the possibility of any dange…

>it just feels like a scam aimed at burning through more tokens.

I have a growing tin foil hat theory that the business model of LLM's is the same as 1-900-psychic numbers of old.

For just 25¢ 1-900-psychic will solve all your problems in just 5 minutes! Still need help?! No problem! We'll work with you until you get your answers for only 10¢ a minute until your happy!

eerily similar

Re: Hallucination Risk Calculator

#37

Post by phs318u »

dingnuts wrote: post #31
photonthug wrote: post #22

Earlier quoted context omitted.

After we fix the all the simple specious reasoning of stuff like Alexander-the-great and agree to out-source certain problems to appropriate tools, the high-dimensional analogs of stuff like Datasaurus[0] and Simpson's paradox[1] etc are still going to be a thing. But we'll be so disconnected from the representation of the problems that we're trying to solve that we won't even be aware of the possibility of any dange…

> it just feels like a scam aimed at burning through more tokens. IT IS A SCAM TO BURN MORE TOKENS. You will know when it is no longer a scam when you either: 1) pay a flat price with NO USAGE LIMITS or 2) pay per token with the ability to mark a response as bullshit & get a refund for those wasted tokens. Until then: the incentives are the same as a casino's which means IT IS A SCAM.

Ding ding ding! We have a winner!

Re: Hallucination Risk Calculator

#38

Post by toss1 »

weinzierl wrote: post #27
ep103 wrote: post #19

Earlier quoted context omitted.

Yesterday I used ChatGPT to transform a csv file. Move around a couple of columns, add a few new ones. Very large file. It got them all right. Except when I really looked through the data, for 3 of the excel cells, it clearly just made up new numbers. I found the first one by accident, the remaining two took longer than it would have taken to modify the file from scratch myself. Watching my coworkers blindly trust ou…

It sometimes happens with simple things. I once pasted the announcement for an event in Claude to check for spelling and grammar. It had a small suggestion for the last sentence and repeated the whole corrected version for me to copy and paste. Only last sentence slightly modified - or so I thought because it had moved the date of the event in the first sentence by one day. Luckily I caught it before posting, but it…

Yup, I always take editing suggestions and implement them manually, then re-feed the edited version back in for new suggestions if needed. Never let it edit your stuff directly —— the risk of stealth random errors sneaking in is too great.

Just because every competent human we know would edit ONLY the specified parts, or move only the specified columns with a cut/paste operation (or similar deterministically reliable operation), does not mean an LLM will do the same, in fact, it seems to prefer to regenerate everything on the fly. NO, just NO.

Re: Hallucination Risk Calculator

#39

Post by epiccoleman »

ep103 wrote: post #19
photonthug wrote: post #9

From the paper abstract, > (4) we derive the optimal chain-of-thought length as [..math..] with explicit constants I know we probably have to dive into math and abandon metaphor and analogy, but the whole structure of a claim like this just strikes me as bizarre. Chain-of-thought always makes me think of that old joke. Alexander the great was a great general. Great generals are forewarned. Forewarned is forearmed. Fo…

Yesterday I used ChatGPT to transform a csv file. Move around a couple of columns, add a few new ones. Very large file. It got them all right. Except when I really looked through the data, for 3 of the excel cells, it clearly just made up new numbers. I found the first one by accident, the remaining two took longer than it would have taken to modify the file from scratch myself. Watching my coworkers blindly trust ou…

I don't mean to be rude, but this sounds like user error. I don't understand why anyone would use an LLM for this - or at least, why you would let the LLM perform the transformation.

If I was trying to do something like this I would ask the LLM to write a Python script, validate the output by running it against the first handful of rows (like, `head -n 10 thing.csv | python transform-csv.py`).

There are times when statistical / stochastic output is useful. There are other times when you want deterministic output. A transformation on a CSV is the latter.

Re: Hallucination Risk Calculator

#40

Post by photonthug »

jmogly wrote: post #34
photonthug wrote: post #22

Earlier quoted context omitted.

After we fix the all the simple specious reasoning of stuff like Alexander-the-great and agree to out-source certain problems to appropriate tools, the high-dimensional analogs of stuff like Datasaurus[0] and Simpson's paradox[1] etc are still going to be a thing. But we'll be so disconnected from the representation of the problems that we're trying to solve that we won't even be aware of the possibility of any dange…

To me it’s a problem of if a piece of information is not well represented in the training data the llm will always tend towards bad token predictions for related to said information. I think the next big thing in LLM’s could be figuring out how to tell if a token was just a “fill in” or “guess” vs a well predicted token. That way you can have some sort of governor that can kill a response if it is getting too guessy,…

A related topic you might want to look into here is called nucleus sampling. Similar to temperature but also different.. it's been surprising to me that people don't talk about it more often, and that lots of systems won't expose the knobs for it.
Post reply on HN