Anthropic Claude 3.5 can create icalendar files, so I did this

Post by bhl » Sun, Aug 25, 2024, 4:34 PM UTC

wenc wrote: post #3

You just have to double check the results whenever you tell Claude to extract lists and data. 99.9% of it will be correct, but sometimes 1 or 2 records are off. This kind of error is especially hard to notice because you're so impressed that Claude managed to do the extract task at all -- plus the results look wholly plausible upon eyeballing -- that you wouldn't expect anything to be wrong at all. But LLMs can get t…

Post by ada1981 » Sun, Aug 25, 2024, 4:42 PM UTC

oneshtein wrote: post #132

wanderingmind wrote: post #90

Earlier quoted context omitted.

Kind of a noob question, is it possible to design a GAN type network with LLM, where one (or many) LLMs generate outputs, while a few other LLMs validate or discriminate them and thus improving generator LLMs accuracy.

Yes, you can use AI to spot errors in AI output. Done before with good results, but it requires to run 2 different, but equally good, AI models in parallel, which is way more expensive than 1 model.

Post by kovezd » Sun, Aug 25, 2024, 4:47 PM UTC

disillusioned wrote: post #25

wenc wrote: post #3

You just have to double check the results whenever you tell Claude to extract lists and data. 99.9% of it will be correct, but sometimes 1 or 2 records are off. This kind of error is especially hard to notice because you're so impressed that Claude managed to do the extract task at all -- plus the results look wholly plausible upon eyeballing -- that you wouldn't expect anything to be wrong at all. But LLMs can get t…

The "off by one" predilection of LLMs is going to lead to this massive erosion of trust in whatever "Truth" is supposed to be, and it's terrifying and going to make for a bumpy couple of years. (Or the complete collapse of objective knowledge, on a long enough time horizon.) It's one thing to ask an LLM when George Washington was born, and have it return "May 20, 2020." It's another thing to ask it, and have it matte…

Post by croes » Sun, Aug 25, 2024, 4:56 PM UTC

onlyrealcuzzo wrote: post #91

wenc wrote: post #3

You just have to double check the results whenever you tell Claude to extract lists and data. 99.9% of it will be correct, but sometimes 1 or 2 records are off. This kind of error is especially hard to notice because you're so impressed that Claude managed to do the extract task at all -- plus the results look wholly plausible upon eyeballing -- that you wouldn't expect anything to be wrong at all. But LLMs can get t…

> 99.9% of it will be correct, but sometimes 1 or 2 records are off. This kind of error is especially hard to notice because you're so impressed that Claude managed to do the extract task at all -- plus the results look wholly plausible upon eyeballing -- that you wouldn't expect anything to be wrong at all. That's far better than I would do on my own. I doubt I'd even be 99% accurate. If it's really 99.9% accurate f…

Post by rpdillon » Sun, Aug 25, 2024, 5:27 PM UTC

stavros wrote: post #112

rpdillon wrote: post #103

Earlier quoted context omitted.

> to me trust means exactly "I don't need to verify". If you use the slightly weaker definition that trust means you have confidence in someone, then the adage makes sense.

The issue here is that the only value of the adage is in the sleight of hand it lets you perform. If someone asks "don't you trust me?" (ie "do you have to verify what I do/say?"), you can say "trust, but verify!", and kind of make it sound like you do trust them, but also you don't really. The adage doesn't work under any definition of trust other than the one it's conflicting with itself about.