Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic Claude 3.5 can create icalendar files, so I did this

gregsramblings.com

Post reply on HN
Print view View original

1–10 of 174 posts

Re: Anthropic Claude 3.5 can create icalendar files, so I did this

#3

Post by wenc »

You just have to double check the results whenever you tell Claude to extract lists and data.

99.9% of it will be correct, but sometimes 1 or 2 records are off. This kind of error is especially hard to notice because you're so impressed that Claude managed to do the extract task at all -- plus the results look wholly plausible upon eyeballing -- that you wouldn't expect anything to be wrong at all.

But LLMs can get things ever slightly wrong when it comes to long lists/tables. I've been bitten by this before.

Trust but verify.

(edit: if the answer is "machine verifiable", one approach to ask an LLM to write a Python validator which it can execute internally. ChatGPT can execute code. I believe Sonnet 3.5 can too, but I haven't tried.)

Re: Anthropic Claude 3.5 can create icalendar files, so I did this

#4

Post by pplante »

I just did this with my kids various school calendars all of which are locked up in gross PDF files. Claude only made one tiny mistake which it was able to correct when I asked it why it made the mistake. It actually pointed out the confusing formatting used by the author.

I tried to do the same with ChatGPT a few weeks ago and was unable to get very far.

Re: Anthropic Claude 3.5 can create icalendar files, so I did this

#6

Post by thelittleone »

wenc wrote: post #3

You just have to double check the results whenever you tell Claude to extract lists and data. 99.9% of it will be correct, but sometimes 1 or 2 records are off. This kind of error is especially hard to notice because you're so impressed that Claude managed to do the extract task at all -- plus the results look wholly plausible upon eyeballing -- that you wouldn't expect anything to be wrong at all. But LLMs can get t…

Can solve that to have a similar or lower error probability as a human by running the results through a verification agent.

Re: Anthropic Claude 3.5 can create icalendar files, so I did this

#7

Post by buro9 »

I'm finding Claude to hallucinate less than ChatGPT, and to be far more accurate at coding than CoPilot. Pleasantly surprised on both counts.

Example hallucinations from ChatGPT include researching the dates of historical events for the company I work at, trivially verifiable by me but I was being lazy... ChatGPT told me about blog posts that never existed and I could prove never existed, Claude was spot on with dates and source links (but only appeared to have data through to the end of last year).

On coding, CoPilot came up with reasonable suggestions but Claude was able to take a file as an input and match the style of the code within the repo.

Claude, for me, is starting to hint at what a highly productive assistant can achieve.

Re: Anthropic Claude 3.5 can create icalendar files, so I did this

#9

Post by romeros »

buro9 wrote: post #7

I'm finding Claude to hallucinate less than ChatGPT, and to be far more accurate at coding than CoPilot. Pleasantly surprised on both counts. Example hallucinations from ChatGPT include researching the dates of historical events for the company I work at, trivially verifiable by me but I was being lazy... ChatGPT told me about blog posts that never existed and I could prove never existed, Claude was spot on with date…

For some reason I felt Claude was Yahoo to ChatGPT's Google in the initial days. It just felt like ChatGPT had an insurmountable moat.

I played with Claude and it is just insanely superior compared to ChatGpt as of now. Someone on HN commented the same thing a few months back but I did not take it very seriously. But, Claude is just superior to Chatgpt.

Post reply on HN