Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ArxivGPT: Chrome extension that summarizes arxived research papers using ChatGPT

github.com

Post reply on HN
Print view View original

51–60 of 105 posts

Re: ArxivGPT: Chrome extension that summarizes arxived research papers using ChatGPT

#51

Post by williamcotton »

herbst wrote: post #16

Ive used ChatGPT to make short factual videos for YouTube and honestly it's a bit worrying with supposed 'facts' I would not suggest anyone to use ChatGPT outputs for actual knowledge at this point.

ChatGPT works best with analytic prompts that respond with translations and not synthesis.

Meaning, if all the facts exist in the prompt then the likelihood of synthesizing fiction is diminished.

There are a number of ways to use the principle of analytic augmentation to add most or if not all of the facts required for a truthful response, ranging from simple “prompt engineering” to evaluating code to document embedding in latent space.

For example, if you use prompt engineering to k-shot a task to turn math word problems into executable JavaScript, meaning LLMs are only translators and the computations are done by a software interpreter, then the results are much more likely to be truthful.

Sampling from a number of variations on a prompt can lead to a more accurate outcome if say 1:10 times the translation attempt has a different answer.

Re: ArxivGPT: Chrome extension that summarizes arxived research papers using ChatGPT

#52

Post by mepian »

magicalhippo wrote: post #45
herbst wrote: post #16

Ive used ChatGPT to make short factual videos for YouTube and honestly it's a bit worrying with supposed 'facts' I would not suggest anyone to use ChatGPT outputs for actual knowledge at this point.

I've not used ChatGPT yet (almost feel like a luddite), what happens when you tell it that it's wrong? Will it get it right after a retry or two?

Sometimes by accident it will, but most of the time it keeps making things up with no regard to factual and logical correctness. It has no notion of truth.

Re: ArxivGPT: Chrome extension that summarizes arxived research papers using ChatGPT

#53

Post by machiaweliczny »

tmalsburg2 wrote: post #37
asabla wrote: post #8

Earlier quoted context omitted.

I like the way you flipped it around. I'm a big believer of using all these new AI services/programs as tools to enhance my workflows, not replacing them.

"The best model was truthful on 58% of questions, while human performance was 94%." https://arxiv.org/abs/2109.07958

If you can measure it you can improve it

Re: ArxivGPT: Chrome extension that summarizes arxived research papers using ChatGPT

#55

Post by ModernMech »

ramraj07 wrote: post #38
goldemerald wrote: post #5

The example here is a bit worrying for the peer review process. I am not looking forward to my "peers" reviewing my paper by putting it through LLMs and blindly copy pasting the output. I can already imagine emailing the Area Chair and saying "While reviewer 2 is detailed, the questions show a severe lack of basic understanding. We believe the contents are AI generated." Then again, perhaps LLMs could simply be incor…

If you generate a question using an LLM, what’s to stop me from answering it using an LLM? And who verifies if the answer is correct? An LLM?

In the future, no one knows how to do anything anymore, and it’s LLMs all the way down.

Re: ArxivGPT: Chrome extension that summarizes arxived research papers using ChatGPT

#56

Post by Maken »

ramraj07 wrote: post #38
goldemerald wrote: post #5

The example here is a bit worrying for the peer review process. I am not looking forward to my "peers" reviewing my paper by putting it through LLMs and blindly copy pasting the output. I can already imagine emailing the Area Chair and saying "While reviewer 2 is detailed, the questions show a severe lack of basic understanding. We believe the contents are AI generated." Then again, perhaps LLMs could simply be incor…

If you generate a question using an LLM, what’s to stop me from answering it using an LLM? And who verifies if the answer is correct? An LLM?

Finally, full automation of the review process is here.

Re: ArxivGPT: Chrome extension that summarizes arxived research papers using ChatGPT

#57

Post by irthomasthomas »

Caution. Language models do not know what is salient to a human. They also have a strong bias toward information that they have seen frequently. Research will contain a larger amount of new information and it's that new information which is most valuable to us, but least relevant according to the models.

Re: ArxivGPT: Chrome extension that summarizes arxived research papers using ChatGPT

#58

Post by Lockal »

machiaweliczny wrote: post #53
tmalsburg2 wrote: post #37

Earlier quoted context omitted.

"The best model was truthful on 58% of questions, while human performance was 94%." https://arxiv.org/abs/2109.07958

If you can measure it you can improve it

When a measure becomes a target, it ceases to be a good measure https://en.wikipedia.org/wiki/Goodhart%27s_law

Re: ArxivGPT: Chrome extension that summarizes arxived research papers using ChatGPT

#59

Post by tmalsburg2 »

sigmoid10 wrote: post #42
tmalsburg2 wrote: post #37

Earlier quoted context omitted.

"The best model was truthful on 58% of questions, while human performance was 94%." https://arxiv.org/abs/2109.07958

A lot of people cite these numbers as cynics or to dissuade others' optimism, but if you actually work in the field and know what it was like just 5 years ago, these numbers should be extremely worrying to everyone who's afraid of automation. Even that linked paper from 2021 is already outdated. We don't need another revolution, we just need maybe a dozen small to medium insights. The steps from GPT3 to 3.5 alone wer…

The training data contains tons of false information and the training objective is simply to reproduce that information. It's not at all surprising that these models fail to distinguish truth from falsehood, and no incremental change will change that. The problem is paradigmatic. And calling people cynics for pointing out the obvious and serious shortcomings of these models is poor form IMO.

Re: ArxivGPT: Chrome extension that summarizes arxived research papers using ChatGPT

#60

Post by MagicMoonlight »

sigmoid10 wrote: post #42
tmalsburg2 wrote: post #37

Earlier quoted context omitted.

"The best model was truthful on 58% of questions, while human performance was 94%." https://arxiv.org/abs/2109.07958

A lot of people cite these numbers as cynics or to dissuade others' optimism, but if you actually work in the field and know what it was like just 5 years ago, these numbers should be extremely worrying to everyone who's afraid of automation. Even that linked paper from 2021 is already outdated. We don't need another revolution, we just need maybe a dozen small to medium insights. The steps from GPT3 to 3.5 alone wer…

These models inherently cannot be truthful because there is no intelligence behind them that can have any sort of intent at all.

It’s literally monkeys with typewriters pressing keys randomly.

Until we get new models which have true understanding, they will never be truly useful.

Post reply on HN