Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Consensus: Use AI to find insights in research papers

consensus.app

Post reply on HN
Print view View original

31–40 of 54 posts

Re: Consensus: Use AI to find insights in research papers

#31

Post by bee_rider »

xigency wrote: post #12
cj wrote: post #9

Earlier quoted context omitted.

Every time I hear or read "prompt engineering", I can't help but cringe a bit. I'm not sure why, but it's the same reaction I would have if I heard someone say "Google search query engineering". Comparing to google search, there definitely is skill involved in knowing how to google things well. We're all accustomed to googling things many times per day so I think a lot of people forget that being able to google thing…

Effective prompting has more to do with theater than engineering.

Yes, and-gineering.

Re: Consensus: Use AI to find insights in research papers

#33

Post by andsoitis »

gobengo wrote: post #32
andsoitis wrote: post #29

Earlier quoted context omitted.

I think my point stands that Google's answer is more accurate.

... depends on who u ask

the answer gives me both the current consensus AND the latest possibly new consensus.

the Consensus site gives me... not even an accurate current consensus.

Re: Consensus: Use AI to find insights in research papers

#34

Post by whatshisface »

andsoitis wrote: post #29
whatshisface wrote: post #26

Earlier quoted context omitted.

For reference the actual consensus is 13.787±0.020Gy.

I think my point stands that Google's answer is more accurate.

A reply on HN is not always the counterpoint. :-)

Re: Consensus: Use AI to find insights in research papers

#35

Post by llmllmllm »

SamBam wrote: post #30
bee_rider wrote: post #27

Earlier quoted context omitted.

FWIW is is pretty common to read the abstract, skim the intro, skip to the conclusion/results, and then only then read the rest of the paper (if you even need to at that point). https://www.science.org/content/article/how-seriously-read-s... So, this is pretty quick, you might not need a tool for it.

Actually, what I was asking about was how to handle my storage and later retrieval of hundreds of papers. I don't mean just the database tool -- I currently use Zotero and make sure articles are tagged -- but I'm wondering if I can build a system for my articles where I can ask ChatGPT "are there any studies that show X causes Y" and have it use my articles directly, and not give me bullshit citations like it does 90…

As part of my testing of https://flowch.ai (which we are developing), I've been uploading multiple PDFs (including scientific papers). I then ask a question and ask it to cite its sources as part of the prompt and it'll give the PDFs it's using for the data / quotes it's using. Works well!

It's currently free to use :)

Re: Consensus: Use AI to find insights in research papers

#36

Post by chaxor »

SamBam wrote: post #30
bee_rider wrote: post #27

Earlier quoted context omitted.

FWIW is is pretty common to read the abstract, skim the intro, skip to the conclusion/results, and then only then read the rest of the paper (if you even need to at that point). https://www.science.org/content/article/how-seriously-read-s... So, this is pretty quick, you might not need a tool for it.

Actually, what I was asking about was how to handle my storage and later retrieval of hundreds of papers. I don't mean just the database tool -- I currently use Zotero and make sure articles are tagged -- but I'm wondering if I can build a system for my articles where I can ask ChatGPT "are there any studies that show X causes Y" and have it use my articles directly, and not give me bullshit citations like it does 90…

First you just extract all of the text from the PDFs. It's helpful to use arxiv instead, since it gives you latex. Then you can parse to separate content from style. Then store that in a a DuckDB database, with zstd compression since it's great on text. Then, just use some encoder model (decoder models are all the rage but they're not necessary here) to process all of these texts into Qdrant.

Then use your Qdrant database happily ever after, with whatever you like, such as Vicuna or Guanaco 30b GPTQ. Sprinkle your langcgain or whatever you like. Galpaca 30b may be appropriate for science text.

Re: Consensus: Use AI to find insights in research papers

#37

Post by samuell »

dandiep wrote: post #28

I think https://elicit.org/ is also worth a mention in this context.

Elicit is very useful! It can't sometimes answer some very specific and complex questions, but it also doesn't tend to hallucinate anywhere as much as chatgpt. Rather it will be quite upfront about when it can't really find an answer to something.

Re: Consensus: Use AI to find insights in research papers

#38

Post by dragonwriter »

SamBam wrote: post #14
didibus wrote: post #2

I've been doing this with chatGPT directly. But it's cutoff data is 2021. It's pretty good though at telling you even the research name so you can validate.

As someone who is starting a master's in science and will be looking at lots of research papers, I've been wondering what the best use of this could be. If I have my own PDFs, I guess I could get ChatGPT to create summaries in some structured way, perhaps in a single file with citation:summary, and then send up that file with every question I ask?

> If I have my own PDFs, I guess I could get ChatGPT to create summaries in some structured way, perhaps in a single file with citation:summary, and then send up that file with every question I ask?

Extract the text, put it into a database (a vectordb would be the hot thing for with an LLM, is probably ideal, but ChatGPT does a pretty good job using Wikipedia as a “database” with a dirt simple ReAct pattern implementation, so you probably don’t need to be that fancy to get value out of it) and then use tooling to let ChatGPT use that database for questions.

Re: Consensus: Use AI to find insights in research papers

#39

Post by userbinator »

pmarreck wrote: post #16

I asked it about UBI and I guess there isn't enough data yet, because it called it "fiscally unbearable and morally unacceptable" (which seems to be what only 1 uncited source said)! At least it admitted consensus was low... https://consensus.app/results/?q=Does%20universal%20basic%20... Summary Top 10 papers analyzed Some studies suggest that universal basic income (UBI) can generate support for structural reforms a…

It's not surprising that highly political topics are likely to generate such results.

Re: Consensus: Use AI to find insights in research papers

#40

Post by chs20 »

samuell wrote: post #37
dandiep wrote: post #28

I think https://elicit.org/ is also worth a mention in this context.

Elicit is very useful! It can't sometimes answer some very specific and complex questions, but it also doesn't tend to hallucinate anywhere as much as chatgpt. Rather it will be quite upfront about when it can't really find an answer to something.

From which database are the papers that it searches?
Post reply on HN