Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Consensus: Use AI to find insights in research papers

consensus.app

Post reply on HN
Print view View original

21–30 of 54 posts

Re: Consensus: Use AI to find insights in research papers

#21

Post by orzig »

pmarreck wrote: post #16

I asked it about UBI and I guess there isn't enough data yet, because it called it "fiscally unbearable and morally unacceptable" (which seems to be what only 1 uncited source said)! At least it admitted consensus was low... https://consensus.app/results/?q=Does%20universal%20basic%20... Summary Top 10 papers analyzed Some studies suggest that universal basic income (UBI) can generate support for structural reforms a…

I have used it for several weeks, I would recommend a more specific question. On the UBI front, I had somewhat more success (though no consensus) with a concrete question on employment : https://consensus.app/results/?q=Does%20UBI%20decrease%20emp...

Re: Consensus: Use AI to find insights in research papers

#22

Post by madmaxmcfly »

malux85 wrote: post #6
acqbu wrote: post #5

Is this another SaaS making API calls to OpenAI or is it more than that? If so, how?

Embed articles and throw the results in a vector database. Throw up a search result that just uses cosine similarity on the vector search with questionable metrics and no explanations on how things are calculated. Charge yearly because you know people will churn after a month or two. Profit - Every "AI startup" in the last 2 months

Llama Index must be getting some pretty good use

Re: Consensus: Use AI to find insights in research papers

#23

Post by reissbaker »

cj wrote: post #9
fzliu wrote: post #8

Earlier quoted context omitted.

I'll play DA here - there's quite a bit of engineering surrounding these apps that can appear hidden to folks from the outside looking in. Various levels of prompt engineering and in-context learning might be necessary to get optimal results, and this could mean significantly more complexity at the application level.

Every time I hear or read "prompt engineering", I can't help but cringe a bit. I'm not sure why, but it's the same reaction I would have if I heard someone say "Google search query engineering". Comparing to google search, there definitely is skill involved in knowing how to google things well. We're all accustomed to googling things many times per day so I think a lot of people forget that being able to google thing…

IMO you're right that "prompt engineering" is a cringe-y term because it implies what you're doing is mostly writing prompts. That being said, I don't think that's what it actually entails, any more than "backend engineering" is mostly writing SQL queries. Prompt engineering is building the systems around the prompts e.g. writing the LangChain or whatever code, parsing stuff and interacting with structured DBs, message queues, etc (and occasionally writing prompts too, but that's a relatively smaller part). It requires some domain-specific knowledge e.g. chain of thought and retrieval augmented generation techniques, some basic linear algebra, keeping up to date with new models and new ways of running them (ggml? gptq? openai functions vs logit masking llama 2?), but it's more or less backend engineering with a twist.

I've seen some of the more serious people switch to the term "Applied AI" which I think encompasses the role a lot better. Also I've seen a decent number of grifter types saying they're "prompt engineers" when what they mean is they're writing prompts into ChatGPT's UI, which I think is part of what drives the cringing feeling when you hear the phrase "prompt engineer" and probably drives some of the movement away from the term for engineers.

Re: Consensus: Use AI to find insights in research papers

#24

Post by carlossouza »

acqbu wrote: post #5

Is this another SaaS making API calls to OpenAI or is it more than that? If so, how?

Yes, probably. But isn’t GitHub Copilot, Jasper, Perplexity and other great apps as well?

Judging from the engagement in their slack channel, it looks like they are onto something.

Re: Consensus: Use AI to find insights in research papers

#25

Post by andsoitis »

"how old is the universe?"

The age of the observable universe is 11 billion years, which is in close agreement with the inflationary model prediction that the age of the universe is two-thirds of the Hubble time.

https://consensus.app/details/billion-years-close-agreement-...

In contrast, when I ask Google "how old is the universe" I get as top result:

26.7 billion years old Current estimates place the Big Bang 13.8 billion years ago. University of Ottawa adjunct professor Rajendra Gupta has calculated that it is, in fact, 26.7 billion years old – nearly twice as old as the current accepted model. extracted from https://cosmosmagazine.com/space/astrophysics/universe-27-bi....

Re: Consensus: Use AI to find insights in research papers

#26

Post by whatshisface »

andsoitis wrote: post #25

"how old is the universe?" The age of the observable universe is 11 billion years, which is in close agreement with the inflationary model prediction that the age of the universe is two-thirds of the Hubble time. https://consensus.app/details/billion-years-close-agreement-... In contrast, when I ask Google "how old is the universe" I get as top result: 26.7 billion years old Current estimates place the Big Bang 13.8…

For reference the actual consensus is 13.787±0.020Gy.

Re: Consensus: Use AI to find insights in research papers

#27

Post by bee_rider »

SamBam wrote: post #14
didibus wrote: post #2

I've been doing this with chatGPT directly. But it's cutoff data is 2021. It's pretty good though at telling you even the research name so you can validate.

As someone who is starting a master's in science and will be looking at lots of research papers, I've been wondering what the best use of this could be. If I have my own PDFs, I guess I could get ChatGPT to create summaries in some structured way, perhaps in a single file with citation:summary, and then send up that file with every question I ask?

FWIW is is pretty common to read the abstract, skim the intro, skip to the conclusion/results, and then only then read the rest of the paper (if you even need to at that point).

https://www.science.org/content/article/how-seriously-read-s...

So, this is pretty quick, you might not need a tool for it.

Re: Consensus: Use AI to find insights in research papers

#29

Post by andsoitis »

whatshisface wrote: post #26
andsoitis wrote: post #25

"how old is the universe?" The age of the observable universe is 11 billion years, which is in close agreement with the inflationary model prediction that the age of the universe is two-thirds of the Hubble time. https://consensus.app/details/billion-years-close-agreement-... In contrast, when I ask Google "how old is the universe" I get as top result: 26.7 billion years old Current estimates place the Big Bang 13.8…

For reference the actual consensus is 13.787±0.020Gy.

I think my point stands that Google's answer is more accurate.

Re: Consensus: Use AI to find insights in research papers

#30

Post by SamBam »

bee_rider wrote: post #27
SamBam wrote: post #14

Earlier quoted context omitted.

As someone who is starting a master's in science and will be looking at lots of research papers, I've been wondering what the best use of this could be. If I have my own PDFs, I guess I could get ChatGPT to create summaries in some structured way, perhaps in a single file with citation:summary, and then send up that file with every question I ask?

FWIW is is pretty common to read the abstract, skim the intro, skip to the conclusion/results, and then only then read the rest of the paper (if you even need to at that point). https://www.science.org/content/article/how-seriously-read-s... So, this is pretty quick, you might not need a tool for it.

Actually, what I was asking about was how to handle my storage and later retrieval of hundreds of papers. I don't mean just the database tool -- I currently use Zotero and make sure articles are tagged -- but I'm wondering if I can build a system for my articles where I can ask ChatGPT "are there any studies that show X causes Y" and have it use my articles directly, and not give me bullshit citations like it does 90% of the time now.

(To be clear, I'm pleasantly surprised when it does give me some real, useful citations of articles I didn't know about, but I'd like to get it's accuracy way up, at least for articles I have in my possession that I could feed it for context.)

Post reply on HN