31–40 of 53 posts
In fact I'd be surprised if a bunch of scientists weren't already doing that. But I'm not sure of the utility of that.
In fact, using something like our new chat-ai, we wouldn't even need to understand the connection.
As someone who probably puts way too much time into literature reviews, my big hope is that literature reviews at the beginning if a research project will be revolutionized. Things are re-discovered across adjacent fields of study all the time. There are also a bunch of times I've come across a paper when I was a year into a project, and wished that I'd had it at the beginning of the project.
I think it might be exactly what you are looking for.
Here's the heart of how they say it can be done > Two areas in particular look promising. The first is “literature-based discovery” (LBD), which involves analysing existing scientific literature, using ChatGPT-style language analysis, to look for new hypotheses, connections or ideas that humans may have missed. LBD is showing promise in identifying new experiments to try—and even suggesting potential research collabo…
Here's the heart of how they say it can be done > Two areas in particular look promising. The first is “literature-based discovery” (LBD), which involves analysing existing scientific literature, using ChatGPT-style language analysis, to look for new hypotheses, connections or ideas that humans may have missed. LBD is showing promise in identifying new experiments to try—and even suggesting potential research collabo…
I think the problems with replication are a separate axis to the problem that there is too much being published for anyone to actually read.
AI in general — never mind LLMs, even the much ones running search engines — help with the content overload.
> This sounds basically of just asking a GPT to suggest ideas for experiments?
I think the car analogy here is that if what you've suggesting was google maps, what the article is suggesting is Level 5 autonomy with no steering wheel and an opaque wall instead of a windscreen.
I have absolutely no idea how hard such a level of automation might be to actually implement, not least because of Moravec's paradox: https://en.wikipedia.org/wiki/Moravec's_paradox
For the literature review piece the key problem is that LLMs are exquisitely bad at working with even the simplest kind of scientific evidence: citations [1, 2]. They will get better, but it is not clear that LLMs can deal effectively with the very sparse kind of evidence that appears in the literature. Also, generating hypotheses isn't exactly the rate limiting step, the bigger issue tends to be when you get people with pet projects/hypotheses in positions of power that dictate funding priorities (e.g. the decades long Alzheimer's Aβ disaster).
For automation and instrumentation of labs the vision is on point and there is interest, and active work, if not large amounts of funding, to bring that vision to reality [3, 4, 5]. However, we simply don't have the tooling needed to be able to express the full complexity of experimental protocols in a way that can be verified. Sure you can write a python script to control a robot, but it is exceptionally difficult to extract the scientific meaning from that.
My PhD work was to develop a formal language for scientific protocols, and I'll be continuing to develop it, but there is still a long way to go.
1. https://doi.org/10.7759/cureus.39238 2. https://doi.org/10.1016/j.mcpdig.2023.05.004 3. https://www.youtube.com/watch?v=_gXiVOmaVSo&t=865s 4. https://doi.org/10.1109/JIOT.2020.2995323 5. https://ccc.ucsf.edu/sites/ccc.ucsf.edu/files/Marshall_W_CCC...
We are still quite far from being able to implement these kinds of things at scale. For the literature review piece the key problem is that LLMs are exquisitely bad at working with even the simplest kind of scientific evidence: citations [1, 2]. They will get better, but it is not clear that LLMs can deal effectively with the very sparse kind of evidence that appears in the literature. Also, generating hypotheses isn…
Our key insight is that the process of citation needs to be handled outside the LLM. They're good for text processing and summarization but as you said, the LLM itself is poor at citation.
Here's the heart of how they say it can be done > Two areas in particular look promising. The first is “literature-based discovery” (LBD), which involves analysing existing scientific literature, using ChatGPT-style language analysis, to look for new hypotheses, connections or ideas that humans may have missed. LBD is showing promise in identifying new experiments to try—and even suggesting potential research collabo…
Lots of time can be saved by automating those steps (and many researchers don't enjoy it so their job satisfaction could be increased). Also the resulting output could be improved if the researcher had a well structured summary to use as the foundation of their outline.
Improve search with semantic search (search by concept not keyword) Improve refinement by preprocessing and summarizing Don't print, display clean and concise data. Summarize, cite and display.
This stops short of literature based discovery, you have to bring your own research question.
We've also had some luck finding a gap in existing research. We did a POC where we scraped pubmed and graphed study results by concept. We then used the graphed concepts to explore the conceptual space.
It seems that vitamin D protects against cancer and heart disease. It seems that vitamin D supplementation protects against cancer but not heart disease. Is this because of some previously unknown effect of sun exposure (the primary natural source of vitamin D) or is it just that people with adequate vitamin D go outside a lot more and therefore also get more exercise? Don't know, would love to read the paper if someone studies it ;- )
We are still quite far from being able to implement these kinds of things at scale. For the literature review piece the key problem is that LLMs are exquisitely bad at working with even the simplest kind of scientific evidence: citations [1, 2]. They will get better, but it is not clear that LLMs can deal effectively with the very sparse kind of evidence that appears in the literature. Also, generating hypotheses isn…
It's actually possible to use LLMs to assist literature reviews. We built a product that works smashingly. The key is to keyword extract, use a vector database and do search based generation. Our key insight is that the process of citation needs to be handled outside the LLM. They're good for text processing and summarization but as you said, the LLM itself is poor at citation. https://studyrecon.ai
I can imagine taking the citation tree and using the LLM to compact the hypotheses, results, etc. for each node in the tree and sticking that in the vector database could get you pretty far.
Here's the heart of how they say it can be done > Two areas in particular look promising. The first is “literature-based discovery” (LBD), which involves analysing existing scientific literature, using ChatGPT-style language analysis, to look for new hypotheses, connections or ideas that humans may have missed. LBD is showing promise in identifying new experiments to try—and even suggesting potential research collabo…
Yeah, and then the funding agencies summarize the applications by running them through an LLM.
But will we maintain control of the above-human-ability, autonomous AI systems these companies are racing to build? This is the AI control problem.
If not, then "AI can automate science" isn't much of a counterpoint or reason to be optimistic -- science may be automated, but not under any human's control and not for any human's benefit. In fact, if we're in this situation, the ability of AI systems to automate science is worse news than otherwise, in the same way that the invention of science by humans was bad (or at best, very mixed) news for the animals of Earth.