Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ResearchAgent: Iterative Research Idea Generation Using LLMs

arxiv.org

Post reply on HN
Print view View original

41–50 of 66 posts

Re: ResearchAgent: Iterative Research Idea Generation Using LLMs

#41

Post by rsfern »

golol wrote: post #35
robbomacrae wrote: post #33

Earlier quoted context omitted.

I'm generally a proponent of AI and LLM but to me the decision was the right one. You are tasking people with implementing an idea generated by an algorithmic model with (I'm guessing) zero oversight that might have very little training that teaches it the importance of coming up with ideas worth implementing. Some may be more useful than others so it won't be fair from an accomplishment or motivation point of view.…

As long as all participants are well-informed then there is absolutely no ethical issue...

How do you make sure the participants are well informed? What if an idea suggested by a model turns out to be dangerous to implement, but nobody at the hackathon has quite the relevant experience to notice?

Re: ResearchAgent: Iterative Research Idea Generation Using LLMs

#43

Post by ssn »

robwwilliams wrote: post #16
pedalpete wrote: post #8

I've found where LLMs can be useful in this context is around free-associations. Because they don't really "know" about things, they regularly grasp at straws or misconstrue intended meaning. This, along with the volume of language (let's not call it knowledge) result in the LLMs occasionally bringing in a new element which can be useful.

This approach is already useful in functional genomics. A common type of question requires analysis of hundreds of potentially functional sequence variants. Hybrid LLM+ approaches are beginning to improve efficiency of ranking candidates and even proposing tests and soon I hope—higher order non-linear interactions among DNA variants.

I am interested in this. Can you point to a reference about the application of LLMs to sequence secreening? Thanks.

Re: ResearchAgent: Iterative Research Idea Generation Using LLMs

#44

Post by jimmySixDOF »

robbomacrae wrote: post #33
srcreigh wrote: post #20

Earlier quoted context omitted.

That’s pretty wild. What was the reason behind failing ethics review?

I'm generally a proponent of AI and LLM but to me the decision was the right one. You are tasking people with implementing an idea generated by an algorithmic model with (I'm guessing) zero oversight that might have very little training that teaches it the importance of coming up with ideas worth implementing. Some may be more useful than others so it won't be fair from an accomplishment or motivation point of view.…

Individual ideas can be subject to Ethical Review Board approvals and that should go for a hackathon project same as any study proposed in Academia or drug trial etc -- but to apply some wavey handed lum sum out of bounds lable just based on source seems like arbitrary opinionated overreach.

Re: ResearchAgent: Iterative Research Idea Generation Using LLMs

#45

Post by wizzwizz4 »

SubiculumCode wrote: post #29

In some fields of research, the amount of literature out there is stupendous, and with little hope of a human reading, much less understanding the whole literature. Its becoming a major problem in some fields, and I think, in some ways, approaches that can combine knowledge algorithmically are needed, perhaps llms.

Traditionally, that's what meta-analyses and published reviews of the literature have been for.

Re: ResearchAgent: Iterative Research Idea Generation Using LLMs

#46

Post by CJefferson »

srcreigh wrote: post #20
KhoomeiK wrote: post #18

A group of PhD students at Stanford recently wanted to take AI/ML research ideas generated by LLMs like this and have teams of engineers execute on them at a hackathon. We were getting things prepared at AGI House SF to host the hackathon with them when we learned that the study did not pass ethical review . I think automating science is an important research direction nonetheless.

That’s pretty wild. What was the reason behind failing ethics review?

One obvious problem is, what if the ideas were obviously unethical?

I would personally let this pass ethics if someone read all the generated ideas, and took personal responsibility for them passing the basic ethics rules, or got them through the ethics committee if required, exactly the same as they would their own ideas.

Re: ResearchAgent: Iterative Research Idea Generation Using LLMs

#47

Post by krageon »

not-chatgpt wrote: post #2

Cool idea. Never gonna work. LLMs are still generative models that spits out training data, incapable of highly abstract creative tasks like research. I still remember all the GPT-2 based startup idea generators that spits out pseudo-feasible startups.

When you're this confident and making blanket statements that are this unilateral, that should tell you you need to take a step back and question yourself.

Re: ResearchAgent: Iterative Research Idea Generation Using LLMs

#48

Post by throwup238 »

malux85 wrote: post #31
ec109685 wrote: post #21

Earlier quoted context omitted.

Hallucinations are lies. So not the same thing.

Hallucinations are not always lies, they are more like a transformation in the abstraction space.

That is some weapons grade spin :-)

Re: ResearchAgent: Iterative Research Idea Generation Using LLMs

#49

Post by naasking »

rsfern wrote: post #41
golol wrote: post #35

Earlier quoted context omitted.

As long as all participants are well-informed then there is absolutely no ethical issue...

How do you make sure the participants are well informed? What if an idea suggested by a model turns out to be dangerous to implement, but nobody at the hackathon has quite the relevant experience to notice?

Such as?

Re: ResearchAgent: Iterative Research Idea Generation Using LLMs

#50

Post by falcor84 »

tokai wrote: post #42
UncleOxidant wrote: post #22

The ideas aren't the hard part.

This. Any researcher should, over a lunch, be able to generate more idea than can be tackled in a life time.

The fact that a human expert can also do it doesn't mean the AI isn't valuable. Even if you just consider the monetary aspect, those few API calls would definitely be cheaper than buying the researcher lunch. But the big benefit is being able to generate those ideas immediately and autonomously every time there's new data.
Post reply on HN