Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Science

claude.com

Post reply on HN
Print view View original

181–190 of 199 posts

Re: Claude Science

#181

Post by [deleted] »

ricksunny wrote: post #163
lebovic wrote: post #44

I built one of the connected tools included in this launch (the Biomni HPC [1]), and I have spent an inordinate amount of my life working on this problem. (I also worked at Anthropic, but not on this product.) As other comments have pointed out, this is for data science – but it's capable of more than making plots and writing papers [2]. It has integrations with many databases and computational tools, including a res…

Thank you for this summary. Especially interested about the wetlab & CRO tie-in. What is meant by a ‘researcher’s institutional cluster’?

[deleted]

Re: Claude Science

#182

Post by lebovic »

dominotw wrote: post #172
jessetemp wrote: post #91

Earlier quoted context omitted.

How do you validate this kind of work to weed out any confabulating by the LLMs?

i love how gp posted a glowing review and then dipped out.

After spending years on a problem, it's exciting to see it start to get more attention and move towards being meaningfully solved.

But I try to limit my time on HN, and I thought someone who works on Claude Science might respond to this thread later.

Re: Claude Science

#183

Post by ahknight »

packeted wrote: post #152
yuppiepuppie wrote: post #148

Earlier quoted context omitted.

Not sure how to feel about this. I think its super cool that you can dive into this, but it sucks that its your son that has this condition for which you have to do this analysis. I hope it all turns out well. Quick question: where did you get your genome read and get the raw files? As far as I know, as service like 23andme does not give you back the raw files.

Thanks for the kind words. Actually we got the trio whole genome sequencing through our neurologist/geneticist a couple of years ago. It was performed by a company called GeneDx. They interpreted the data at the time which is how we got to a diagnosis but knowing I'd want to dive in to it later I asked for the raw data. They provided it as the raw CRAM files and also the VCF (variant call files) which are a bit small…

I can vouch for Nebula. Just follow their recommendation to not get the 1x service if you want to use the data for medical reasons. The more passes they make, the more accurate the result.

Downside: those files are HUUUUUGE. Have a good reason to do it before pulling one down and trying to work with it.

Re: Claude Science

#184

Post by packeted »

ahknight wrote: post #183
packeted wrote: post #152

Earlier quoted context omitted.

Thanks for the kind words. Actually we got the trio whole genome sequencing through our neurologist/geneticist a couple of years ago. It was performed by a company called GeneDx. They interpreted the data at the time which is how we got to a diagnosis but knowing I'd want to dive in to it later I asked for the raw data. They provided it as the raw CRAM files and also the VCF (variant call files) which are a bit small…

I can vouch for Nebula. Just follow their recommendation to not get the 1x service if you want to use the data for medical reasons. The more passes they make, the more accurate the result. Downside: those files are HUUUUUGE. Have a good reason to do it before pulling one down and trying to work with it.

Files are huge for sure (mine are like 25Gb x 3 of us) but Claude Science / Macbook Pro M5 Max knocked out my analyses in minutes using command line tools like bcftools and scripting. Honestly think this opens up WGS data to much easier analysis and refreshing those analyses in minutes as new discoveries are made.

Re: Claude Science

#185

Post by cantdutchthis »

pheewma wrote: post #168
IanCal wrote: post #138

Earlier quoted context omitted.

There’s a dead sibling comment but I’d also recommend looking at marimo, I just used it to do some analysis for my brother in law and has Claude write the whole thing. It tracks variables used across cells to see what needs re-running. It’s also got an in built AI helper thing where you can put an api key but I’ve not tried that yet.

Just to add to this: Marimo notebooks are set up in a way that ends up being easier/more-token-efficient for agents to work in. They're just python files with some custom formatting to be read by their front-end UI, sort of like cell notation in (# %%) but with a lot of QoL bells and whistles built in. Jupyter notebooks in comparison, are mostly json and are pretty messy to work with as far as git diffs goes, which i…

Not to mention the fact that marimo-pair also supports non-Claude models.

Re: Claude Science

#187

Post by ep103 »

lebovic wrote: post #180
jessetemp wrote: post #91

Earlier quoted context omitted.

How do you validate this kind of work to weed out any confabulating by the LLMs?

I assume they do hallucinate, just like with coding or finding vulnerabilities. You can try to minimize it (e.g. with a reviewer agent, which Claude Science and Biomni have), but nothing is perfect, so I limit autonomous work to verifiable problems and review it.

Honestly, this is how all AI should be used, in most non-trivial scenearios

Re: Claude Science

#188

Post by ml_more »

jessetemp wrote: post #91
lebovic wrote: post #44

I built one of the connected tools included in this launch (the Biomni HPC [1]), and I have spent an inordinate amount of my life working on this problem. (I also worked at Anthropic, but not on this product.) As other comments have pointed out, this is for data science – but it's capable of more than making plots and writing papers [2]. It has integrations with many databases and computational tools, including a res…

How do you validate this kind of work to weed out any confabulating by the LLMs?

When you set up your Claude Science instance you can see that they're connecting to crossref, semantic scholar, pubmed, ArXiv, FDA. They instruct the LLM to validate citations.

My testing with this technique indicates that method they seem to be using (rag with an instruction to check sources) will reduce the confabulation rate for citations from the base rate 50-60% for regular models (e.g. regular Claude) to 5-15% (depending on how they implemented it). On the one hand this is way better. On the other hand it's just good enough that your spot check will look good and your work will still contain hallucinations (which is probably worse than obviously bad).

Getting to zero confabulation would require a different process. (stand-alone validation engine running in parallel in real-time which is hard but not impossible.)

Re: Claude Science

#190

Post by ml_more »

ai_fry_ur_brain wrote: post #87
PotatoFarmsKing wrote: post #74

Before LLMs the tech groups I followed were ripping with discussions about this and that topic, what to use and when; I believe these discussions sparked the creation of many frameworks and tools out of "this seems like a good idea, wouldn't hurt to implement it". Unfortunately it all resolves around LLMs nowadays and how to make some LLM work some way or another, we don't even discuss the very topics the groups were…

Well LLMs are largely useless and people are realizing that.

LLMs with a verification layer work great (code with tests) I know my field quite well and I can one-shot many useful things. I can't trust any of it but I can trust tests and verification tools.
Post reply on HN