Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Science

claude.com

Post reply on HN
Print view View original

171–180 of 199 posts

Re: Claude Science

#171

Post by maleldil »

Daishiman wrote: post #77
trollbridge wrote: post #71

Earlier quoted context omitted.

There is an obscure topic where I have read basically every single dissertation, study, etc on that topic (or even just articles that mention it). It is very noticeable how much briefer older publications were. It would be impossible to do that today. I guess I could have an LLM just summarise all the papers…

What's the reason for this? Publish-or-perish? Papers have to be more thorough? Extra junk tacked on for the sake of showing lengthier papers?

CS conference papers often have page limits (e.g. ACL ARR is 8 pages double-column), so most paper main bodies are exactly that, as it's seen as sloppy if you don't use the full count. I've had someone point out that it's best to use the entire page 8 without leaving any gaps.

There are also appendices, which reviewers aren't required to read. If there's something relevant that doesn't fit in the main body and you don't put it in the appendix, reviewers will point it out and ask for it, and it will influence their grades.

In the end, publishing papers these days is about convincing reviewers rather than actually writing a good paper. And you usually have reviewers asking for all sorts of things.

Re: Claude Science

#172

Post by dominotw »

jessetemp wrote: post #91
lebovic wrote: post #44

I built one of the connected tools included in this launch (the Biomni HPC [1]), and I have spent an inordinate amount of my life working on this problem. (I also worked at Anthropic, but not on this product.) As other comments have pointed out, this is for data science – but it's capable of more than making plots and writing papers [2]. It has integrations with many databases and computational tools, including a res…

How do you validate this kind of work to weed out any confabulating by the LLMs?

i love how gp posted a glowing review and then dipped out.

Re: Claude Science

#173

Post by foxyv »

applicative wrote: post #119
foxyv wrote: post #89

Earlier quoted context omitted.

Raw dog Chat LLMs are pretty worthless. But run an agent with tool invocation and they get scary good. It's amazing how much reasoning is packed into the English language. Provide your model with enough information and it can pull some miracles out of thin air. It's not the "Replace humans" level yet, but you can automate a lot of stuff you wouldn't expect to be able to automate.

What you are saying, if I follow, is that LLMs basically worthless: it turns out that coding is so simple that verifiable rewards can tune weights surprisingly well for that one peculiar task. ('agentic' is fancy word for letting them run what they write - not to put too fine a point on it.) You've made the most damning remark against Planet LLM I've read.

LLMs are essentially a text prediction engine. This can be used for basic reasoning tasks, however the LLM doesn't have much in the way of actual knowledge. However, it can use knowledge that exists to predict text better. For instance, if you plug a bunch of scientific articles into it, it will be really good at answering questions about the subject of those articles.

The problem is that context windows are very short. 200k-1M tokens or so. This means that the model needs to focus down on very specific information if possible. This is what makes tool using, reasoning, and agentic AI very powerful. The model can find the most relevant information it needs within its limited context and generate relevant answers to questions. The LLM pulls from web searches, documentation, long term memories in graph databases, and database queries to answer the questions using real information.

Re: Claude Science

#174

Post by foxyv »

ai_fry_ur_brain wrote: post #120
foxyv wrote: post #89

Earlier quoted context omitted.

Raw dog Chat LLMs are pretty worthless. But run an agent with tool invocation and they get scary good. It's amazing how much reasoning is packed into the English language. Provide your model with enough information and it can pull some miracles out of thin air. It's not the "Replace humans" level yet, but you can automate a lot of stuff you wouldn't expect to be able to automate.

No its not doing magic. Im impressed when anyone can play a guitar, because I dont know anything about playing a guitar. Someone who's been playing the guitar for years isnt impressed by all guitar players. This seems the case with many people using llms to write code. They think everything an llm does is magical . It will never be able to replace humans with two brain cells.

I am using miracle in the sense of: "So astounding as to suggest a miracle; phenomenal."

It isn't magic, it's just math.

Re: Claude Science

#178

Post by maherbeg »

letmetweakit wrote: post #142
packeted wrote: post #136

I watched the announcement and gave it a spin as I'm a heavy user of cowork/code. So far I'm super impressed. I used it to analyze my whole genome sequencing data I have as my son has a rare genetic condition. I used it to answer a question I'd asked a few bioinformaticians to help me with but never got a satisfactory answer, it solved it in about a minute - whether his n-of-1 de novo, heterozygous single nucleotide…

You're not worried your whole genome is being sent over to some commercial entity?

I wouldn't care where my data went if it was to help my children.

Re: Claude Science

#180

Post by lebovic »

jessetemp wrote: post #91
lebovic wrote: post #44

I built one of the connected tools included in this launch (the Biomni HPC [1]), and I have spent an inordinate amount of my life working on this problem. (I also worked at Anthropic, but not on this product.) As other comments have pointed out, this is for data science – but it's capable of more than making plots and writing papers [2]. It has integrations with many databases and computational tools, including a res…

How do you validate this kind of work to weed out any confabulating by the LLMs?

I assume they do hallucinate, just like with coding or finding vulnerabilities.

You can try to minimize it (e.g. with a reviewer agent, which Claude Science and Biomni have), but nothing is perfect, so I limit autonomous work to verifiable problems and review it.

Post reply on HN