Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Integrations

anthropic.com

Post reply on HN
Print view View original

201–210 of 266 posts

Re: Claude Integrations

#201

Post by fakedang »

tomrod wrote: post #168
Balgair wrote: post #151

Earlier quoted context omitted.

I think we have very different definitions of the word 'research' then. I'd say that what you're saying is 'synthesis'. The 'Intro/Discussion' sections of a journal article. For me, 'research' means the work of going through and getting all the data in the first place. Like, going out and collecting dino bones in the hot sun, measuring all the soil samples, etc. - that is research. For me, asking these models to go c…

It's excellent at producing short literature reviews on open access papers and data. It has no sense of judgment, trusting most sources unless instructed otherwise.

Gemini's Deep Research is very good at discriminating between sources though, in my experience (haven't tried Claude or Perplexity). It finds really obscure but very relevant documents that don't even show up in Google Search for the same queries. It also discounts results that are otherwise irrelevant or very low-value from the final report. But again, it is just a starting point as the generated report is too short, and I make sure to check all the references it gives once again. But that's where I find its value.

Re: Claude Integrations

#202

Post by franga2000 »

jeswin wrote: post #171
garrickvanburen wrote: post #158

Earlier quoted context omitted.

My current hypothesis: the more familiar you are with a topic the worse the results from any LLM.

> My current hypothesis: the more familiar you are with a topic the worse the results from any LLM. That's not really true, since your prompts are also getting better. Better input leads to better output remains true, even with LLMs (when you see it as a tool).

Being more familiar with the topic definitely doesn't always make your prompts better. For a lot of things it doesn't really change (explain X, compare X and Y...) - and this is what is being discussed it. For giving "building" instructions (like writing code) it helps a bit, but even if you know exactly what you want it to write, getting it to do that is pretty much trial and errror (too much detail makes it follow word-for-word and produce bad code, too little and it misses important parts or makes dumb mistakes).

Re: Claude Integrations

#203

Post by Manfred »

insin wrote: post #131
n_ary wrote: post #30

Is this the beginning of the apps for everything era and finally the SaaS for your LLM begins? Initially we had internet but value came when instead of installed apps, webapps arrived to become SaaS. Now if LLMs can use specific remote MCP which is another SaaS for your LLM, the remote MCP powered service can charge a subscription to do wonderful things and voila! Let the new golden age of SaaS for LLMs begin and the…

It's perfect, nobody will have time to care about how many 9s your service has because the nondeterministic failure mode now sitting slap-bang in the middle is their problem!

Imagine dynamic subscription rates based on vibes where you won't even notice price hikes because not even the supplier can explain what they are.

Re: Claude Integrations

#204

Post by suyash »

tiberriver256 wrote: post #141
user_7832 wrote: post #26

Earlier quoted context omitted.

I agree with your overall message - rapid growth appears to encourage competition and forces companies to put their best foot forward. However, unfortunately, I cannot shower much praise on Claude 3.7. And if you (or anyone) asks why - 3.7 seems much better than 3.5, surely? - Then I’m moderately sure that you use Claude much more for coding than for any kind of conversation. In my opinion, even 3.5 Haiku (which is a…

3.7 did score higher in coding benchmarks but in practice 3.5 is much better at coding. 3.7 ignores instructions and does things you didn't ask it to do.

That has been the most annoying thing about it, so glad not paying for it anymore.

Re: Claude Integrations

#205

Post by user_7832 »

garrickvanburen wrote: post #158
user_7832 wrote: post #26

Earlier quoted context omitted.

I agree with your overall message - rapid growth appears to encourage competition and forces companies to put their best foot forward. However, unfortunately, I cannot shower much praise on Claude 3.7. And if you (or anyone) asks why - 3.7 seems much better than 3.5, surely? - Then I’m moderately sure that you use Claude much more for coding than for any kind of conversation. In my opinion, even 3.5 Haiku (which is a…

My current hypothesis: the more familiar you are with a topic the worse the results from any LLM.

That is certainly the case in niche topics where published information is lacking, or needs common sense to synthesize proper outputs [1].

However in this specific example, I don't remember if it was chatgpt or gemini or 3.5 Haiku but the other(s) explained it well enough. I think I re-asked 3.5 Haiku at a later point of time, and to my complete non-surprise, it gave an answer that was quite decent.

1 - For example, the field of DIY audio - which was funnily enough the source of my question. I'm no speaker designer, but combining creativity with engineering basics/rules of thumb seems to be something LLms struggle with terribly. Ask them to design a speaker and they come up with the most vanilla, tired, textbook design - despite several existing market products that are already so much ahead/innovative.

I'm confident that if you asked an LLM an identical question for which there is more discourse - eg make an interesting/innovative phone - you'd get relatively much better results.

Re: Claude Integrations

#207

Post by kruxigt »

UncleEntity wrote: post #173
tiberriver256 wrote: post #141

Earlier quoted context omitted.

3.7 did score higher in coding benchmarks but in practice 3.5 is much better at coding. 3.7 ignores instructions and does things you didn't ask it to do.

I think it just does that to eat up your token quota and get you to upgrade. Like, ask it a simple question and it comes up with a full repo, complete with a README and a Makefile, when all you wanted to know was how efficient a particular algorithm would be in the included code. Can't wait until the add research to the Pro plan because, you know, I have questions...

[dead]

Re: Claude Integrations

#208

Post by kruxigt »

snewman wrote: post #193
fastball wrote: post #178

Earlier quoted context omitted.

Seems clear to me that Claude 3.7 suffers from overfitting, probably due to Anthropic seeing that 3.5 was a smash hit in the LLM coding space and deciding their North star for 3.7 should be coding benchmarks (which, like all benchmarks, do not properly capture the process of real-world coding). If it was actually good they would've named it 4.0, the fact that they went from 3.5 to 3.7 (weird jump) speaks volumes imo.

The numbering jump is because there was "Claude 3.5" and then "Claude 3.5 (new)" and they decided to retroactively stop the madness and rename the later to 3.6 (which is what everyone was calling it anyway).

[dead]

Re: Claude Integrations

#210

Post by jm547ster »

jeswin wrote: post #171
garrickvanburen wrote: post #158

Earlier quoted context omitted.

My current hypothesis: the more familiar you are with a topic the worse the results from any LLM.

> My current hypothesis: the more familiar you are with a topic the worse the results from any LLM. That's not really true, since your prompts are also getting better. Better input leads to better output remains true, even with LLMs (when you see it as a tool).

The opposite may be true, the more effective the model the lazier the prompting as it can seemingly handle not being micromanaged as with earlier versions.
Post reply on HN