Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Integrations

anthropic.com

Post reply on HN
Print view View original

171–180 of 266 posts

Re: Claude Integrations

#171

Post by jeswin »

garrickvanburen wrote: post #158
user_7832 wrote: post #26

Earlier quoted context omitted.

I agree with your overall message - rapid growth appears to encourage competition and forces companies to put their best foot forward. However, unfortunately, I cannot shower much praise on Claude 3.7. And if you (or anyone) asks why - 3.7 seems much better than 3.5, surely? - Then I’m moderately sure that you use Claude much more for coding than for any kind of conversation. In my opinion, even 3.5 Haiku (which is a…

My current hypothesis: the more familiar you are with a topic the worse the results from any LLM.

> My current hypothesis: the more familiar you are with a topic the worse the results from any LLM.

That's not really true, since your prompts are also getting better. Better input leads to better output remains true, even with LLMs (when you see it as a tool).

Re: Claude Integrations

#173

Post by UncleEntity »

tiberriver256 wrote: post #141
user_7832 wrote: post #26

Earlier quoted context omitted.

I agree with your overall message - rapid growth appears to encourage competition and forces companies to put their best foot forward. However, unfortunately, I cannot shower much praise on Claude 3.7. And if you (or anyone) asks why - 3.7 seems much better than 3.5, surely? - Then I’m moderately sure that you use Claude much more for coding than for any kind of conversation. In my opinion, even 3.5 Haiku (which is a…

3.7 did score higher in coding benchmarks but in practice 3.5 is much better at coding. 3.7 ignores instructions and does things you didn't ask it to do.

I think it just does that to eat up your token quota and get you to upgrade.

Like, ask it a simple question and it comes up with a full repo, complete with a README and a Makefile, when all you wanted to know was how efficient a particular algorithm would be in the included code.

Can't wait until the add research to the Pro plan because, you know, I have questions...

Re: Claude Integrations

#174

Post by mac-mc »

garrickvanburen wrote: post #158
user_7832 wrote: post #26

Earlier quoted context omitted.

I agree with your overall message - rapid growth appears to encourage competition and forces companies to put their best foot forward. However, unfortunately, I cannot shower much praise on Claude 3.7. And if you (or anyone) asks why - 3.7 seems much better than 3.5, surely? - Then I’m moderately sure that you use Claude much more for coding than for any kind of conversation. In my opinion, even 3.5 Haiku (which is a…

My current hypothesis: the more familiar you are with a topic the worse the results from any LLM.

He was saying that 3.5 is better than 3.7 on the same topic he knows well tho.

Re: Claude Integrations

#175

Post by marifjeren »

cruffle_duffle wrote: post #166
marifjeren wrote: post #161

Earlier quoted context omitted.

That github issue is closed but: > major concern that server hosters are on the hook to implement authorization Doesn't it make perfect sense for server hosters to implement that? If Claude wants access to my Jira instance on my behalf, and Jira hosts a remote MCP server that aids in exposing the resources I own, isn't it obvious Jira should be responsible for authorization? How else would they do it?

The authorization server and resource server can be separate entities. Meaning that jira instance can validate the token but not be the one issuing it or handling credentials.

Yes, this is true of OAuth, which is exactly what the latest Model context protocol is using.. What's the concern again?

I guess maybe you are saying the onus is NOT on the MCP server but on the authorization server.

Anyway while technically true this is mostly just distracting because:

1. in my experience the resource server and the authorization server are almost always maintained by the same company -- Jira/Atlassian being an example

2. the resource server still minimally has the responsibility of identifying and integrating with some authorization server, and *someone* has to be the authorization server, so I'm not sure deferring the responsibility to that unidentified party is a strong defense against the critique anyway. The strong defense is: of course the MCP server should have these responsibilities.

Re: Claude Integrations

#176

Post by XenophileJKO »

wyre wrote: post #159
Balgair wrote: post #150

Earlier quoted context omitted.

Yeah, that was intentional, well, somewhat. The project requires the full list of every known city in the western hemisphere and also Japan, Korea, and Taiwan. But that dataset is just maddeningly large, if it is possible at all. Like, I expect it to take me years, as I have to do a lot of translations. So, I figured that I'd be nice and just as for the top 250 for the various models. There's a lot more data that we'…

If you have the data, but need to parse all of it, couldn’t you upload it to your LLM of choice (with a large enough context window) and have it finish your project?

Well remember listing/ranking things are structurally hard for these models because you have to keep track of what it has listed and what it hasn't, etc.

Re: Claude Integrations

#177

Post by vineyardmike »

UncleEntity wrote: post #173
tiberriver256 wrote: post #141

Earlier quoted context omitted.

3.7 did score higher in coding benchmarks but in practice 3.5 is much better at coding. 3.7 ignores instructions and does things you didn't ask it to do.

I think it just does that to eat up your token quota and get you to upgrade. Like, ask it a simple question and it comes up with a full repo, complete with a README and a Makefile, when all you wanted to know was how efficient a particular algorithm would be in the included code. Can't wait until the add research to the Pro plan because, you know, I have questions...

> I think it just does that to eat up your token quota and get you to upgrade.

If you pay for a subscription then they don’t have an incentive to use more tokens for the same answer.

It’s definitely because feedback from people has “taught” it that more boilerplate is better. It’s the same reason ChatGPT is annoyingly complementary.

Re: Claude Integrations

#178

Post by fastball »

user_7832 wrote: post #26
throwup238 wrote: post #16

The leap frogging at this point is getting insane (in a good way, I guess?). The amount of time each state of the art feature gets before it's supplanted is a few weeks at this point. LLMs were always a fun novelty for me until OpenAI DeepResearch which started to actually come up with useful results on more complex programming questions (where I needed to write all the code by hand but had to pull together lots of d…

I agree with your overall message - rapid growth appears to encourage competition and forces companies to put their best foot forward. However, unfortunately, I cannot shower much praise on Claude 3.7. And if you (or anyone) asks why - 3.7 seems much better than 3.5, surely? - Then I’m moderately sure that you use Claude much more for coding than for any kind of conversation. In my opinion, even 3.5 Haiku (which is a…

Seems clear to me that Claude 3.7 suffers from overfitting, probably due to Anthropic seeing that 3.5 was a smash hit in the LLM coding space and deciding their North star for 3.7 should be coding benchmarks (which, like all benchmarks, do not properly capture the process of real-world coding).

If it was actually good they would've named it 4.0, the fact that they went from 3.5 to 3.7 (weird jump) speaks volumes imo.

Re: Claude Integrations

#179

Post by meander_water »

marifjeren wrote: post #175
cruffle_duffle wrote: post #166

Earlier quoted context omitted.

The authorization server and resource server can be separate entities. Meaning that jira instance can validate the token but not be the one issuing it or handling credentials.

Yes, this is true of OAuth, which is exactly what the latest Model context protocol is using.. What's the concern again? I guess maybe you are saying the onus is NOT on the MCP server but on the authorization server. Anyway while technically true this is mostly just distracting because: 1. in my experience the resource server and the authorization server are almost always maintained by the same company -- Jira/Atlass…

I think the pain points will be mostly for enterprise customers who want to integrate servers into their auth systems.

For example, say you have a JIRA self hosted instance with SSO to entra id. You can't just install an MCP server off the shelf because authZ and resources are tightly coupled and implementation specific. It would be much easier if the server only handled providing resources, and authZ was offloaded to a provider of your choosing.

Re: Claude Integrations

#180

Post by marifjeren »

meander_water wrote: post #179
marifjeren wrote: post #175

Earlier quoted context omitted.

Yes, this is true of OAuth, which is exactly what the latest Model context protocol is using.. What's the concern again? I guess maybe you are saying the onus is NOT on the MCP server but on the authorization server. Anyway while technically true this is mostly just distracting because: 1. in my experience the resource server and the authorization server are almost always maintained by the same company -- Jira/Atlass…

I think the pain points will be mostly for enterprise customers who want to integrate servers into their auth systems. For example, say you have a JIRA self hosted instance with SSO to entra id. You can't just install an MCP server off the shelf because authZ and resources are tightly coupled and implementation specific. It would be much easier if the server only handled providing resources, and authZ was offloaded t…

I'm under the impression that what you described is exactly how the new model context protocol works, since it's using oauth and is therefore unaware of any of the authentication (eg SSO) details. Your authentication process could be done via carrier pigeon and Claude would be none the wiser.
Post reply on HN