211–220 of 266 posts
Edit: Actually right in the tickets themselves would probably be better and not require MCP... but still
Earlier quoted context omitted.
I agree with your overall message - rapid growth appears to encourage competition and forces companies to put their best foot forward. However, unfortunately, I cannot shower much praise on Claude 3.7. And if you (or anyone) asks why - 3.7 seems much better than 3.5, surely? - Then I’m moderately sure that you use Claude much more for coding than for any kind of conversation. In my opinion, even 3.5 Haiku (which is a…
My current hypothesis: the more familiar you are with a topic the worse the results from any LLM.
Earlier quoted context omitted.
3.7 did score higher in coding benchmarks but in practice 3.5 is much better at coding. 3.7 ignores instructions and does things you didn't ask it to do.
That has been the most annoying thing about it, so glad not paying for it anymore.
I think we are coming to a new automated technology ecosystem where LLMs will orchestrate many different parts of software with each other, speeding up the launch, evolution and monitoring of products.
The leap frogging at this point is getting insane (in a good way, I guess?). The amount of time each state of the art feature gets before it's supplanted is a few weeks at this point. LLMs were always a fun novelty for me until OpenAI DeepResearch which started to actually come up with useful results on more complex programming questions (where I needed to write all the code by hand but had to pull together lots of d…
Calling some APIs is leap-frogging? You could do this with GPT-3, nothing has changed except it's branded under a new name and tries to establish a (flawed) standard. If there was truly any innovation still happening in OpenAI, Anthropic, etc., they would be working on models only, not on side features that someone could already develop over a weekend.
Earlier quoted context omitted.
why is finetuning talked about so much less than RAG? is it not viable at all?
RAG is infinitely more accessible and cheaper than finetuning. But it is true that finetuning is getting severely overlooked in situations where it would outperform alternatives like RAG.
How many epochs do you run?
Earlier quoted context omitted.
why is finetuning talked about so much less than RAG? is it not viable at all?
There were initial difficulties in finetuning that made it less appealing early on, and that's snowballed a bit into having more of a focus on RAG. Some of the issues still exist, of course: * Finetuning takes time and compute; for one-off queries using in-context learning is vastly more efficient (i.e., look it up with RAG). * Early results with finetuning had trouble reliably memorizing information. We've got a muc…
I had no idea that fine tuning for adding information is viable now. Last I checked (year+ back) it seemed to not work well.
An AI that is capable of responding to a "How do I do X" prompt with "Hey this seems related to a ticket that was already opened on your Jira 2 months ago", or "There is a document about this in Sharepoint", it would bring me such immense value, I think I might cry. Edit: Actually right in the tickets themselves would probably be better and not require MCP... but still
So if you ask it “who is in charge of marketing” it will read it off sharepoint instead of answering generically
Anthropic's strategy seems to go towards "AI as universal glue". They want to tie Claude into all the tools teams already live in (Jira, Confluence, Zapier, etc.). That's a smart move for enterprise adoption, but it also feels like they're compensating for a plateau in core model capabilities. Both OpenAI and Google continue to push the frontier on reasoning, multimodality, and efficiency whereas Claude's recent rele…
I would expect Slack do this. Maybe Slack and Claude should merge one day, given MS and Google has their own core models.
Earlier quoted context omitted.
I agree with your overall message - rapid growth appears to encourage competition and forces companies to put their best foot forward. However, unfortunately, I cannot shower much praise on Claude 3.7. And if you (or anyone) asks why - 3.7 seems much better than 3.5, surely? - Then I’m moderately sure that you use Claude much more for coding than for any kind of conversation. In my opinion, even 3.5 Haiku (which is a…
3.7 did score higher in coding benchmarks but in practice 3.5 is much better at coding. 3.7 ignores instructions and does things you didn't ask it to do.