241–250 of 266 posts
Earlier quoted context omitted.
My current hypothesis: the more familiar you are with a topic the worse the results from any LLM.
Amen to this. As soon as you ask an LLM to explain something in detail that you’re a domain expert in, that’s when you notice the flaws.
For example, I’m not a domain expert, but I was looking for an RC motor for a toy project and OpenAI had happily tried to source a few, with Deep Research. Only the best candidate it had picked contained an obvious typo in the motor spec (68 grams instead of 680 grams), which is just impossible for a motor of specified dimensions.
Earlier quoted context omitted.
Amen to this. As soon as you ask an LLM to explain something in detail that you’re a domain expert in, that’s when you notice the flaws.
Yes, it’s particularly bad when the information found on the web is flawed. For example, I’m not a domain expert, but I was looking for an RC motor for a toy project and OpenAI had happily tried to source a few, with Deep Research. Only the best candidate it had picked contained an obvious typo in the motor spec (68 grams instead of 680 grams), which is just impossible for a motor of specified dimensions.
It's funny you say that because I was going to echo your parents sentiment and point out it's exactly the same with any news article you read.
The majority if content these LLMs are consuming is not from domain experts.
The leap frogging at this point is getting insane (in a good way, I guess?). The amount of time each state of the art feature gets before it's supplanted is a few weeks at this point. LLMs were always a fun novelty for me until OpenAI DeepResearch which started to actually come up with useful results on more complex programming questions (where I needed to write all the code by hand but had to pull together lots of d…
I agree with your overall message - rapid growth appears to encourage competition and forces companies to put their best foot forward. However, unfortunately, I cannot shower much praise on Claude 3.7. And if you (or anyone) asks why - 3.7 seems much better than 3.5, surely? - Then I’m moderately sure that you use Claude much more for coding than for any kind of conversation. In my opinion, even 3.5 Haiku (which is a…
3.7 is definitively better at coding but you feel it lost a bit of maneuverability at other domains. For someone who wants code generated, it doesn't matter but I've found myself using DeepSeek first and then getting code output by 3.7.
Earlier quoted context omitted.
Amen to this. As soon as you ask an LLM to explain something in detail that you’re a domain expert in, that’s when you notice the flaws.
Yes, it’s particularly bad when the information found on the web is flawed. For example, I’m not a domain expert, but I was looking for an RC motor for a toy project and OpenAI had happily tried to source a few, with Deep Research. Only the best candidate it had picked contained an obvious typo in the motor spec (68 grams instead of 680 grams), which is just impossible for a motor of specified dimensions.
Looks like this is possible due to the relatively recent addition of OAuth2.1 to the MCP spec [0] to allow secure comms to remote servers. However, there's a major concern that server hosters are on the hook to implement authorization. Ongoing discussion here [1]. [0] https://modelcontextprotocol.io/specification/2025-03-26 [1] https://github.com/modelcontextprotocol/modelcontextprotocol...
That github issue is closed but: > major concern that server hosters are on the hook to implement authorization Doesn't it make perfect sense for server hosters to implement that? If Claude wants access to my Jira instance on my behalf, and Jira hosts a remote MCP server that aids in exposing the resources I own, isn't it obvious Jira should be responsible for authorization? How else would they do it?
> Web search is now globally available to all Claude.ai paid plans.
I don't know why web search is such a big deal. You can implement it with any LLM that offers an API and function calling.
Its only a matter of time where folks write user stories and an LLM takes over for the first draft, then iterate from there. Btw, that speaks to how important it is to get clear business requirements for work.