551–560 of 944 posts
Earlier quoted context omitted.
My vibe question checking suggests otherwise. Even o3-mini-high is not as good as r1, even though it's faster than r1. Considering o3-mini is more expensive per token. It's not clear o3-mini-high is cheaper than r1 either even r1 probably consumes more token per answer.
well in my anecdotal tests, o3 mini (free) performed better than r1
I've been using cursor since it launched, sticking almost exclusively to claude-3.5-sonnet because it is incredibly consistent, and rarely loses the plot. As subsequent models have been released, most of which claim to be better at coding, I've switched cursor to it to give them a try. o1, o1-pro, deepseek-r1, and the now o3-mini. All of these models suffer from the exact same "adhd." As an example, in a NextJS app,…
Context length possibly. Prompt adherence drops off with context, and anything above 20k tokens is pushing it. I get the best results by presenting the smallest amount of context possible, including removing comments and main methods and functions that it doesn't need to see. It's a bit more work (not that much if you have a script that does it for you), but the results are worth it. You could test in the chatgpt app…
Context alone shouldn't be the reason that sonnet succeeds consistently, but others (some which have even bigger context windows) fail.
I've been using cursor since it launched, sticking almost exclusively to claude-3.5-sonnet because it is incredibly consistent, and rarely loses the plot. As subsequent models have been released, most of which claim to be better at coding, I've switched cursor to it to give them a try. o1, o1-pro, deepseek-r1, and the now o3-mini. All of these models suffer from the exact same "adhd." As an example, in a NextJS app,…
Try using Gemini thinking with Cursor. It barely works. Cmd-k outputs the thinking into the code. Its unusable in chat because the formatting sucks.
Is there some relationship between Cursor and Anthropic, i wonder. Plenty of other platforms seem very eager to give users model flexibility, but Cursor seems to be lacking.
I could be wrong, just an observation.
I used o3-mini to summarize this thread so far. Here's the result: https://gist.github.com/simonw/09e5922be0cbb85894cf05e6d75ae... For 18,936 input, 2,905 output it cost 3.3612 cents. Here's the script I used to do it: https://til.simonwillison.net/llms/claude-hacker-news-themes...
Earlier quoted context omitted.
Context length possibly. Prompt adherence drops off with context, and anything above 20k tokens is pushing it. I get the best results by presenting the smallest amount of context possible, including removing comments and main methods and functions that it doesn't need to see. It's a bit more work (not that much if you have a script that does it for you), but the results are worth it. You could test in the chatgpt app…
If it's a context issue, it's an issue with how cursor itself sends the context to these reasoning LLMs. Context alone shouldn't be the reason that sonnet succeeds consistently, but others (some which have even bigger context windows) fail.
The advertised context length being longer doesn't necessarily map 1:1 with the actual ability the models have to perform difficult tasks over that full context. See for example the plots of performance on ARC vs context length for o-series models.
I've been using cursor since it launched, sticking almost exclusively to claude-3.5-sonnet because it is incredibly consistent, and rarely loses the plot. As subsequent models have been released, most of which claim to be better at coding, I've switched cursor to it to give them a try. o1, o1-pro, deepseek-r1, and the now o3-mini. All of these models suffer from the exact same "adhd." As an example, in a NextJS app,…
And on the side of not being able to get models to understand something specific, there’s a place in a current project where I use a special Unicode apostrophe during some string parsing because a third-party API needs it. But any code modifications by the AI to that file always replace it with a standard ascii apostrophe. I even added a comment on that line to the effect of “never replaced this apostrophe, it’s important to leave it exactly as it is!” And also put that in my cursor rules, and sometimes directly in the prompt as well, but it always replaces it even for completely unrelated changes. I’ve had to manually fix it like 10 times in the last day, it’s infuriating.
I used o3-mini to summarize this thread so far. Here's the result: https://gist.github.com/simonw/09e5922be0cbb85894cf05e6d75ae... For 18,936 input, 2,905 output it cost 3.3612 cents. Here's the script I used to do it: https://til.simonwillison.net/llms/claude-hacker-news-themes...
For those that like simpler ways (although dependent on Google) NotebookLM does all that in 2 clicks. And you can ask it questions about it, references are provided.
llm -c "did anyone talk about pricing?"Earlier quoted context omitted.
There's no moat, and they have to work even harder. Competition is good.
It is still curious though as far as what is actually being automated? I find huge value in these models as an augmentation of my intelligence and as a kind of cybernetic partner. I can't think of anything that can actually be automated though in terms of white collar jobs. The white collar model test case I have in mind is a bank analyst under a bank operations manger. I have done both in the past but there is somet…
Earlier quoted context omitted.
My vibe question checking suggests otherwise. Even o3-mini-high is not as good as r1, even though it's faster than r1. Considering o3-mini is more expensive per token. It's not clear o3-mini-high is cheaper than r1 either even r1 probably consumes more token per answer.
well in my anecdotal tests, o3 mini (free) performed better than r1
I used o3-mini to summarize this thread so far. Here's the result: https://gist.github.com/simonw/09e5922be0cbb85894cf05e6d75ae... For 18,936 input, 2,905 output it cost 3.3612 cents. Here's the script I used to do it: https://til.simonwillison.net/llms/claude-hacker-news-themes...
I noticed that it thought that GoatInGrey wrote “openai is no longer relevant.” However, they were just quoting a different user (buyucu) who was the person who first wrote that.
I originally did that to save on tokens but modern models have much larger input windows so I may not need to do that any more.