921–930 of 1001 posts
Earlier quoted context omitted.
How does it stack up against Grok3? I've seen some discussion that Grok3 is good for coding.
It isn't available over api yet, as far as I know. So it can't be really tested independently.
Earlier quoted context omitted.
I'm sure lots of potential students / bootcampers are now not going into programming (or if they are, the smart ones try to go into niches like A.I and skip web/backend/android altogether). This will work against the numbers of jobs being reduced by A.I. It will take a few years though to play out , but at some point we will see smaller amounts of people trying to get into the field and applying for jobs, certainly f…
When I say 10 years I say that I've probably wanted to work in this field since maybe 10. Computing is my autistic hyperfixation. This is why I'm so frustrated.
You can get your HN profile analyzed by it and it's pretty funny :) https://hn-wrapped.kadoa.com/ I'm using this to test the humor of new models.
Earlier quoted context omitted.
I have a very long request to do like this, did you use a specific CLI tool ? (Thank you in advance)
I used my own CLI tool LLM, which can handle these long requests in streaming mode (Anthropic won't let you do a non-streaming request for long output replies like this). uv tool install llm llm install llm-anthropic llm keys set anthropic # paste in API key llm -m claude-3.7-sonnet -o thinking 1 'your prompt goes here'
Earlier quoted context omitted.
> 225 coding exercises from Exercism Has there been any effort taken to reduce data leakage of this test set? Sounds like these exercises were available on the internet pre-2023, so they'll probably be included in the training data for any modern model, no?
I like to make up my own tests, that way you know it is actually thinking. Tests that require thinking about the physical world are the most revealing. My new favourite is: You have 2 minutes to cool down a cup of coffee to the lowest temp you can. You have two options: 1. Add cold milk immediately, then let it sit for 2 mins. 2. Let it sit for 2 mins, then add cold milk. Which one cools the coffee to the lowest temp…
For small models and when attention is "taken up", these sorts of questions really send a model for a loop. Agreed - especially noticeable with small reasoning models.
Earlier quoted context omitted.
I think most language servers have a feature like this right?
Moving a function or class? Yes. But moving arbitrary lines of code into their own function in a new module is still a PITA, particularly when the lines of code are not consecutive.
Earlier quoted context omitted.
Serious answer: learn to code. You still need to know what good code looks like to use these tools. If you go forward in your career trusting the output of LLMs without the skills to evaluate the correctness, style, functionality of that code then you will have problems. People still write low level machine code today, despite compilers having existed for 70+ (?) years. We'll always need full-stack humans who underst…
+1 to this. There has never been a better time to learn to code - the learning curve is being shaved down by these new LLM-based tools, and the amount of value people with programming literacy can produce is going up by an order of magnitude. People who know both coding and LLMs will be a whole lot more attractive to hire to build software than people who just know LLMs for many years to come.
Earlier quoted context omitted.
Have you tried Claude 3.7 + Deepseek as the architect? Seeing as "DeepSeek R1 + claude-3-5-sonnet-20241022" is the second place option, "DeepSeek R1 + claude-3-7" would hopefully be the highest ranking choice so far?
It looks like Sonnet 3.7 (extended thinking) would be a better architect than R1. I'll be trying out Sonnet 3.7 extended thinking + Sonnet 3.5 or Flash 2.0, which I assume would be at the top of the leaderboard.
I'd like to see that benchmark, but R1 + 3.7 should be cheaper than 3.7T + 3.7
Earlier quoted context omitted.
It looks like Sonnet 3.7 (extended thinking) would be a better architect than R1. I'll be trying out Sonnet 3.7 extended thinking + Sonnet 3.5 or Flash 2.0, which I assume would be at the top of the leaderboard.
given 3.5 and 3.7 cost the same, it doesn't make sense to use 3.5 here. I'd like to see that benchmark, but R1 + 3.7 should be cheaper than 3.7T + 3.7
Flash 2.0 got 100% on the edit format, and it's extremely cheap, so I'm pretty curious how that would score.
Earlier quoted context omitted.
The fact that the answer is interesting makes me suspect that it's not a good test for thinking. I remember reading the explanation for the answer somewhere on the internet years ago, and it's stayed with me ever since. It's interesting enough that it's probably been written about multiple times in multiple places. So I think it would probably stay with a transformer trained on large volumes of data from the internet…
Out of curiosity, what is the answer? From your comment, it seems like the more obvious choice is the incorrect one. EDIT: By the more obvious one, I mean letting it cool and then adding milk. As the temperature difference between the coffee and the surrounding air is higher, the coffee cools down faster. Is this wrong?