951–960 of 1001 posts
Earlier quoted context omitted.
Perhaps use pastebin for synthetic content next time?
> synthetic content I haven't heard this phrase. Thank you, I'll certainly be using it.
https://en.m.wikipedia.org/wiki/Synthetic_media
I believe your message got downvoted because is synthetic, but because is wrong or not of interest.
You can get your HN profile analyzed by it and it's pretty funny :) https://hn-wrapped.kadoa.com/ I'm using this to test the humor of new models.
On the nail
Earlier quoted context omitted.
I like to make up my own tests, that way you know it is actually thinking. Tests that require thinking about the physical world are the most revealing. My new favourite is: You have 2 minutes to cool down a cup of coffee to the lowest temp you can. You have two options: 1. Add cold milk immediately, then let it sit for 2 mins. 2. Let it sit for 2 mins, then add cold milk. Which one cools the coffee to the lowest temp…
No need for thinking, that question can be found discussed and explained many times online and has almost certainly been part of the training data.
They get caught up in the idea that adding milk first cools it fastest and can’t escape from that
Earlier quoted context omitted.
Out of curiosity, what is the answer? From your comment, it seems like the more obvious choice is the incorrect one. EDIT: By the more obvious one, I mean letting it cool and then adding milk. As the temperature difference between the coffee and the surrounding air is higher, the coffee cools down faster. Is this wrong?
Isn't the answer milk first, then let sit? You only have 2 minutes, so if you're adding the milk after 2 minutes have already elapsed, then you've already exceeded the time limit, meaning the final measurement would take place before the milk is even poured in.
The bigger the temp difference the more cooling. So by putting the milk in right away you make the temp difference between the surroundings and the coffee smaller = less cooling over your 2 mins.
I like puzzles with a satisfying answer
Earlier quoted context omitted.
I like to make up my own tests, that way you know it is actually thinking. Tests that require thinking about the physical world are the most revealing. My new favourite is: You have 2 minutes to cool down a cup of coffee to the lowest temp you can. You have two options: 1. Add cold milk immediately, then let it sit for 2 mins. 2. Let it sit for 2 mins, then add cold milk. Which one cools the coffee to the lowest temp…
> Phrased this way without any help, all but the thinking models get it wrong I C&P'd it into Claude 3.7 with thinking, and it gave the correct answer (which I'm pretty sure is #2). Including the CoT, where it actually does math (which I haven't checked), and final response. # THINKING Let's analyze the two options. Option 1: Add cold milk immediately, then let it sit for 2 mins. Option 2: Let it sit for 2 mins, then…
It was meant to be an example of thinking being better…
Earlier quoted context omitted.
> 225 coding exercises from Exercism Has there been any effort taken to reduce data leakage of this test set? Sounds like these exercises were available on the internet pre-2023, so they'll probably be included in the training data for any modern model, no?
I like to make up my own tests, that way you know it is actually thinking. Tests that require thinking about the physical world are the most revealing. My new favourite is: You have 2 minutes to cool down a cup of coffee to the lowest temp you can. You have two options: 1. Add cold milk immediately, then let it sit for 2 mins. 2. Let it sit for 2 mins, then add cold milk. Which one cools the coffee to the lowest temp…
You can get your HN profile analyzed by it and it's pretty funny :) https://hn-wrapped.kadoa.com/ I'm using this to test the humor of new models.
Lol!
I got this working with my LLM tool (new plugin version: llm-anthropic 0.14) and figured out a bunch of things about the model in the process. My detailed notes are here: https://simonwillison.net/2025/Feb/25/llm-anthropic-014/ One of the most exciting new capabilities is that this model has a 120,000 token output limit - up from just 8,000 for the previous Claude 3.5 Sonnet model and way higher than any other model…
I got this working with my LLM tool (new plugin version: llm-anthropic 0.14) and figured out a bunch of things about the model in the process. My detailed notes are here: https://simonwillison.net/2025/Feb/25/llm-anthropic-014/ One of the most exciting new capabilities is that this model has a 120,000 token output limit - up from just 8,000 for the previous Claude 3.5 Sonnet model and way higher than any other model…
Simon, do you write anywhere how do you manage to be so... active? Between your programming tools, blogging, job (I assume you work?) where do you find the time/energy?
I really want to be able to see what specifically is changing, not just the entire new file.
Also, if the user provides a file for modification, make that available as Version 0 (or whatever), so we can diff against that.