421–430 of 513 posts
Earlier quoted context omitted.
I also like Perplexity’s 3/day limit! If I use them up (which I almost never do) I can just refresh the next day
I've only ever had to use DeepResearch for academic literature review. What do you guys use it for which hits your quotas so quickly?
Earlier quoted context omitted.
I'm surprised and a little disappointed by the result concerning instructions at the top, because it's incompatible with prompt caching: I would much rather cache the part of the prompt that includes the long document and then swap out the user question at the end.
Wouldn’t it be the other way around? If the instructions are at the top the LV cache entries can be pre computed and cached. If they’re at the bottom the entries at the lower layers will have a dependency on the user input.
[Long system instructions - 200 tokens]
[Very long document for reference - 5000 tokens]
[User query - 32 tokens]
But the recommendation is to use this, where in this case you can only cache the first 200 tokens and need to prefill 5264 tokens every time the user submits a new query.
[Long system instructions - 200 tokens]
[User query - 32 tokens]
[Very long document for reference - 5000 tokens]
[Long system instructions - 200 tokens]
[User query - 32 tokens]As a ChatGPT user, I'm weirdly happy that it's not available there yet. I already have to make a conscious choice between - 4o (can search the web, use Canvas, evaluate Python server-side, generate images, but has no chain of thought) - o3-mini (web search, CoT, canvas, but no image generation) - o1 (CoT, maybe better than o3, but no canvas or web search and also no images) - Deep Research (very powerful, but I have…
Earlier quoted context omitted.
My experience is it often produces terrible diagrams. Things clearly overlap, lines make no sense. I'm not surprised as if you told me to layout a diagram in XML/YAML there would be obvious mistakes and layout issues. I'm not really certain a text output model can ever do well here.
I had a latex tikz diagram problem which sonnet 3.7 couldn't handle even after 10 attempts. Gemini 2.5 Pro solved it on the second try.
Could any one guess the reason as to why they didn't ship this in the chat UI?
Now you can imagine introducing a newer "type" of model like 4.1 that's better at following instructions and better at coding to bring a sort of overhead thats already too much with the given options.
OpenAI confirmed somewhere that they have already incorporated the enhancements made in 4.1 to 4o model in ChatGPT UI. I assume they would delegate to 4.1 model if the prompt doesn't require specific 4o capabilities.
Also one of the improvements made to 4.1 is following instructions. This type of thing is better suited for agentic use cases that are typically used in the form of an API.
Earlier quoted context omitted.
Yes, and they don't make snapshots for chatgpt-4o-latest, but they made them for GPT 4.1, that's why 4.1 is only useful for API, since their ChatGPT product already has the better model.
Okay so is GPT 4.1 literally just the current chatpt-4o-latest or not?
Earlier quoted context omitted.
Yes, confirmed by citing Aider benchmarks: https://openai.com/index/gpt-4-1/ Which means that these models are _absolutely_ not SOTA, and Gemini 2.5 pro is much better, and Sonnet is better, and even R1 is better. Sorry Sam, you are losing the game.
Aren’t all of these reasoning models? Won’t the reasoning models of openAI benchmarked against these be a test of if Sam is losing?
Earlier quoted context omitted.
Usually when we’re doing it in practice there’s _somewhat_ more awareness of the mechanics than just throwing random obstructions in and hoping for the best.
LLMs are still very young. We'll get there in time. I don't see how it's any different than optimizing for new CPU/GPU architectures other than the fact that the latter is now a decades-old practice.
Earlier quoted context omitted.
Wouldn’t it be the other way around? If the instructions are at the top the LV cache entries can be pre computed and cached. If they’re at the bottom the entries at the lower layers will have a dependency on the user input.
It's placing instructions AND user query at top and bottom. So if you have a prompt like this: [Long system instructions - 200 tokens] [Very long document for reference - 5000 tokens] [User query - 32 tokens] The key-values for first 5200 tokens can be cached and it's efficient to swap out the user query for a different one, you only need to prefill 32 tokens and generate output. But the recommendation is to use this…
Here's a summary of this Hacker News thread created by GPT-4.1 (the full sized model) when the conversation hit 164 comments: https://gist.github.com/simonw/93b2a67a54667ac46a247e7c5a2fe... I think it did very well - it's clearly good at instruction following. Total token cost: 11,758 input, 2,743 output = 4.546 cents. Same experiment run with GPT-4.1 mini: https://gist.github.com/simonw/325e6e5e63d449cc5394e92b8f2a3…
Hey Simon, I love how you generates these summaries and share them on every model release. Do you have a quick script that allows you to do that? Would love to take a look if possible :)
[1] https://llm.datasette.io/en/stable/plugins/directory.html#fr...