251–260 of 697 posts
The flash models, for coding are reckless in my experience. I have a Ultimate subscription, get good quota, but still use Opus 4.6 as it's much more reliable if you manage the context window carefully.
If by reckless you mean commit, push, deploy without me asking it to, the I agree!
It is more expensive per task than 5.6-sol high: https://artificialanalysis.ai/models/gemini-3-8-flash#price-...
Is there any reason to even use 3.1 Pro now?
Earlier quoted context omitted.
One thing in your comment surprised me: "when a thing opens and closes" Why you would rely on the model's weights to know opening hours, instead of having the model call a web search tool to verify it on the official site?
I wasn't trying to be precise originally, I just tried to fit activities into "morning / evening" buckets. I did the whole itinerary with Opus first, but when I gave it to Gemini 3.7 Flash to review, it started correcting it with "this place will close 5PM" or "this place is closed for good". It was right on every nit, so it was surprising how well the model knows these things. If I ever release this I'll probably ne…
tools=[{"type": "google_search"}]
Pelicans (thinking effort high, medium, low): https://tools.simonwillison.net/markdown-svg-renderer?url=ht... - high cost 8.9742 cents Here are the 3.7 pelicans for comparison: https://tools.simonwillison.net/markdown-svg-renderer.html?u... - high cost 8.4387 cents (I think thinking level low is a regression on 3.8 compared to 3.7.)
I mean no offense but these pelicans are a bit tiresome and a very meaningless benchmark. There's no real difference between any of these svgs across models and model versions anymore.
Earlier quoted context omitted.
One thing in your comment surprised me: "when a thing opens and closes" Why you would rely on the model's weights to know opening hours, instead of having the model call a web search tool to verify it on the official site?
Maybe the model does some tool calling on its own to figure out the times?
Not to rain on anyone's parade but I find it strange how excited and giddy people on HN get for any new X.X model releases. Pumping it straight to the top, clamoring to use it, check and compare benchmarks, bragging about it being your "daily driver"? Are you people truly this excited about this crap? I mean I guess if you work for Google or Anthropic or whatever I could see it??? Otherwise, are these just bot commen…
I'll wait for Astra and Grok 4.7 announcements but probably getting at least one Ultra subscription.
Since testing 3.7 on Pro for last two weeks I'm realizing just how long I'm waiting on other models. I've been multitasking to compensate but it's exhausting so I'd rather not.
The speed combined with the fact that this thing is really good at HTML JavaScript is pretty exciting. Here's what I got for 1.8 cents and 13 seconds from the prompt "make me a cool thing in html": https://gisthost.github.io/?6a77bc41a81718c6aaa10d4ab243c59f Transcript here (it was part of a chat): https://gist.github.com/simonw/b6149a49d327164d67d62c3d12992...
Mission accomplished. That's both cool and fast.
and probably a barely modified knock-off of some github project that it trained on