201–210 of 692 posts
Earlier quoted context omitted.
Maybe the margins are just very large for Google because they predict so much demand for 3.5?
This combined with locally runnable models getting pretty good recently (e.g. Qwen 3.6) tells me that it's time to seriously consider local dev setup again
The pelican is a lot : https://github.com/simonw/llm-gemini/issues/133#issuecomment... Not a great bicycle though, it forgot the bar between the pedals and the back wheel and weirdly tangled the other bars. Expensive too - that pelican cost 13 cents: https://www.llm-prices.com/#it=11&ot=14403&sel=gemini-3.5-fl...
funny that when I try the same prompt, gemini generates an image, not an SVG. something is not right.
Is there a good benchmark tracking hallucinations? The models are all incredibly good now, even the open ones, and my hope is that the rate of hallucinations is something that's falling off in concert with larger and larger context lengths.
People complain about them incessantly, but I can almost never get people to actually post receipts. Every provider allows sharing chats, and anyone can share a prompt that reliably produces hallucinations. More often than not, people are using images in responses that go awry. Which is fair, the models are sold as multi-modal, but image analyses is still at gpt-4.0 text-analyses levels. Also knowledge cutoff issues,…
Knowledge cutoff: January 2025 Latest update: May 2026 I have a very bad feeling about this lag.
With strong tool use, it maybe doesn't even matter that the models are using older data. They can search for updated information. Though most models currently don't, without a little nudge in that direction.
Also, I believe the Qwen 3 series are all based on the same base model, with just fine-tuning/post-training to improve them on various metrics. Maybe everything in the Gemini 3 series is the same, and maybe they're concurrently training the Gemini 4 base model with updated knowledge as we speak.
Earlier quoted context omitted.
Deepseek had another moment a few weeks ago. V4 isn't far behind the US frontier, and so far its flash variant seems a very reliable coder and costs a pittance.
Deepseek V4 (not flash) trippled in price too by the way (from Deepseek). Get used to this pattern. This is what you get for relying on the generosity of billionaires. Keep offshoring your thinking ability to a machine and let me know how competitive you. Hint, you wont be. There's nothing special about being able to use an LLM.
Per million input/output tokens: Gemini 2.5 flash: $0.30/$2.50 Gemini 3.0 flash preview: $0.50/$3.00 Gemini 3.5 flash: $1.50/$9.00 Interesting pricing direction. I don't think we have ever seen a 3x price increase for in the immediate next same-sized model (and lol @ 3 only ever getting a preview). 3.5 flash costs similar to Gemini 2.5 pro which was $1.25/$10
And they are using this to power search answers?
> Create animated SVG of a frog on a boat rowing through jungle river. Single page self contained HTML page with SVG 3.5 Flash: Thinking Medium - 7516 tokens https://gistpreview.github.io/?5c9858fd2057e678b55d563d9bff0... 3.5 Flash: Thinking High - 7280 tokens https://gistpreview.github.io/?1cab3d70064349d08cf5952cdc165... 3.1 Pro - 28,258 tokens https://gistpreview.github.io/?6bf3da2f80487608b9525bce53018... Though…
Can you try with a more complex story such as "three little pigs"? I tried but it created a storybook instead of the SVG animation. I am looking to partially imitate Godogen [1][2] which is really great, even for animations. [1] https://github.com/htdt/godogen [2] https://drive.google.com/file/d/1ozZmWcSwieZQG0muYjbj7Xjhhlz...
Now think, plan how to tell this story in a cartoon, make scene outline and then generate SVG animation story for "Three Little Pigs" in self contained HTML page. Just single animation no control buttons.
Actual results for models, one shot:
Gemini 3.5 Flash - Three Little Pigs - 9,050 tokens:
https://gistpreview.github.io/?ed9faa53604035005cae86c63c766...
Gemini 3.1 Pro - Three Little Pigs - 24,272 tokens:
https://gistpreview.github.io/?f506bbfd9b4459c8cd55d89605af8...
Gemini 3 Flash - Three Little Pigs - 5,350 tokens:
https://gistpreview.github.io/?f58eff069cf916031c97d560b0e35...
Gemma 4 31B IT - Three Little Pigs - 5,494 tokens:
https://gistpreview.github.io/?a3aa75abbe8fd7818b73f6fa55ee6...
Gemma 4 26B A4B IT - Three Iittle Pigs - 6,375 tokens:
https://gistpreview.github.io/?1e631caebeb54f9f0cd6d0e3d4d5e...
> Create animated SVG of a frog on a boat rowing through jungle river. Single page self contained HTML page with SVG 3.5 Flash: Thinking Medium - 7516 tokens https://gistpreview.github.io/?5c9858fd2057e678b55d563d9bff0... 3.5 Flash: Thinking High - 7280 tokens https://gistpreview.github.io/?1cab3d70064349d08cf5952cdc165... 3.1 Pro - 28,258 tokens https://gistpreview.github.io/?6bf3da2f80487608b9525bce53018... Though…
hesamation/Qwen3.6-35B-A3B-Claude-4.6-Opus-Reasoning-Distilled-GGUF @ Q6_K 8112 tokens @ 52.97 TPS, 0.85s TTFT https://gistpreview.github.io/?7bdefff99aca89d1bc12405323bd4... Full session: https://gist.github.com/abtinf/7bdefff99aca89d1bc12405323bd4... Generated with LM Studio on a Macbook Pro M2 Max https://huggingface.co/hesamation/Qwen3.6-35B-A3B-Claude-4.6...
Earlier quoted context omitted.
People complain about them incessantly, but I can almost never get people to actually post receipts. Every provider allows sharing chats, and anyone can share a prompt that reliably produces hallucinations. More often than not, people are using images in responses that go awry. Which is fair, the models are sold as multi-modal, but image analyses is still at gpt-4.0 text-analyses levels. Also knowledge cutoff issues,…
"People complain about them incessantly, but I can almost never get people to actually post receipts." ...my chats are all pretty long and involve personal conversations, or I've deleted them. It's a lot to ask for someone to post receipts. The number of complaints is enough data. No matter how big the model is there will be edge cases where it has no data or is out of date. In these cases it just makes stuff up. You…