411–420 of 475 posts
Good improvements from 1.1 and 1.2[0], but when I tested 1.3 it was very slow (through openrouter).
[0]: https://aibenchy.com/compare/meta-muse-spark-1-3-high/meta-m...
Does anyone know what the license for this model is? Specifically any word on restrictions about what it can be used for?
Nice, Muse Spark is so good and keeps improving, but it's still not the best choice for any use-case. The Sol models are in their own league currently in terms of cost/speed/performance. Good improvements from 1.1 and 1.2[0], but when I tested 1.3 it was very slow (through openrouter). [0]: https://aibenchy.com/compare/meta-muse-spark-1-3-high/meta-m...
Earlier quoted context omitted.
The more generic your prompt, the more generic the response. It's a regression to the "mean" of the training data aka GIGO for AI. It's like when you ask your average person off the street to draw a house - it'll almost always be square with a triangle roof, one door, and two windows. In the pelican/bike example, it's probably a bit of a self-perpetuating snowball too. If the earliest examples were bike left-to-right…
Well, all the LLMs are being trained on previous pelicans, so they look the same.
llm -m meta-ai/muse-spark-1.3 "Generate an SVG of a pelican riding a bicycle" https://tools.simonwillison.net/markdown-svg-renderer?url=ht... 4.2266 cents, 38 seconds. For comparison here's Muse Spark 1.2, which animated it without me asking it to: https://tools.simonwillison.net/markdown-svg-renderer?url=ht... The 1.3 one is definitely better - better bicycle frame, better wing, better pelican hat. UPDATE: Here's an…
Earlier quoted context omitted.
Not to mention, this is hyper competitive against even Chinese providers given its multi-modal support. Muse Spark 1.3 supports Text, Image, Video, File, Audio inputs. We've only started to see models from China include image and video inputs recently.
Muse Spark may be competitive in capabilities but it’s not for serious works since Meta trains on your prompts so no ZDR, in contrast Chinese provider like Z.AI promises ZDR which is more attractive to big corps.
I do not trust any provider, US or Chinese when they say they will not train on my data. I still use these services, but I am under no illusion that any of these people are trustworthy bunch.
Earlier quoted context omitted.
We should just consider the pelican bench as saturated and mostly meaningless.
Someone tested this, and it doesn't look to be saturated. https://dylancastillo.co/posts/pelicanmaxxing.html Simon made I think a very good argument for why it's still useful, if not the most robust benchmark in the world. https://simonwillison.net/2026/Jul/16/kimi-k3/
They could still pelicanmaxxing but the RL for "pelican riding a bicycle" does incidentally improve " ".
Or they could've predicted someone would check if they're pelicanmaxxing or the benchmark would switch eventually, so they preemptively RL'd a mixture of animals and vehicles.
llm -m meta-ai/muse-spark-1.3 "Generate an SVG of a pelican riding a bicycle" https://tools.simonwillison.net/markdown-svg-renderer?url=ht... 4.2266 cents, 38 seconds. For comparison here's Muse Spark 1.2, which animated it without me asking it to: https://tools.simonwillison.net/markdown-svg-renderer?url=ht... The 1.3 one is definitely better - better bicycle frame, better wing, better pelican hat. UPDATE: Here's an…
Earlier quoted context omitted.
It's really interesting, isn't it? They almost always cycle from left to right - but I have had a few which cycle in the other direction. The 2D / flat ground feels reasonable for a SVG, which implies a vector illustration.
It's my impression that it's common in western culture, where text is read left to right, and timelines are visualized as going from left to right, to also animate things going from left to right, since westerners thus have an instinct that "right = forward", so it "feels right" (familiar). I wonder to which degree this is reflected in the training data? And if you'd be more likely to get left-facing pelicans if you…
She told me that "left to right" denoted progression in the story, "right to left" told the viewer the subject was "exiting" the current scene.
She didn't go into the details of WHY, and I probably didn't probe deeper, but it stuck with me, and I notice it all the time in film and television.