Where do these models even have this data to learn from. There must be massive computer use datasets? I have a hard time believing it’s emergent if it’s able to do something this good.
"Drawing" the Mona Lisa with GPT-5.6, Claude, Gemini, and Grok
61–70 of 110 posts
Re: "Drawing" the Mona Lisa with GPT-5.6, Claude, Gemini, and Grok
#62The most interesting result for me is that they apparently prompted the models to optimize for SSIM, but many of the models trend worse over time. I suppose because viewing the canvas always comes after drawing, and they didn't give "revert to previous" capability as part of the toolkit.
Which in turn kinda jives with my experience of using these models for code: to some extent they only seem to have a concept of "forward", which invariably leads to "write more code to fix previous problems created", rather than taking a step back and removing broken things entirely.
Re: "Drawing" the Mona Lisa with GPT-5.6, Claude, Gemini, and Grok
#63For capabilities reference:
I made a lower effort but similar scaffold for LLMs to do iterative drawing in Nov 2024, with Sonnet 3.5 as the artist: https://paritybits.me/llm-drawing-with-eyes-open/
Quite a difference.
Re: "Drawing" the Mona Lisa with GPT-5.6, Claude, Gemini, and Grok
#64Earlier quoted context omitted.
I do think they are going to stretch their lead in value if Anthropic doesn't wake up and stop YOLOing tokens. Kimi is an amazing achievement, but it has the same (or worse) kitchen sink approach as Fable. At work, even if Fable is technically better I much prefer Sol because it is so much faster and concise.
To me GPT is quite slow. I use it with max reasoning though but Fable at max as well
For me personally, there is no point in using more reasoning than medium or sometimes high. It just gets so much slower and expensive, meaning that i have to wait longer to evaluate the results and provide feedback or continue to the next task.
Re: "Drawing" the Mona Lisa with GPT-5.6, Claude, Gemini, and Grok
#65[flagged]
Re: "Drawing" the Mona Lisa with GPT-5.6, Claude, Gemini, and Grok
#66Shows about the same level of technical mastery that present days painters.
Re: "Drawing" the Mona Lisa with GPT-5.6, Claude, Gemini, and Grok
#67The difference in cost is pretty incredible.
He mentions they used quite different methods to get to the end-result. I'd love to know how much changing the prompt impacts things: like telling claude to only draw and blend, similar to sol, etc.
There's likely not training data for this and Claude seemed to spend a lot of time blending the background.
Re: "Drawing" the Mona Lisa with GPT-5.6, Claude, Gemini, and Grok
#68Weird choice of SSIM/RMSE. By feeding these back into the model, the agent is actively degrading its artistic output. This is only valid for the target reference I think?
Alternatively, I think much better results could be had by computing the cosine similarly using DINOv2 ONNX (@xenova/transformers).
My guess is that the results are greatly limited by relying on the current metrics.
Even better, attach toModelOutput directly to drawTool so it returns the rendered canvas image immediately upon drawing.
Re: "Drawing" the Mona Lisa with GPT-5.6, Claude, Gemini, and Grok
#69I'm missing some context here.
First of all, this is a marketing piece, made for engagement.
Second, in production, you would adjust your approach to the model's strengths and weaknesses. For example, if you found that a model does not respond to the tools you are offering, then you would find a way to make it more responsive. Also, we know that different models handle prompts and system prompts differently. So comparing several models from different providers against exactly the same prompt is quite a naive way of exploring the capabilities of those models or comparing them to each other. You may have just happened to “speak the language“ of one model better than that of another. Why would that be the model’s fault and not yours?
Third, none of these results mean anything if you do not run each process on each model several times to check for consistency. Your model might just have gotten lucky. This point is even more true when you're talking about a progressive task where one step builds on the previous one. The model might just have painted itself into a corner.
Re: "Drawing" the Mona Lisa with GPT-5.6, Claude, Gemini, and Grok
#70For capabilities reference: I made a lower effort but similar scaffold for LLMs to do iterative drawing in Nov 2024, with Sonnet 3.5 as the artist: https://paritybits.me/llm-drawing-with-eyes-open/ Quite a difference.
The way the LLM describes its own work at the end is hilarious.