Alright results are in! I've re-run all my editing based adherence related prompts through Nano Banana Pro. NB Pro managed to successfully pass SHRDLU, the M&M Van Halen test (as verified independently by Simon), and the Scorpio street test - all of which the original NB failed. Model results 1. Nano Banana Pro: 10 / 12 2. Seedream4: 9 / 12 3. Nano Banana: 7 / 12 4. Qwen Image Edit: 6 / 12 https://genai-showdown.spec…
I think Nano banana pro’s answer to the giraffe edit is far superior to the Seedream response, but you passed Seedream and failed NB pro.
Yeah I think that's a fair critique. It kind of looks like a bad cut-and-replace job (if you zoom in you can even see part of the neck is missing). I might give it some more attempts to see if it can do a better job. I agree that Seedream could definitely be called out as a fail since it might just be a trick of perspective.
Have you ever considered a “partial pass”? Perhaps it would be an easy cop out of making a decision if you had to choose something outside of pass/fail.
That's not a bad suggestion. I thought about adding a numerical score but it felt like it was bit overwhelming at the time. Maybe I should revisit it though in the form of:
No, this is squarely on the AI. A human would know what you mean without specific instructions.
Seems like you're making a judgment based on your own experience, but as another commenter pointed out, it was wrong. There are plenty of us out there who would confirm, because people are too flawed to trust. Humans double/triple check, especially under higher stakes conditions (surgery). Heck, humans are so flawed, they'll put the things in the wrong eye socket even knowing full well exactly where they should go -…
“People are too flawed to trust”? You’ve lost the plot. People are trusted to perform complex tasks every single minute of every single day, and they overwhelmingly perform those tasks with minimal errors.
Alright results are in! I've re-run all my editing based adherence related prompts through Nano Banana Pro. NB Pro managed to successfully pass SHRDLU, the M&M Van Halen test (as verified independently by Simon), and the Scorpio street test - all of which the original NB failed. Model results 1. Nano Banana Pro: 10 / 12 2. Seedream4: 9 / 12 3. Nano Banana: 7 / 12 4. Qwen Image Edit: 6 / 12 https://genai-showdown.spec…
thanks, I love your website. Are you planning to do NB Pro for the text-to-image benchmark too?
Outside the time frame of being able to edit my original reply, but I've finally re-run the Text-to-Image portion of the site through NB Pro.
Google has been stomping around like Godzilla this week, and this is the first time I decided to link my card to their AI studio. I had seen people saying that they gave up and went to another platform because it was "impossible to pay". I thought this was strange, but after trying to get a working API key for the past half hour, I see what they mean. Everything is set up, I see a message that says "You're using Paid…
There is an entire business opportunity in just building better user and developer frontends to Google's AI products. It's so incredibly frustrating.
Something I find weird about AI image generation models is that even though they no longer produce weird "artifacts" that give away that the fact that it was AI generated, you can still recognize that it's AI due to stylistic choices. Not all examples they gave were like this. The example they gave of the word "Typography" would have fooled me as human-made. The infographics stood out though. I would have immediately…
I think it's because they're all trained on the same data (everything they could possibly scrape from the open web). The models tend to learn some kind of distribution of what is most likely for a given prompt. It tends to produce things that are very average looking, very "likely", but as a result also predictable and unoriginal. If you want something that looks original, you have to come up with a more original pro…
An more original prompt wont fix things. Modern base models want to eliminate everything that puts their creators at risk, which is anything that is clearly made by someone else, more or less accurately reproducible. If you avoid decent representation of any artist style, or anything/anyone that is likely to go to court, you wont get the chance of an creative synthesis either.
2D animators can still feel safe about their job, I asked it to generate a sprite sheet animation by giving it the final frame of the animation (as a PNG file) and asking in detail what I wanted in the spritesheet, it just gave me mediocre results, I asked for 8 frames and it just repeated a bunch of poses just to reach that number instead of doing what a human would have done with the same request, meaning the in-be…
With local models you can use control net, which is simply speaking, the model trying to adhere to a given wireframe/openpose. Which is more likely to give you an stable result. I have no experience with it, just wanted to point out that there is tooling that is more advanced.
thanks, I love your website. Are you planning to do NB Pro for the text-to-image benchmark too?
Outside the time frame of being able to edit my original reply, but I've finally re-run the Text-to-Image portion of the site through NB Pro. Results gpt-image-1: 10 / 12 Nano Banana Pro: 9 / 12 Nano Banana: 8 / 12 It's worth mentioning that even though it only scored slightly better than the original NB, many of the images are significantly better looking. https://genai-showdown.specr.net?models=nb,nbp
thanks for the update. One small note: for the d20 test, NB Pro had duplications of 13 and 17 too, not just 19.
If it's just the API you're interested in, Fal.ai has put Nano-Banana-Pro up for both generative and editing. A great deal less annoying to sign up for them since they're a pretty generalized provider of lots of AI related models. https://fal.ai/models/fal-ai/nano-banana-pro
Is there a model on Fal.ai that would make it easy to sharpen blurry video footage? I have found some websites, but apparently they are mostly scammy.
The interesting tidbit here is SynthID. While a good first step, it doesn't solve the problem of AI generated content NOT having any kind of watermark. So we can prove that something WITH the ID is AI generated but we can't prove that something without one ISN'T AI generated. Like it would be nice if all photo and video generated by the big players would have some kind of standardized identifier on them - but now you…