This is awesome, and thanks so much for the deep dive into process!! One thing I would suggest is to also post-process the pixel art with something like this tool to have it be even sharper. The details fall off as you get closer, but running this over larger patch areas may really drive the pixel art feel. https://jenissimo.itch.io/unfaker
Oh wow this is great, I threw a few of these tricks at it and never really felt like I nailed it... Might have to revisit after playing around with this more
That quote seriously rubs me the wrong way. "Dragging little boxes around" in a DAW is creative, it constitutes the entire process of composing electronic music. You are notating what notes to play, when and for how long they play, what instrument plays them, and any modifications to the default sound of that instrument. Is writing sheet music tedious? Sure, it can be, when the speed of notating by hand can't keep up…
I'd say that deciding where a transient should go" is creative, manually aligning 15 other tracks over and over again is not (not to mention having to do it in both the DAW and melodyne)... I agree that "push button get image" AI generation is at best a bit cheap, at worst deeply boring. Art is resistance in a medium - but at what point is that resistance just masochism? George Perec took this idea to the max when he…
To be clear, as I said in another reply downthread, I think this particular project is creative, although creative in a fundamentally different way that does not replace existing creative expression. I also don't object to people doing "push button get image" for entertainment (although I do object to it being spammed all over the internet in spaces meant for human art and drowning out people who put effort into what they create, because 10,000 images can be generated in the time it takes to draw a single one). But "push button get image" is not making things yourself. I would give you credit for creating this because you put effort into fine-tuning a model and a bespoke pipeline to make this work at scale, but this project is exceptional and non-representative among generative AI usage, and "push button get image" does not have enough human decision-making involved for the human to really have any claim to have made the thing that gets generated. That is not creativity, and it is not capable of replacing existing expressions of creativity, which you've asserted multiple times in the article and thread. By all means push button and get image for as long as it entertains you, but don't pretend it is something it isn't.
Basically, it's not pixel art at all. It's very cool and I don't mind the use of AI at all but I think calling it pixel art is just very misleading. It's closer to a filter but not quite that either.
It's pixel art, just not the types of pixels most people want in pixel art.
Pixel art has a certain connotation (probably more accurately referred to as 8-bit or 16-bit).
Otherwise every digital image could be classified as pixel art.
Very impressive result! are you taking requests for the next ones? SF :D Tokyo :D Paris :D Milan :D Rome :D Sydney :D Oh man...
Really want to do SF next. Maybe the next gen of models will be reliable enough to automate it but this took WAY too much manual labor for a working man. I’ll get the code up soon if people wanna fork it!
I'd absolutely love to play with this. one idea I had is to train another model to create bitmaps of sidewalks and roads and add a simulation for pedestrians and cars. day/night cycle would also be so cool!
So, wait: this is just based on taking the 40 best/most consistent Nano Banana outputs for a prompt to do pixel-art versions of isometric map tiles? That's all it takes to finetune Qwen to reliably generate tiles in exactly the same style? Also, does someone have an intuition for how the "masking" process worked here to generate seamless tiles? I sort of grok it but not totally.
I think the core idea in "masking" is to provide adjacent pixel art tiles as part of the input when rendering a new tile from photo reference. So part of the input is literal boundary conditions on the output for the new tile. Reference image from the article: https://cannoneyed.com/img/projects/isometric-nyc/training_d... You have to zoom in, but here the inputs on the left are mixed pixel art / photo textures. The…
Thanks for this - I was confused as well. Makes perfect sense now.
> This project is far from perfect, but without generative models, it couldn’t exist. There’s simply no way to do this much work on your own 100 people built this in 1964: https://queensmuseum.org/exhibition/panorama-of-the-city-of-... One person built this in the 21st century: https://gothamist.com/arts-entertainment/truckers-viral-scal... AI certainly let you do it much faster , but it’s wrong to write off doing so…
It feels like whether or not it's possible to make a model of new york misses the point. This kind of project could just as easily scale up to be an isometric map of the entire world, every city, every forest, every mountain hut, and update continuously as new buildings are built and cities grow. Expensive yes, for now- but the speed and scale to make and maintain such representations will absolutely be outside the realm of what was previously possible. Not just maps of the world, maps of human knowledge, history, to such detail that would entail not just tens of thousands of hours but billions or trillions of hours if done by hand.
"map of NYC" does not include Staten Island. That's how we like it
I just couldn't justify spending dozens more hours for Staten Island...
What do you estimate the cost would be to have each tile hand drawn by an artist?
I don't think there are enough artists in the world to achieve this in a reasonable amount of time (1-5 years) and you're probably looking at a $10M cost?
Part of me wonders if you put a kickstarter together if you could raise the funds to have it hand drawn but no way the very artists you hire wouldn't be tempted to use AI themselves.
So, wait: this is just based on taking the 40 best/most consistent Nano Banana outputs for a prompt to do pixel-art versions of isometric map tiles? That's all it takes to finetune Qwen to reliably generate tiles in exactly the same style? Also, does someone have an intuition for how the "masking" process worked here to generate seamless tiles? I sort of grok it but not totally.
So you don't grok it. You understand it, but don't grok it. Respect the martian :)