Live data from Hacker News

Depixelizing Pixel Art

research.microsoft.com

121–130 of 164 posts

Re: Depixelizing Pixel Art

#122
post #66

Earlier quoted context omitted.

Am I crazy for preferring the one on the left (nearest-neighbour)? The other two really mess up certain details such as the point numbers, the stem of the fruits and pokey's eyes. They also tend to cause a very subtle distortion of the outlines of foreground sprites as they move over the background.

As interesting as these smoothing algorithms are, they can't add any information to the sprites that wasn't already there. Why make everything look smooth and plasticky when the original game was carefully hand-drawn, pixel by pixel?

I think there is a purpose for this in video, but I'm not sure it's an all or nothing game. Especially here. For starters, the gameplay looks smoother in nearest neighbor because your eyes defocus while playing. Hence, you need sharp pixel edges to dilettante and your eyes do the rest. In a still shot, that's not the case at all. You want focus and smoothness.

Interesting though...

Re: Depixelizing Pixel Art

#123
post #71

The issue with depixelizing pixel art is that you need an AI that can identify semantically what the features in the sprite are. When you look at Mario you know you're looking at a stout little man with a hat, you can identify his eyes, ears, nose, hair, hat, moustache, shirt, gloves, overalls and shoes. Given that knowledge, if I were to depixelize Mario's sprite, I'll know the rough shape of his shoes and where the…

I've been thinking about doing something like this. Compress images down to half resolution, then train a neural network to blow it back up and guess the pixels as accurately as possible.

Re: Depixelizing Pixel Art

#124
post #68
post #57

Earlier quoted context omitted.

As you say: It depends. E.g. the sword looks better with hq4x, because the result isn't rounded.

Look at Axe Battler 2 - if the sword isn't vertical, then the result is not superior. Unless, of course, you think the guy is wielding a sawtooth blade. Not to mention that muscles have a smoothly-graded tone under the 'ours' algorithm, whereas hq4x has an unlifelike blocky effect.

Honestly I think hq4x and "Ours" both look like ass for both of the Axe Battlers. Nearest neighbor is a better choice for either of them. EPX on either is acceptable.

Re: Depixelizing Pixel Art

#126
post #61
post #38

Earlier quoted context omitted.

> (rather than only available as 1x or 2x scale like on the Mac) It isn't continuous, but there are more than two scaling levels on OS X: https://blog.qt.digia.com/wp-content/uploads/2013/04/qtRetin... More details: https://blog.qt.digia.com/blog/2013/04/25/retina-display-sup...

Right, but as noted in the blog you link the application is still only exposed to 1x or 2x scale (where 1pt = 2px), and then everything is scaled back down. On Windows the app is exposed to the scale factor (for better or worse): https://blog.qt.digia.com/blog/2009/06/26/improving-support-...

Prior to retina display Macs, OS X implemented resolution independence as an arbitrary scale factor, with most of the UI elements being vector rendered. You could test it out with Quartz Debug. TextEdit, for example, scaled very nicely and remained crisp at arbitrary scale factors.

They didn't end up using this approach because the 2x mode is so simple, and will likely be the last scale factor really needed (especially with the additional scaling modes).

Re: Depixelizing Pixel Art

#127
I think the most interesting aspect of this is the internal shading. I haven't seen any other algorithm that manages that while maintaining sharp edges overall.

I imagine it would end up with pretty weird results though when a sprite was on top of a background that with similar enough colors to trigger the shading detection at its edges.

Re: Depixelizing Pixel Art

#128
post #43

I wish they would use this in Windows 8. As far as I can tell, their scaling algorithm for 4k displays is: scale up 2x using nearest neighbor, then randomly drop pixels to get back to 125% or 150%. The result is horrifyingly bad. Not just kind of bad, but unusably bad. (Adding insult to injury, they ClearType the text before scaling.)

While something similar would probably be better, this is heavily optimized for pixel artwork. Lots of curves, shading, and most importantly - limited colors. UI is generally much straighter, less shaded, with far more colors. Here's an example of UI that's scaled with potrace (a similar algorithm): http://i.imgur.com/jDq4M7e.png (left side is the converted side, right side is nearest-neighbor. Top is 3x, bottom is 2…

To be fair, that would look a little better with ClearType off.

Re: Depixelizing Pixel Art

#129
post #71

The issue with depixelizing pixel art is that you need an AI that can identify semantically what the features in the sprite are. When you look at Mario you know you're looking at a stout little man with a hat, you can identify his eyes, ears, nose, hair, hat, moustache, shirt, gloves, overalls and shoes. Given that knowledge, if I were to depixelize Mario's sprite, I'll know the rough shape of his shoes and where the…

I've been thinking about doing something like this. Compress images down to half resolution, then train a neural network to blow it back up and guess the pixels as accurately as possible.

I did something very much like this years ago (upscaling used to be an obsession of mine). The results were pretty disappointing.

But there's a lot of space to explore there in terms of choosing and designing the neural net, choosing the right training set, and figuring out an initial transform for data that is input to the neural net.

Re: Depixelizing Pixel Art

#130
post #16

I wonder why they all fall down on Peach and Peach 2? None of them look nearly as good (to me) as bicubic.

Super2xSaI looks better to me at 4x than bicubic, and EPX does an OK job but the others look pretty terrible.

I think the problem is that those sprites use multiple strategies for handling diagonal outlines. Those look fine when the image is blurred together, but when you try to detect the outlines and upscale them, the differences between techniques are exaggerated.

Post reply on HN