Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Arbitrary-Scale Super-Resolution with Neural Heat Fields

therasr.github.io

Post reply on HN
Print view View original

51–59 of 59 posts

Re: Arbitrary-Scale Super-Resolution with Neural Heat Fields

#51

Post by 0x12A »

karmakaze wrote: post #39
nandometzger wrote: post #18

Earlier quoted context omitted.

Try it yourself. Here is the demo: https://huggingface.co/spaces/prs-eth/thera

Tried it on this image[0] and it was blurry while still being pixelated. [0] https://en.wikipedia.org/wiki/Wing_Commander:_Privateer#/med...

That's an interesting example! Perhaps too out-of-distribution, though. For fair comparison with other methods, we used the DIV2K training set in our paper, which only comprises 800 images. Would be cool to train a version on a much bigger set, potentially including images similar to what you tried :)

Re: Arbitrary-Scale Super-Resolution with Neural Heat Fields

#52

Post by 0x12A »

earthnail wrote: post #41

@0x12A what’s the difference between this version and v1 of the paper from November 2023?

Hi, this is a complete rework, though the core idea remains the same. Results are now much better due to improved engineering, and we compare to recent SOTA methods up until 2025. Also we have some new experiments and worked a lot on figures and presentation :)

Re: Arbitrary-Scale Super-Resolution with Neural Heat Fields

#53

Post by grumbel »

flerchin wrote: post #10

I'd like to see the results in something like Wing Commander Privateer.

As other have mentioned, this models just puts emphasis on pixels and compression artifacts, so it's of not much use for improving old or low quality images.

I tried doing some pixelart->HD conversion with Gemini2.0Flash instead and the results look quite promising:

* https://imgur.com/a/t9F94F1

The images are however all over the place, as it doesn't seem to stick very close to the prompt. Trying to fine tune the image with further chatting often leads to overexposed looking pictures.

All the results are done with prompts along the lines of "here is a pixelart image convert it into a photo" or some variation there of. No img2img, LoRA or anything here, all plain Gemini chat.

Re: Arbitrary-Scale Super-Resolution with Neural Heat Fields

#54

Post by [deleted] »

jiggawatts wrote: post #2

The learned frequency banks reminded me of a notion I had: Instead of learning upscaling or image generation in pixel space , why not reuse the decades of effort that has gone into lossy image compression by generating output in a psychovisually optimal space? Perhaps frequency space (discrete cosine transform) with a perceptually uniform color space like UCS. This would allow models to be optimised so that they spen…

[deleted]

Re: Arbitrary-Scale Super-Resolution with Neural Heat Fields

#55

Post by orbital-decay »

mturnshek wrote: post #4
jiggawatts wrote: post #2

The learned frequency banks reminded me of a notion I had: Instead of learning upscaling or image generation in pixel space , why not reuse the decades of effort that has gone into lossy image compression by generating output in a psychovisually optimal space? Perhaps frequency space (discrete cosine transform) with a perceptually uniform color space like UCS. This would allow models to be optimised so that they spen…

You may already know this, but image generators like Stable Diffusion and Flux already do this in the form of “latent diffusion”. Rather than operate on pixel space directly, they learn to operate on images that have been encoded by a VAE (latents). To generate an image with them, you run the reverse diffusion (actually flow in the case of flux) process they’ve learned and then decode the result using the VAE. These…

>You may already know this, but image generators like Stable Diffusion and Flux already do this in the form of “latent diffusion”.

They... don't. Latents don't meaningfully represent human perception, they represent correlations in the dataset. Parent is talking about the function aligned with actual measured human perception (UCS is an example of that). Whether it's a good idea, and how trivial it is for the model to fit this function automatically, is another question.

Re: Arbitrary-Scale Super-Resolution with Neural Heat Fields

#56

Post by flerchin »

nandometzger wrote: post #18
flerchin wrote: post #10

I'd like to see the results in something like Wing Commander Privateer.

Try it yourself. Here is the demo: https://huggingface.co/spaces/prs-eth/thera

I tried it on these, and the results were really great!

https://www.wcnews.com/chatzone/threads/all-your-base-s-with...

Re: Arbitrary-Scale Super-Resolution with Neural Heat Fields

#57

Post by numba888 »

LoganDark wrote: post #29
i5heu wrote: post #28

Very good work! Sadly this model really does not like nosy images that have codec compression artifacts, at least with my few test images.

I wonder if there is a de-artifacting model out there.

I did it some time ago. It works at low levels. Doing it in hard cases is non-trivial but possible. It's difficult to monetize that's why nobody is doing it seriously.

Re: Arbitrary-Scale Super-Resolution with Neural Heat Fields

#58

Post by dimatura »

LoganDark wrote: post #29
i5heu wrote: post #28

Very good work! Sadly this model really does not like nosy images that have codec compression artifacts, at least with my few test images.

I wonder if there is a de-artifacting model out there.

Yes, there are plenty of them. Not sure what the SOTA is, though. Similar to super-resolution, it is relatively simple to create a nearly-infinite dataset for these; pick a clean image, then introduce JPG artifacts. Then train a model to invert the process.

Re: Arbitrary-Scale Super-Resolution with Neural Heat Fields

#59

Post by LoganDark »

dimatura wrote: post #58
LoganDark wrote: post #29

Earlier quoted context omitted.

I wonder if there is a de-artifacting model out there.

Yes, there are plenty of them. Not sure what the SOTA is, though. Similar to super-resolution, it is relatively simple to create a nearly-infinite dataset for these; pick a clean image, then introduce JPG artifacts. Then train a model to invert the process.

> Similar to super-resolution, it is relatively simple to create a nearly-infinite dataset for these; pick a clean image, then introduce JPG artifacts. Then train a model to invert the process.

Yep, exactly what I was thinking. The thing is it's hard to find enough clean images!

Post reply on HN