Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Moebius: 0.2B image inpainting model with 10B-level performance

hustvl.github.io

Post reply on HN
Print view View original

41–50 of 86 posts

Re: Moebius: 0.2B image inpainting model with 10B-level performance

#41

Post by xrd »

I did an inpainting project for a client a few years ago. They were trying to inpaint banner ads for concert promoters, and find a way to make it easy to produce a bunch of different sized ads for a variety of placements. I was tasked with inpainting Xmas themed ad for a few major singers.

The weirdest thing was when the inpainting tool added strange people to an image. This singer was all decked out in tinsel and red, and the inpainting model added a grumpy old man in a top hat. I don't recall clicking the "Add creepy old man" button.

At the time this was Stable Diffusion on the backend, run by a variety of model hosting services, Amazon being one. They all had different requirements for the input image and that made things really complex. For some the aspect ratio was impossible to meet, and it would fail if the banner was 200x60. For others, you had to resize it before input, which meant you were adding an image with poor resolution to start. Garbage in, garbage out.

All of this to say, there is a lot of preproduction that went into it, and the client never ended up using my attempts.

Re: Moebius: 0.2B image inpainting model with 10B-level performance

#43

Post by giancarlostoro »

xrd wrote: post #41

I did an inpainting project for a client a few years ago. They were trying to inpaint banner ads for concert promoters, and find a way to make it easy to produce a bunch of different sized ads for a variety of placements. I was tasked with inpainting Xmas themed ad for a few major singers. The weirdest thing was when the inpainting tool added strange people to an image. This singer was all decked out in tinsel and re…

> For others, you had to resize it before input, which meant you were adding an image with poor resolution to start.

Thats because small models like SD (Stable Diffusion) are trained on very specific resolutions, its the fancier models that are trained on higher quality, or more diverse sets of resolutions, and if you use a higher quality model to generate lower resolution images, what's actually happening is you're trimming a much bigger image and getting a chunk of it output, at least that's how it feels based on my many hours of experimenting. If I use major models and try to center a thing, I never see it in the center. :) My GPU can only handle so much.

Re: Moebius: 0.2B image inpainting model with 10B-level performance

#44

Post by krackers »

zerobees wrote: post #40
delis-thumbs-7e wrote: post #18

This is the useful AI stuf. There’s so many usecases this makes possible.

Right, and that's what I find frustrating. There are so many use cases where a local, purpose-built model that's dependably good at one thing would really make a difference. But no one is going to throw a billion dollars to give us amazing dust removal, flawless scene segmentation, etc. Instead, you're supposed to upload it to the cloud and ask a big, multimodal frontier model to maybe please do the thing you want an…

The highest return small local model for me has been the in-built OCR that macOS has. It has finally "solved" OCR by making high-quality results accessible to everyone. Yet the state of art outside the apple ecosystem seems to be tesseract (poor results), or extremely heavy VLMs.

Re: Moebius: 0.2B image inpainting model with 10B-level performance

#45

Post by matthewfcarlson »

pattilupone wrote: post #42

I want a version of this for manga (for translation). Right now I think the go-to lightweight inpainting model for anime and manga is LaMa which is several years old now and it feels like there is room for improvement.

I've been working on trying to outpaint an animated program for my son (Leapfrog Letter Factory if you're curious) and then upscale it. Doing so locally has been actually fairly difficult. I wonder if you could retrain or fine tune this model. They mention building an expert, I wonder if that expert could understand more about translating various characters.

Re: Moebius: 0.2B image inpainting model with 10B-level performance

#46

Post by dormento »

michaelfm1211 wrote: post #37

> The core insight of Moebius can be summarized in a single equation: Synergy × (Architecture + Distillation) = Shattering the "Impossible Triangle" of Low Parameters, Fast Inference, and High Quality Is it just me or is it weird seeing these clickbaity AI-generated taglines in an otherwise scientific work?

It IS weird, but it "converts" (ugh...), that's why they coming.

Apart from this, the text details amazing work. Congrats.

Re: Moebius: 0.2B image inpainting model with 10B-level performance

#47

Post by soperj »

michaelfm1211 wrote: post #37

> The core insight of Moebius can be summarized in a single equation: Synergy × (Architecture + Distillation) = Shattering the "Impossible Triangle" of Low Parameters, Fast Inference, and High Quality Is it just me or is it weird seeing these clickbaity AI-generated taglines in an otherwise scientific work?

After "In Good Company" i can't hear (or see) the word Synergy without cringing.

Re: Moebius: 0.2B image inpainting model with 10B-level performance

#48

Post by vunderba »

giancarlostoro wrote: post #43
xrd wrote: post #41

I did an inpainting project for a client a few years ago. They were trying to inpaint banner ads for concert promoters, and find a way to make it easy to produce a bunch of different sized ads for a variety of placements. I was tasked with inpainting Xmas themed ad for a few major singers. The weirdest thing was when the inpainting tool added strange people to an image. This singer was all decked out in tinsel and re…

> For others, you had to resize it before input, which meant you were adding an image with poor resolution to start. Thats because small models like SD (Stable Diffusion) are trained on very specific resolutions, its the fancier models that are trained on higher quality, or more diverse sets of resolutions, and if you use a higher quality model to generate lower resolution images, what's actually happening is you're…

So traditionally, the way you’d do this (and why some UIs like automatic1111 let you configure inpainting so flexibly) is that you didn’t have to shrink the entire image.

The general idea was: you mask the area you want changed, and the model inpaints that region at full resolution. The advantage of masking, compared to plain img2img, is that you’re not sending the entire picture to the model.

With the classic setups like SD 1.5 and SDXL, you’d effectively inpaint at full resolution: take the masked area from a larger image, scale just that region to the model’s native resolution, process it at the full ~1 megapixel then scale it back and composite it into the original. This lets you add MORE detail.

Unfortunately if the OP is using hosted SD models, they might not have that granular control and thus would suffer pretty bad quality loss.

Re: Moebius: 0.2B image inpainting model with 10B-level performance

#50

Post by kevin_thibedeau »

michaelfm1211 wrote: post #37

> The core insight of Moebius can be summarized in a single equation: Synergy × (Architecture + Distillation) = Shattering the "Impossible Triangle" of Low Parameters, Fast Inference, and High Quality Is it just me or is it weird seeing these clickbaity AI-generated taglines in an otherwise scientific work?

It signals a paradigm shift in vacuous prose.
Post reply on HN