Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Moebius: 0.2B image inpainting model with 10B-level performance

hustvl.github.io

Post reply on HN
Print view View original

51–60 of 86 posts

Re: Moebius: 0.2B image inpainting model with 10B-level performance

#51

Post by giancarlostoro »

vunderba wrote: post #48
giancarlostoro wrote: post #43

Earlier quoted context omitted.

> For others, you had to resize it before input, which meant you were adding an image with poor resolution to start. Thats because small models like SD (Stable Diffusion) are trained on very specific resolutions, its the fancier models that are trained on higher quality, or more diverse sets of resolutions, and if you use a higher quality model to generate lower resolution images, what's actually happening is you're…

So traditionally, the way you’d do this (and why some UIs like automatic1111 let you configure inpainting so flexibly) is that you didn’t have to shrink the entire image. The general idea was: you mask the area you want changed, and the model inpaints that region at full resolution. The advantage of masking, compared to plain img2img, is that you’re not sending the entire picture to the model. With the classic setups…

I was kind of speaking more in general I realized, not just strictly inpainting, but yeah that makes sense, though I've had inpainting also limited by the image being too big for my GPU to handle as well. I may be using it incorrectly though, not really experimented with much of that in a while, maybe when I get a newer gaming rig.

Re: Moebius: 0.2B image inpainting model with 10B-level performance

#52

Post by vunderba »

giancarlostoro wrote: post #51
vunderba wrote: post #48

Earlier quoted context omitted.

So traditionally, the way you’d do this (and why some UIs like automatic1111 let you configure inpainting so flexibly) is that you didn’t have to shrink the entire image. The general idea was: you mask the area you want changed, and the model inpaints that region at full resolution. The advantage of masking, compared to plain img2img, is that you’re not sending the entire picture to the model. With the classic setups…

I was kind of speaking more in general I realized, not just strictly inpainting, but yeah that makes sense, though I've had inpainting also limited by the image being too big for my GPU to handle as well. I may be using it incorrectly though, not really experimented with much of that in a while, maybe when I get a newer gaming rig.

Yeah, the landscape also changes a lot as well. It’s just really hard to keep up with everything. Especially if you’re using it casually because some of the UI wrappers (the Gradio-based ones) have more obscure knobs and dials than a TI‑82 calculator.

This is the image I always think of when first introducing someone to ComfyUI or even Automatic1111.

https://imgur.com/a/G0Xlznj

Re: Moebius: 0.2B image inpainting model with 10B-level performance

#53

Post by teroshan »

TeMPOraL wrote: post #16
teroshan wrote: post #11

Earlier quoted context omitted.

Scared for the same reason I found last year's 'Ghibli filter' craze upsetting, I would have personally hated to have seen this artist's legacy used for promoting AI image generation.

In case that happened then the rest of the world would probably appreciate the art, and a subset of it, the artist (and even a small subset of ~whole Internet-connected population is a lot of people ). Some silver lining, perhaps.

Perhaps.

I like the idea that a piece of art, in addition of ultimately ending up as pixels on my screen, is also a window into a world that has been dreamt up by real human imagination, driven by their hopes and fears.

Semiconductors based generation may give me the first part, but not the second.

I'm speaking for myself here, I agree with your point though.

Re: Moebius: 0.2B image inpainting model with 10B-level performance

#54

Post by TeMPOraL »

epolanski wrote: post #36
TeMPOraL wrote: post #17

Earlier quoted context omitted.

Awnings, if I understand correctly (I just learned this word right now), are purely additive attachments to structure exteriors - so perhaps they wouldn't necessarily need a full inpainting model? Wouldn't it be enough to estimate an affine transform for a quad and blend the image of awning directly (and the same with shadow map to fake shade)? Is classical photogrammetry up to such task these days?

I have no idea but I think you might be onto something. So you're saying that, if I can calculate from the picture the position (height, inclination and such), and I can render the model (should be doable) for that height and angle, my best course of action could be to combine original + render and only at the end use a visual model? That could be interesting.

More-less, yes. I was actually thinking about taking a high-resolution rendering of an awning directly facing the camera, and transforming that quad onto the user image - which requires computing the transformation matrix that would right the user image so the building is level and directly facing the camera, and then applying the inverse of that to the quad with your rendering - but I now realize I assumed user photos would mostly be nearly straight images of the building, not at large, odd angles. For general case, you'd need a 3D model (even if approximate), and apply the inverse transform to that, and then render it on top.

This idea rests on the assumption that my understanding of what "awnings" are is correct and matches your project, i.e. additive structures. In that case, your primary problem is adding pixels on top of the user image. Additive modifications are easy to pull off. Inpainting seems like overkill here; it's something that shines when you need to poke holes or replace some of the aspects of the original that is not covered by the part you're adding.

OTOH, it might still be that inpainting is your best bet for operational reasons - additive modification itself may not be a problem, but fixing lighting and shadows might, and current image generations models should handle this in stride.

(I say should because that's my expectation, but I never tested any of the current models on for ability to fix shadows that cover areas similar to the targeted modification, but lay beyond it. It might be that you'll still need a model and a transform estimate just to generate a shadow map as a hint for the model where it needs to act and how.)

Re: Moebius: 0.2B image inpainting model with 10B-level performance

#55

Post by IvanK_net »

james2doyle wrote: post #26
K0IN wrote: post #6

Earlier quoted context omitted.

with this size we could have a interaactive web demo.

Like this? https://huggingface.co/spaces/multimodalart/Moebius

Were you able to make it work? It never works in my case.

Re: Moebius: 0.2B image inpainting model with 10B-level performance

#56

Post by Jackson__ »

michaelfm1211 wrote: post #37

> The core insight of Moebius can be summarized in a single equation: Synergy × (Architecture + Distillation) = Shattering the "Impossible Triangle" of Low Parameters, Fast Inference, and High Quality Is it just me or is it weird seeing these clickbaity AI-generated taglines in an otherwise scientific work?

Judging by the performance of the shown examples, the quality is closer to pre-2022 Photoshop content aware fill than actual 10B models.

I think it is safe to say this is pretty far from a "scientific" work.

Re: Moebius: 0.2B image inpainting model with 10B-level performance

#57

Post by amelius »

lifthrasiir wrote: post #38

Tried a bit, and while it is very impressive for 0.2B model it would be very hard to convince me that this matches with 10B models. It did work reasonably well with natural images but inpainted regions were visibly smoother than surroundings, and performed very badly on novel objects. It is also limited to 512x512 output, which limits its practical usefulness.

Do you think the provided examples are representative of its performance, or do you think they were cherry picked?

Re: Moebius: 0.2B image inpainting model with 10B-level performance

#58

Post by lifthrasiir »

amelius wrote: post #57
lifthrasiir wrote: post #38

Tried a bit, and while it is very impressive for 0.2B model it would be very hard to convince me that this matches with 10B models. It did work reasonably well with natural images but inpainted regions were visibly smoother than surroundings, and performed very badly on novel objects. It is also limited to 512x512 output, which limits its practical usefulness.

Do you think the provided examples are representative of its performance, or do you think they were cherry picked?

Given its limited output dimension it's hard to tell. I haven't exactly tested fine-tuned variants but I think they would work well under certain situations. After all, some (possibly cherry-picked) examples still exhibit similar problems when you inspect them in detail.

Re: Moebius: 0.2B image inpainting model with 10B-level performance

#59

Post by hex4def6 »

james2doyle wrote: post #29

There are some demo spaces using this. This one seems the best (paint your own mask) but it failed on all the images I tried: https://huggingface.co/spaces/multimodalart/Moebius

I've been playing around, got it to work, although quality was a bit crappy. Still playing around with the settings that get exposed, but you're welcome to look at : https://huggingface.co/spaces/jonatei/MoebiusDemo

Note that I'm actively messing with it, so it may break for short periods of time :)

It's also running on the free CPU, so it's like 80 seconds per image...

Re: Moebius: 0.2B image inpainting model with 10B-level performance

#60

Post by stusmall »

inigyou wrote: post #32
stusmall wrote: post #24

Earlier quoted context omitted.

I think 3? I feel like that's often enough. Sometimes it's nice to do a quick dumb ass gag on a whim. If I am anything I am a man who loves a dumb ass gag.

Good on you. I've laughed at many dumbass gags but I've only been a passive consumer of them.

Become the dumbass change you want to see in the world
Post reply on HN