Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Meow – An Image File Format I made because PNGs and JPEGs suck for AI

github.com

Post reply on HN
Print view View original

61–70 of 84 posts

Re: Show HN: Meow – An Image File Format I made because PNGs and JPEGs suck for AI

#61

Post by mousethatroared »

b0a04gl wrote: post #52

using LSB to store structured metadata inside PNG is clever > survives format conversion, stays invisible to standard viewers, and doesn't break compression. but the space is tight. even at 1 bit per channel, that's just 3 bits per pixel on RGB. given that, how are you handling tradeoffs between spatial fidelity (like masks or edges) versus scalar data (like complexity scores)? is there a priority system, or does it…

3 bits per pixel is in the order of megabyte amounts of information.

640x480 a 3 bits -> 900kB.

Thats a lot. Too much. This is why we cant have nice things. Because people keep inventing things to destroy privacy.

Re: Show HN: Meow – An Image File Format I made because PNGs and JPEGs suck for AI

#62

Post by godelski »

michaelt wrote: post #18
RamblingCTO wrote: post #5

> it gets stripped way too easily when sharing that's not a bug, that's a (security) feature

If you're a journalist taking photos of eccentric semi-fugitive John McAfee - you want the location metadata removed when posting online, in case you forgot to remove it yourself. If you're a proud generative AI user, and you don't want anyone deceived by your images, you want the this-was-created-by-ai metadata retained.

That's called a watermark

Re: Show HN: Meow – An Image File Format I made because PNGs and JPEGs suck for AI

#63

Post by whoisyc »

DanHulton wrote: post #41

You have invented essentially an _incredible way_ to poison AI image datasets. Step 1: Create .meow images of vegetables, with "per-pixel metadata" instead encoded to represent human faces. Step 2: Get your images included in the data set of a generative image model. Step 3: Laugh uproariously as every image of a person has vaguely-to-profoundly vegetal features.

This assumes people training AI are going to put in the efforts to extract metadata from a poorly specified “format” with a barely coherent buzzword ridden README file. Realistically, they will just treat any .meow as opaque binary blobs and any png as regular png file.

Re: Show HN: Meow – An Image File Format I made because PNGs and JPEGs suck for AI

#64

Post by whoisyc »

fao_ wrote: post #7

> Instead of storing metadata alongside the image where it can be lost, MEOW ENCODES it directly inside the image pixels using LSB steganography That makes the data much more fragile than metadata fields, though? Any kind of image alteration or re-encoding (which almost all sites do to ensure better compression — discord, imgur, et al) is going to trash the metadata or make it utterly useless. I'll be honest, I don't…

> webp, jpeg 2000, HEIF, jpeg xl, farbfeld

I think you just illustrated how difficult it is to propose a new standard. Webp was not supported by many image related softwares (including the Adobe suite!) for years and earned a bad reputation, HEIF is also poorly supported, JPEG XL was removed from Chrome despite being developed by Google and not supported by any other browser AFAIK. Never heard of farbfeld before.

If the backing from Apple and Google was not enough to drive the adoption of an image format, I fail to see how this thing can go anywhere.

Re: Show HN: Meow – An Image File Format I made because PNGs and JPEGs suck for AI

#65

Post by vunderba »

bastawhiz wrote: post #56

Modifying the image in any way (cropping, resizing, etc) destroys the metadata. This is necessary in basically every application that interacts with any kind of model that uses images, either for token count reasons, file size reasons, model limits, etc. (Source: I work at a genai startup) At inference time, you don't control the inputs, so this is moot. At training time, you've already got lots of other metadata tha…

Was just coming here to say this. Most graphic editors can easily preserve EXIF/IPTC data across edits.

Without an entirely dedicated editor or postprocessing plugin, stenography gets destroyed on modification.

Re: Show HN: Meow – An Image File Format I made because PNGs and JPEGs suck for AI

#66

Post by refulgentis »

gruez wrote: post #58
refulgentis wrote: post #40

Earlier quoted context omitted.

There's a false equivalence there, between being not-even-wrong and "you have a bunch of UNIX commands followed by a Windows batch file execution." Note we both agree on that, you seem to assume I claimed something else, like, cd doesn't exist on windows. Let's say I instead said "this doesn't work on Windows" I spent probably...8 hours? on Windows this week doing dev, and I'm about 70% sure all of those commands wil…

>There's a false equivalence there, between being not-even-wrong and "you have a bunch of UNIX commands followed by a Windows batch file execution." >Note we both agree on that, you seem to assume I claimed something else, like, cd doesn't exist on windows. No, you made a specific claim of "Your setup instructions use UNIX commands up until they require running a Windows batch file", when those "UNIX commands" were "…

> Your remark about "you seem to assume I claimed something else, like, cd doesn't exist on windows" is absurd at best and verges on bad faith that I'm not even going to engage with it.

You seem very upset, at least, I'm not used to people being this aggressive on HN, and I've been here for 15 years. I apologize for my contribution to that, if not my sole responsibility for it.

I remain fascinated by your process, I never have heard bad faith invoked when someone points at their actual words.

Generally, it is rare someone invokes "bad faith" when someone else's thoughts don't match their expectations.

I just...can't lie to you. I can't claim I thought it wouldn't work on Windows. I thought the opposite! That the sequence had 0% chance of working on not-Windows, and a 70% chance of working on Windows.

>> Are you 100% sure those commands will work? > Again, my claim isn't that the project works 100%, or even that it's not AI generated,

Oh! I'm referring to the commands, not the project :) The project can output "APRIL FOOLS!", as far as I care for this exercise.

> it's that your critique makes little sense either.

Oh, interesting - happy to hear more beyond that I must have meant pip/Python aren't available on Windows. If that's your sole issue, well, more power to you :) I do want to avoid lying to you just to avoid an aggressive conversation, you may not be even meaning to be aggressive. With the principle of "don't lie", I can't say I had something else in my head that matches your understanding so far, I presume something like "They are UNIX commands follows by Windows commands" [and thus this won't work on Windows]

> claiming "pip" and "python" are "UNIX commands"

Do you think I thought pip/Python wasn't on Windows? Sorry, no - in fact that's what I was using on Windows this week! (well, porting Python code to Dart) I just was 70% sure the commands as written would not work on Windows, and I suppose there's an implication I'm 100% sure they wouldn't work on not-Windows given the .bat file. Beyond that, nada.

>> separately, you misunderstand the quote re: glass houses

> No, I agree that you don't have to be 100% correct to accuse people of posting dumb stuff, but you shouldn't be posting dumb stuff either.

Intriguing, as always: "Did you write this with AI?" followed by a kind inquiry into the meaning of that, followed by "people in glass shouldn't throw stones" meant "you said something wrong when you said something else is wrong, but its cool, that's fine" - "shouldn't" seems to bely that interpretation, but I'm sure I have it wrong.

P.s. all the best, my friend. :)

Re: Show HN: Meow – An Image File Format I made because PNGs and JPEGs suck for AI

#67

Post by vunderba »

kuberwastaken wrote: post #55
ai_critic wrote: post #17

Reality check: Your extra data is a big JSON blob. Okay, fine. File formats dating back to Targa ( https://en.wikipedia.org/wiki/Truevision_TGA ) support arbitrary text blobs if you're weird enough. PNG itself has both EXIF data and a more general text chunk mechanism (both compressed and uncompressed, https://www.libpng.org/pub/png/spec/1.2/PNG-Chunks.html#C.An... , section 4.2.3, you probably want iTXt chunks). exi…

Hi! Thanks for checking it out, means a lot :) Yes, it is a big JSON blob atm, haha and t's definitely still a POC, but the idea is to avoid having a separate JSON file that adds to the complexity. While EXIF data works pretty well for most basic stuff, it's not enough for everything one might need for AI specific stuff, especially for things like attention maps and saliency regions. I'm currently working on redundan…

> it's not enough for everything one might need for AI specific stuff, especially for things like attention maps and saliency regions.

Why not exactly? ComfyUI encodes an absolute bonker amount of information (all arbitrary JSON) into workflow PNG files without any issues.

Re: Show HN: Meow – An Image File Format I made because PNGs and JPEGs suck for AI

#68

Post by vunderba »

The amount of information you can encode using EXIF/IPTC doesn't have an upper-bound the way that using stenography is inherently capped by the resolution of the image. What happens when you want to encode more information using the MEOW format than you have "pixels" (which seems like a very real possibility with thumbnail or smaller pictures)?

Re: Show HN: Meow – An Image File Format I made because PNGs and JPEGs suck for AI

#69

Post by ai_critic »

kuberwastaken wrote: post #55
ai_critic wrote: post #17

Reality check: Your extra data is a big JSON blob. Okay, fine. File formats dating back to Targa ( https://en.wikipedia.org/wiki/Truevision_TGA ) support arbitrary text blobs if you're weird enough. PNG itself has both EXIF data and a more general text chunk mechanism (both compressed and uncompressed, https://www.libpng.org/pub/png/spec/1.2/PNG-Chunks.html#C.An... , section 4.2.3, you probably want iTXt chunks). exi…

Hi! Thanks for checking it out, means a lot :) Yes, it is a big JSON blob atm, haha and t's definitely still a POC, but the idea is to avoid having a separate JSON file that adds to the complexity. While EXIF data works pretty well for most basic stuff, it's not enough for everything one might need for AI specific stuff, especially for things like attention maps and saliency regions. I'm currently working on redundan…

> While EXIF data works pretty well for most basic stuff, it's not enough for everything one might need for AI specific stuff, especially for things like attention maps and saliency regions.

That's why I mentioned that you put anything, include binary data--which includes images--into the chunks in a PNG. I think Pillow even supports this (there are some PRs, like https://github.com/python-pillow/Pillow/pull/4292 , that suggest this).

Your problem domain is:

* Have something that looks like a PNG...

* ...that doesn't need supporting files outside itself...

* ...that can also store textual data (e.g., that JSON blob of bounding boxes and whatnot)...

* ...and can also store image data (e.g., attention maps and saliency regions).

What I'm telling you is that the PNG file format already supports all of this stuff, you just need to be smart enough to read the spec and apply the affordances it gives you.

> I'm currently working on redundancy and error correction to deal with the resizing problem. Having a separate file format, even if it's a headache and adds another one to the list (well, another cute-sounding one at least), gives more customization options and makes it easier to associate the properties directly.

In the 90s, we'd already spent vast sums of gold and blood and tears solving the "holy shit, how do we encode multiple things in images so that they can survive an image pipeline, be extensible to end users, and be compressed reliably."

None of this has been new for three decades. Nothing you are going to do is going to be a value add over correctly using the file format you already have.

I promise that you aren't going to see anything particularly new or exciting in this AI goldrush that isn't an isomorphism of something much smarter, much better-paid people solved back when image formats were still a novel problem domain (again, in the 1990s).

Re: Show HN: Meow – An Image File Format I made because PNGs and JPEGs suck for AI

#70

Post by ai_critic »

vunderba wrote: post #67
kuberwastaken wrote: post #55

Earlier quoted context omitted.

Hi! Thanks for checking it out, means a lot :) Yes, it is a big JSON blob atm, haha and t's definitely still a POC, but the idea is to avoid having a separate JSON file that adds to the complexity. While EXIF data works pretty well for most basic stuff, it's not enough for everything one might need for AI specific stuff, especially for things like attention maps and saliency regions. I'm currently working on redundan…

> it's not enough for everything one might need for AI specific stuff, especially for things like attention maps and saliency regions. Why not exactly? ComfyUI encodes an absolute bonker amount of information (all arbitrary JSON) into workflow PNG files without any issues.

Indeed. And character cards for chatbots (like in SillyTavern) have supported this for years.
Post reply on HN