41–50 of 84 posts
Step 1: Create .meow images of vegetables, with "per-pixel metadata" instead encoded to represent human faces. Step 2: Get your images included in the data set of a generative image model. Step 3: Laugh uproariously as every image of a person has vaguely-to-profoundly vegetal features.
You generated pretty much ~all of this with Claude (c.f. ASCII diagrams with emojis on each line to "prove" various not-even-wrong claims it was told to justify), and the work is mediocre enough that it's worth full-throatedly criticizing both the work quality and that you inflicted this upon the world. Look how many confused comments there are due to the page claiming features you don't have, don't understand, and d…
>what's an "attention map"? with maximum charity, if we had some sort of attention-as-in-LLM-like structure precached, how would it apply beyond one model? By “attention map” I meant a visual representation of where a model can focuses its “attention” when analyzing an image — basically, a heatmap highlighting important regions that influence the model’s output. It isn't something that is very useful now but might be.
> You reinvented EXIF/JPEG metadata with naivete Partly true (at least for now) the core idea was to experiment with alternative metadata or feature embedding, not to replace well-established standards. It's not where I NEED it to be yet but as far as metadata usecases go, it's pretty cool.
> Your setup instructions use UNIX commands up until they require running a Windows batch file It's easier to set windows up to directly open other file formats, it's just a thing (and I'm on windows - so)
I don't understand the purpose of effectively hard-coding things like edge detection and attention weight maps into the image. There are various ways to do edge detection and various ways to focus attention, so having that fixed and encoded into the image instead of synthesizing it on demand to suit your particular ends seems suboptimal. Wouldn't the kind of metadata that's most useful be things that can't be synthes…
It’s definitely not finished, more like a poc right now for storing richer, AI-relevant metadata in a portable way. Appreciate you for taking the time to check it out.
No one runs an edge detection first then sends the image as a screenshot and then trains ai on it. That's an absurd workflow. Maybe your format could have some use, but I don't find your motivation convincing.
> Maybe your format could have some use, but I don't find your motivation convincing.
That's respectable and it very much still is a POC, I hope to keep working on it to be actually great :)
This is not a good idea in practice. Why not bundle the metadata as JSON or Protobuf via an aux file?
I do like the idea of storing it steganographically, which also serves as a watermark. But it requires a ton of redundancy and error correction, perhaps enough to survive a few rounds of not-too-lossy reencoding. I dunno how much bandwidth is available before it damages the image.
Maybe I'm jaded, but I fail to see how a bespoke file format is a better solution than bundling a normal image and a JSON/XML document containing metadata that adheres to a defined specification. It feels like creating a custom format with backwards PNG compatibility and using steganography to cram metadata inside is an inefficient and over-engineered alternative to a .tar.gz with "image.png" and "metadata.json"
When images get loose in the wild, this can be very helpful.