For the unfamiliar, could someone explain what I'm looking at? The abstract was a little too concrete (heh) for me to follow.
The original NCA is probably a helpful intro:
https://distill.pub/2020/growing-ca/
Show HN: High-Res Neural Cellular Automata
21–30 of 56 posts
Re: Show HN: High-Res Neural Cellular Automata
#22At a glance it looks like it could be just iterative texture sampling. The difference is when creating each pixel, there’s no coordinate to look up, instead it’s using only a set of rules like Conway’s game of life. But the rules come from a neural network trained on the image, so… it’s kind of memorizing enough information to effectively do the same thing as texture sampling, but using only local information. I’m su…
To me, it is intriguing as a toy model for how cells are able to grow into complex tissue and organisms based only on local information, and how they are able to repair and recover harmed tissue.
Of course, this is as close to cells, as neurons from neural networks are to real neurons. And I have no idea what it could be applied to (inpainting/outpainting?), but it’s interesting as exploratory research.
Re: Show HN: High-Res Neural Cellular Automata
#23Really interesting demo, nicely done :) Would be fun if switching the "Target Image" when using the second brush mode in the Growing Demo didn't erase/reset the existing canvas, so we could "stamp" new things on top of other images. Small thing perhaps but I got sad when it disappeared when I wanted to merge a kitten on top of the chameleon but couldn't :(
You can, just enable the 'transition' switch.
That seems to be something else? It takes the current image and "transforms" it into the new target.
Re: Show HN: High-Res Neural Cellular Automata
#24The automata just completely destroys the image if I draw too much over the stabilized image with the brush. 5 horizontal swipes are enough to destroy the kitty, is that to be expected? EDIT: video here: https://imgur.com/a/ItZGd5X
With the old model (and I suspect this one too) it's trained to generate from a single 'seed' pixel in the center of the image. If you erase the center of the image, that's when it completely collapses.
Have you actually tried that? If you specifically erase the center, the image does change a lot at first, but rebuilds itself eventually (albeit to a slightly different final state). It's uncanny how "biological" is feels!
Re: Show HN: High-Res Neural Cellular Automata
#25The automata just completely destroys the image if I draw too much over the stabilized image with the brush. 5 horizontal swipes are enough to destroy the kitty, is that to be expected? EDIT: video here: https://imgur.com/a/ItZGd5X
With the old model (and I suspect this one too) it's trained to generate from a single 'seed' pixel in the center of the image. If you erase the center of the image, that's when it completely collapses.
It must be more general than that, otherwise the cells wouldn’t be able to repair their area if the damage came from the wrong direction (repair is not center-out).
The model generally learns to generate each pixel from its surroundings, even if the surroundings are partially missing.
Re: Show HN: High-Res Neural Cellular Automata
#26Earlier quoted context omitted.
With the old model (and I suspect this one too) it's trained to generate from a single 'seed' pixel in the center of the image. If you erase the center of the image, that's when it completely collapses.
Have you actually tried that? If you specifically erase the center, the image does change a lot at first, but rebuilds itself eventually (albeit to a slightly different final state). It's uncanny how "biological" is feels!
If you hold the eraser for a second at the center, I find that it destroys the image more often than not.
Re: Show HN: High-Res Neural Cellular Automata
#27Earlier quoted context omitted.
With the old model (and I suspect this one too) it's trained to generate from a single 'seed' pixel in the center of the image. If you erase the center of the image, that's when it completely collapses.
It must be more general than that, otherwise the cells wouldn’t be able to repair their area if the damage came from the wrong direction (repair is not center-out). The model generally learns to generate each pixel from its surroundings, even if the surroundings are partially missing.
There's hidden state in the model which presumably it uses to communicate position, ie there's the 3 colors but then a bunch of other channels that the model can use how it wants.
Re: Show HN: High-Res Neural Cellular Automata
#28Earlier quoted context omitted.
With the old model (and I suspect this one too) it's trained to generate from a single 'seed' pixel in the center of the image. If you erase the center of the image, that's when it completely collapses.
Have you actually tried that? If you specifically erase the center, the image does change a lot at first, but rebuilds itself eventually (albeit to a slightly different final state). It's uncanny how "biological" is feels!
I have yes.. You need to erase a larger amount of the center, but it almost always results in a collapse wheras erasing around the center typically regrows.
Re: Show HN: High-Res Neural Cellular Automata
#29Why are the images always generated in the same orientation (upright)? Do the cells have awareness of what is "up"?
IIRC training starts with the initial state and the end state, and the end state is always oriented the same way. It would be interesting to see what would happen if the end state was rotated randomly though I suspect it wouldn't work so well.
Re: Show HN: High-Res Neural Cellular Automata
#30I've always loved the original work and it's nice to see they're still working on it. I've always wondered if there was a way to connect this back to infrastructure rather than images. Something you could run on a cluster and if portions of it failed it would regenerate the system.
Like a bio inspired Kubernettes.....Bionettes.