Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: High-Res Neural Cellular Automata

cells2pixels.github.io

Post reply on HN
Print view View original

11–20 of 56 posts

Re: Show HN: High-Res Neural Cellular Automata

#11

Post by esychology »

whilenot-dev wrote: post #8

The automata just completely destroys the image if I draw too much over the stabilized image with the brush. 5 horizontal swipes are enough to destroy the kitty, is that to be expected? EDIT: video here: https://imgur.com/a/ItZGd5X

The NeuralCA both generates and maintains the pattern. Because the NCA was not exposed to damage or erasure during training, its regeneration capability is a purely emergent phenomenon. However, this ability remains somewhat brittle, particularly when the central regions of the pattern are erased.

Re: Show HN: High-Res Neural Cellular Automata

#13

Post by esychology »

hidelooktropic wrote: post #12

For the unfamiliar, could someone explain what I'm looking at? The abstract was a little too concrete (heh) for me to follow.

If you're familiar with CAs (e.g. Conway's Game of Life), you can think of a NeuralCA as a CA where the update rule is given by a neural network. Here we optimize the neural net weights so that it behaves a certain way (e.g. grow a lizard from a single seed).

Re: Show HN: High-Res Neural Cellular Automata

#14

Post by flir »

esychology wrote: post #13
hidelooktropic wrote: post #12

For the unfamiliar, could someone explain what I'm looking at? The abstract was a little too concrete (heh) for me to follow.

If you're familiar with CAs (e.g. Conway's Game of Life), you can think of a NeuralCA as a CA where the update rule is given by a neural network. Here we optimize the neural net weights so that it behaves a certain way (e.g. grow a lizard from a single seed).

What are the inputs to the NN? The whole grid, or just nearby cells? What happens if two NNs overlap on the same grid? (Gonna go read the paper).

Re: Show HN: High-Res Neural Cellular Automata

#15

Post by esychology »

flir wrote: post #14
esychology wrote: post #13

Earlier quoted context omitted.

If you're familiar with CAs (e.g. Conway's Game of Life), you can think of a NeuralCA as a CA where the update rule is given by a neural network. Here we optimize the neural net weights so that it behaves a certain way (e.g. grow a lizard from a single seed).

What are the inputs to the NN? The whole grid, or just nearby cells? What happens if two NNs overlap on the same grid? (Gonna go read the paper).

The input to the NN is just the 3x3 neighborhood around a cell. We can overlap two NNs on the same grid (through interpolation). Checkout https://meshnca.github.io to see the effect. When the brush is in graft mode, it basically allows you to paint some regions that will follow a different NN rule.

Re: Show HN: High-Res Neural Cellular Automata

#16

Post by WhiteNoiz3 »

whilenot-dev wrote: post #8

The automata just completely destroys the image if I draw too much over the stabilized image with the brush. 5 horizontal swipes are enough to destroy the kitty, is that to be expected? EDIT: video here: https://imgur.com/a/ItZGd5X

With the old model (and I suspect this one too) it's trained to generate from a single 'seed' pixel in the center of the image. If you erase the center of the image, that's when it completely collapses.

Re: Show HN: High-Res Neural Cellular Automata

#17

Post by flir »

esychology wrote: post #15
flir wrote: post #14

Earlier quoted context omitted.

What are the inputs to the NN? The whole grid, or just nearby cells? What happens if two NNs overlap on the same grid? (Gonna go read the paper).

The input to the NN is just the 3x3 neighborhood around a cell. We can overlap two NNs on the same grid (through interpolation). Checkout https://meshnca.github.io to see the effect. When the brush is in graft mode, it basically allows you to paint some regions that will follow a different NN rule.

> The input to the NN is just the 3x3 neighborhood around a cell.

Well that sounds like black magic. Nice. Thanks for the reply.

Re: Show HN: High-Res Neural Cellular Automata

#18

Post by esychology »

jekude wrote: post #6

The abstract implies that strictly local updates are a hinderance to high res, however i would have thought there would be an interesting way to get speed up gains from neighbor-only traffic on GPUs CAM-style. am i making that up?

I think performance is not the only issue for scaling to larger grids. CUDA Convolution implementation already utilizes coalescing to improve performance. The main bottleneck is that in larger grids, cells are further apart, and it takes more steps for them to be able to communicate.

Re: Show HN: High-Res Neural Cellular Automata

#19

Post by mackenney »

esychology wrote: post #11
whilenot-dev wrote: post #8

The automata just completely destroys the image if I draw too much over the stabilized image with the brush. 5 horizontal swipes are enough to destroy the kitty, is that to be expected? EDIT: video here: https://imgur.com/a/ItZGd5X

The NeuralCA both generates and maintains the pattern. Because the NCA was not exposed to damage or erasure during training, its regeneration capability is a purely emergent phenomenon. However, this ability remains somewhat brittle, particularly when the central regions of the pattern are erased.

I would love to see two seeds competing for space in the grid

Re: Show HN: High-Res Neural Cellular Automata

#20

Post by WhitneyLand »

At a glance it looks like it could be just iterative texture sampling.

The difference is when creating each pixel, there’s no coordinate to look up, instead it’s using only a set of rules like Conway’s game of life.

But the rules come from a neural network trained on the image, so… it’s kind of memorizing enough information to effectively do the same thing as texture sampling, but using only local information.

I’m sure I’m missing something about how it works or what makes it interesting…

Post reply on HN