Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Un-0: Generating Images with Coupled Oscillators

unconv.ai

Post reply on HN
Print view View original

41–50 of 59 posts

Re: Un-0: Generating Images with Coupled Oscillators

#41

Post by vintermann »

em3rgent0rdr wrote: post #27
andybak wrote: post #5

When I first learned about computer science at the age of 11 or so (and in 1982 or so) the first page of the text book put digital and analogue computers on what seemed to be an equal footing. And then proceeded to ignore the latter for the rest of the book. Apart from a few notable exceptions ( https://en.wikipedia.org/wiki/Phillips_Machine ) I've often wondered about analogue computing.

Noise and component imprecision has always limited analog computing.

In neutral networks, we seem to be pushing towards ever lower precision floats, and we use noise for all sorts of useful things.

Re: Un-0: Generating Images with Coupled Oscillators

#42

Post by vessenes »

em3rgent0rdr wrote: post #27
andybak wrote: post #5

When I first learned about computer science at the age of 11 or so (and in 1982 or so) the first page of the text book put digital and analogue computers on what seemed to be an equal footing. And then proceeded to ignore the latter for the rest of the book. Apart from a few notable exceptions ( https://en.wikipedia.org/wiki/Phillips_Machine ) I've often wondered about analogue computing.

Noise and component imprecision has always limited analog computing.

Also true: all computing is analog computing.

Re: Un-0: Generating Images with Coupled Oscillators

#43

Post by vessenes »

OutOfHere wrote: post #8

Can this even make an image having more than one "class"? Can it make an image of an astronaut riding a horse on the moon?

Yes, I had the same question. I don’t think so, as currently designed. It trains to specific points / classes in an embedding space. They didn’t discuss how one might go to non-trained points in the paper as far as I could read, and they did show some visualization around the idea that the runs aim at / around set points in the space.

Re: Un-0: Generating Images with Coupled Oscillators

#44

Post by vessenes »

Very cool. I’m reminded of Wolfram’s pitch that neural nets are a search through the very broad computational complexity of the function space they describe; he did a little work to show that you could find similar behavior in other function spaces. These oscillators are yet again a different function space, and its cool they can be harnessed in this way.

The question of what physical / electronic phenomena is the most efficient yet large enough function space to be used for inference is a really good one to think about. I have no suggestions.

Re: Un-0: Generating Images with Coupled Oscillators

#45

Post by kannanvijayan »

WhitneyLand wrote: post #12

It’s not clear to me how this would ever be practical since it seems dependent on n^2 scaling. You’ve got to wonder when you have an image generation demo why would you possibly have 64 x 64 pixel output as your demo? If I’m understanding this properly to generate a 4K image, you need like 5 trillion point to point connections on the chip. Even if power use from the oscillators is zero that’s going to be an issue.

I read through the article, and I'm not sure this is dependent on quadratic scaling.

Are they allowing all oscillators to influence all others, or are they picking modalities where the influences can be limited to some maximal fixed degree?

One would imagine that there'd be a variety of different topologies available to explore. Even if during training the treatment was fully connected, one could imagine the training itself biasing towards a maximal fixed degree per oscillator, and then inference later operating on a quantized version of that that drops the low-weight influences to zero.

Re: Un-0: Generating Images with Coupled Oscillators

#46

Post by mdnahas »

andybak wrote: post #5

When I first learned about computer science at the age of 11 or so (and in 1982 or so) the first page of the text book put digital and analogue computers on what seemed to be an equal footing. And then proceeded to ignore the latter for the rest of the book. Apart from a few notable exceptions ( https://en.wikipedia.org/wiki/Phillips_Machine ) I've often wondered about analogue computing.

My father designed processors. He says all electronics are analog. Some just pretends to act digital.

Re: Un-0: Generating Images with Coupled Oscillators

#47

Post by jerf »

andybak wrote: post #5

When I first learned about computer science at the age of 11 or so (and in 1982 or so) the first page of the text book put digital and analogue computers on what seemed to be an equal footing. And then proceeded to ignore the latter for the rest of the book. Apart from a few notable exceptions ( https://en.wikipedia.org/wiki/Phillips_Machine ) I've often wondered about analogue computing.

If you want to understand the issue with analog computers, design a SHA-256 circuit for one of them and consider the consequences of trying to push a megabyte of data through it. While that is an extreme example I chose precisely to make the issues clear, much real computing has many of the same characteristics, just distributed a bit more widely in time and space.

Or, to put it another way, you can make anything sound good if you consider only the positives and anything sound bad if you only consider the negatives. Analog computing sounds amazing when you read the brochure and consider only the positives. But when you bring the negatives back in, it makes sense why it is not frequently used. It is not a case of the mainstream keeping some great idea down because, uh, Big Digital or something, it's a case where digital computing turns out to be a stonking good idea and it's hard for the analog world to compete and it's virtually impossible for them to ever be anything but a niche.

Neural networks are an interesting possibility for a future successful niche, although even so, it would be neural networks specifically that may grow in importance and not analog computing in general. And I still wouldn't guarantee it'll be a good idea... we may have a lot of trouble keeping what would be very deeply nested analog circuitry stable in the real world and digital may still win out, e.g., an analog neural net that has a noticeable personality shift when it gets warmer may not be the best engineering solution. That's a question for 20 or 30 years from now.

Re: Un-0: Generating Images with Coupled Oscillators

#48

Post by ShinyLeftPad »

jerf wrote: post #47
andybak wrote: post #5

When I first learned about computer science at the age of 11 or so (and in 1982 or so) the first page of the text book put digital and analogue computers on what seemed to be an equal footing. And then proceeded to ignore the latter for the rest of the book. Apart from a few notable exceptions ( https://en.wikipedia.org/wiki/Phillips_Machine ) I've often wondered about analogue computing.

If you want to understand the issue with analog computers, design a SHA-256 circuit for one of them and consider the consequences of trying to push a megabyte of data through it. While that is an extreme example I chose precisely to make the issues clear, much real computing has many of the same characteristics, just distributed a bit more widely in time and space. Or, to put it another way, you can make anything sou…

Is SHA-256 something that makes sense in analog realm, or is it something that only needs to exist due to digital constraints?

Re: Un-0: Generating Images with Coupled Oscillators

#49

Post by alfiedotwtf »

fc417fc802 wrote: post #19
ainch wrote: post #9

This method is cool and the post explains it well. It would, however, be good to get more detail on the energy efficiency they flag as their motivation: is this model actually more energy efficient than the comparators they highlight?

It seems like total parameter count is more or less on par with conventional approaches so any gains won't be from there. We can implement coupled oscillators in hardware but are the couplings and frequencies programmable? If they're being streamed in I guess you'd still have a memory bandwidth bottleneck and associated energy usage. If not then the fair comparison is to a conventional model hardcoded in an ASIC whic…

Do the parameters in these harmonic systems compress better? Instead of needing to hold individual parameters for each oscillator, could groupings of oscillators be instead be described with its output over a given time and then just reverse that output to get the original parameters (I’m thinking the output is like an FFT of the oscillators which is a single value, then do an inverse FFT to get the original oscillator parameters etc)
Post reply on HN