These stories remind me of a story from Discover Magazine https://www.discovermagazine.com/technology/evolving-a-consc...
A researcher was using a process to "evolve" a FPGA and the result was a circuit that was super efficient but worked in ways that were unexpected: part of the circuit seemed unconnected to the rest but if removed the whole thing stopped working and it would only work at a specific temperature.
Reverse engineering a neural network's clever solution to binary addition
41–50 of 160 posts
Re: Reverse engineering a neural network's clever solution to binary addition
#42Earlier quoted context omitted.
Is this true? If it can add, it can probably subtract? If it can add and subtract it may be able to multiply (repeated addition) and divide? If it can multiply it can do exponents? I don’t know, but cannot jump to your conclusion without much more domain knowledge.
> If it can add and subtract it may be able to multiply (repeated addition) and divide? If it can multiply it can do exponents? It was just a simple feed-forward network. It can't do arbitrary amounts of repeated addition (nor repeat any other operation arbitrarily often).
None of these require arbitrary amounts of repeated addition though. E.g. multiplying two 8 bit numbers requires at most 7 additions.
Re: Reverse engineering a neural network's clever solution to binary addition
#43A couple questions:
1. How much of this outcome is due to the unusual (pseudo) periodic activation function? Seems like a lot of the DAC-like behavior is coming from the periodicity of the first layer’s output, which seems to be due to the unique activation function.
2. Would the behavior of the network change if the binary strings were encoded differently? The author encodes them as 1D arrays with 1 corresponding to 1 and 0 corresponding to -1, which is an unusual way of doing things. What if the author encoded them as literal binary arrays (i.e. 1->1, 0->0)? What about one-hot arrays (i.e. 2D arrays with 1->[1, 0] and 0->[0, 1]), which is the most common way to encode categorical data?
Re: Reverse engineering a neural network's clever solution to binary addition
#44These stories remind me of a story from Discover Magazine https://www.discovermagazine.com/technology/evolving-a-consc... A researcher was using a process to "evolve" a FPGA and the result was a circuit that was super efficient but worked in ways that were unexpected: part of the circuit seemed unconnected to the rest but if removed the whole thing stopped working and it would only work at a specific temperature.
Ah, the good old radio component you can program into fpgas.
Re: Reverse engineering a neural network's clever solution to binary addition
#45The essay linked from the article is interesting: http://www.incompleteideas.net/IncIdeas/BitterLesson.html
One thing that the essay doesn't consider is the importance of efficiency in computation. Efficiency is important because in practice it is often the factor which most limits the scalability of a computational system.
The human brain only consumes around 20W [1], but for numerical calculations it is massively outclassed by an ARM chip consuming a tenth of that. Conversely, digital models of neural networks need a huge power budget to get anywhere close to a brain; this estimate [2] puts training GPT-3 at about a TWh, which is about six million years' of power for a single brain.
[1] https://www.pnas.org/doi/10.1073/pnas.2107022118
[2] https://www.numenta.com/blog/2022/05/24/ai-is-harming-our-pl....
Re: Reverse engineering a neural network's clever solution to binary addition
#46Earlier quoted context omitted.
The Fourier transform is also linear, so the same solution should work. No clue if an NN would find it though.
What's an analog implementation of a Fourier transform look like? It sounds interesting!
A prism and CCD sensor. Probably bit more fuckery to get phase info out of that tho.
Re: Reverse engineering a neural network's clever solution to binary addition
#47Very interesting. But I missed how the network handled the overflow. Spotted one small typo: digital to audio converter should be digital to analog.
It converted the inputs to analog so there isn't really a notion of 'overflow'.
Of course it is, there is whole business around overflowing analog signals in right way in analog guitar effects pedal
Re: Reverse engineering a neural network's clever solution to binary addition
#48Very interesting. But I missed how the network handled the overflow. Spotted one small typo: digital to audio converter should be digital to analog.
It converted the inputs to analog so there isn't really a notion of 'overflow'.
Putting aside the idea that "analog" doesn't have overflow. (Any physical implementation of that analog signal would have limits in the real world. If nothing else, then how much current or voltage your PSU can supply, or how much current can go through your wire.)
But that is not even the real issue. The real issue that it is not analog, it is just "analog" with quotes. The neural network is executed by a digital computer, on digital hardware. So that "analog" is just the underlying float type of the of the CPU or GPU executing the network. That is still very much digital. Just on a different abstraction level.
Re: Reverse engineering a neural network's clever solution to binary addition
#49Earlier quoted context omitted.
> If it can add and subtract it may be able to multiply (repeated addition) and divide? If it can multiply it can do exponents? It was just a simple feed-forward network. It can't do arbitrary amounts of repeated addition (nor repeat any other operation arbitrarily often).
None of these require arbitrary amounts of repeated addition though. E.g. multiplying two 8 bit numbers requires at most 7 additions.
Are you doing the equivalent of repeated squaring here?
Otherwise, you'd need up to 255 (or so) additions to multiply two 8 bit numbers, I think?
Re: Reverse engineering a neural network's clever solution to binary addition
#50These stories remind me of a story from Discover Magazine https://www.discovermagazine.com/technology/evolving-a-consc... A researcher was using a process to "evolve" a FPGA and the result was a circuit that was super efficient but worked in ways that were unexpected: part of the circuit seemed unconnected to the rest but if removed the whole thing stopped working and it would only work at a specific temperature.
Ah, the good old radio component you can program into fpgas.
Very easy to design a radio component into electronics, much harder to design it out.