Reverse engineering a neural network's clever solution to binary addition
1–10 of 160 posts
Re: Reverse engineering a neural network's clever solution to binary addition
#2Interesting idea to reverse engineer the network. Are there other sources that have done this?
Re: Reverse engineering a neural network's clever solution to binary addition
#3That's awesome - the network's solution is essentially to convert the input to analog, perform the actual addition in analog, and then convert that back to digital. And the first two parts of that all happened in the input weights, no less.
Re: Reverse engineering a neural network's clever solution to binary addition
#4Very interesting. But I missed how the network handled the overflow.
Spotted one small typo: digital to audio converter should be digital to analog.
Re: Reverse engineering a neural network's clever solution to binary addition
#5Awesome reverse engineering project. Next up: reverse engineer a NN's solution to Fourier Transform!
Re: Reverse engineering a neural network's clever solution to binary addition
#6Very interesting. But I missed how the network handled the overflow. Spotted one small typo: digital to audio converter should be digital to analog.
Do you know how DAC/ADC circuits work? Build one that can handle one more bit of input than you have and you have overflow handling.
Re: Reverse engineering a neural network's clever solution to binary addition
#7Interesting idea to reverse engineer the network. Are there other sources that have done this?
The field of NN explainability tries, but usually there's only handwavy things to be done (because too many weights). This project involved intentionally building very very small networks that can be understood completely.
Re: Reverse engineering a neural network's clever solution to binary addition
#8Interesting idea to reverse engineer the network. Are there other sources that have done this?
Maybe not exactly what you had in mind, but is a lot of literature in general on trying to extract interpretations from neural network models, and to a lesser extent from other complicated nonlinear models like gradient boosted trees.
Somewhat famously, you can plot the weight activations on a heatmap from a CNN for image processing and obtain a visual representation of the "filter" that the model has learned, which the model (conceptually) slides across the image until it matches something. For example: https://towardsdatascience.com/convolutional-neural-network-...
Many techniques don't look directly at the numbers in the model. Instead, they construct inputs to the model that attempt to trace out its behavior under various constraints. Examples include Partial Dependence, LIME, and SHAP.
Also, those "deep dream" images that were popular a couple years ago are generated by running parts of a deep NN model without running the whole thing.
Re: Reverse engineering a neural network's clever solution to binary addition
#9Awesome reverse engineering project. Next up: reverse engineer a NN's solution to Fourier Transform!
The Fourier transform is also linear, so the same solution should work. No clue if an NN would find it though.
Re: Reverse engineering a neural network's clever solution to binary addition
#10Very interesting. But I missed how the network handled the overflow. Spotted one small typo: digital to audio converter should be digital to analog.
Oh nice catch on the typo - thanks!!