111–120 of 160 posts
Earlier quoted context omitted.
It's sad that those pesky regulators from the FCC make it so hard to get devices with unintended radio functionality these days!
I know, if people really cared about devices rejecting interference the free market would sort that out, amirite?
Think "property rights" are required for a free market.
In a similar vein, in the 1980s a "superoptimizer" was built for optimizing snippets of assembler by doing an exhaustive search of all combinations of instructions. It found several delightful and unexpected optimizations, which quickly became incorporated into compiler code generators.
These stories remind me of a story from Discover Magazine https://www.discovermagazine.com/technology/evolving-a-consc... A researcher was using a process to "evolve" a FPGA and the result was a circuit that was super efficient but worked in ways that were unexpected: part of the circuit seemed unconnected to the rest but if removed the whole thing stopped working and it would only work at a specific temperature.
These stories remind me of a story from Discover Magazine https://www.discovermagazine.com/technology/evolving-a-consc... A researcher was using a process to "evolve" a FPGA and the result was a circuit that was super efficient but worked in ways that were unexpected: part of the circuit seemed unconnected to the rest but if removed the whole thing stopped working and it would only work at a specific temperature.
I can't believe someone dropped a link to this story. I remember reading this and feeling like it broadened my sense of what evolutionary processes can produce.
That's awesome - the network's solution is essentially to convert the input to analog, perform the actual addition in analog , and then convert that back to digital. And the first two parts of that all happened in the input weights, no less.
Maybe in the future we will think of the word "digital" as archaic and "analog" will be the moniker for advanced tech.
Earlier quoted context omitted.
> a TWh, which is about six million years' of power i'm not a physics guy but wanted to check this - a Watt is a per second measure, and a Terawatt is 1 trillion watts, so 1 TWh is 50 billion seconds of 20 Watts, which is 1585 years of power for a single brain, not 6 million. i'm sure i got this wrong as i'm not a physics guy but where did i go wrong here? a more neutral article (that doesn't have a clear "AI is harm…
Watts are actually a time independent measurement, note that the TWh has "hour" affixed to the end. This is 1 Tera Watt over the course of one hour, not one second. Your numbers are off by a factor of 3600. 1TWh / 20 Watt brain = 50,000,000,000 (50 Billion) Hours. 50 Billion Hours / (24h * 365.25) = 5,703,855.8 Years
Impressive. Am I right that what is really going on here is that the network is implementing the "+" operator by actually having the CPU of the host carrying out the "+" when executing the network? I.e., the network converts the binary input and output to floating point, and then it is the CPU of the host the network is running on that really does the addition in floating point. So usually one does a bunch of FLOPs t…
That's an accurate enough description of what's going on here, yeah, but I'm very curious if this could be implemented in hardware. What we've got is an inexact not-quite-binary adder, but one that's potentially smaller and faster than the regular binary ones.
For instance instead of evaluating an exponential function in digital logic, it might be quicker and more energy-efficient to just evaluate it using a diode as an analog voltage, if the value is available as an analog voltage and if some noise is tolerable, as done with analog computers.
Earlier quoted context omitted.
That's an accurate enough description of what's going on here, yeah, but I'm very curious if this could be implemented in hardware. What we've got is an inexact not-quite-binary adder, but one that's potentially smaller and faster than the regular binary ones.
It can be implemented in hardware but the implementation would be more complex than a digital adder based on logical gates.
Earlier quoted context omitted.
An 8 bit adder is too small and allows such 'hack'. He should try training a 32 or 64 bit adder, decrease the weights accuracy to bfloat16, introduce dropout regularization (or other kind of noise) to get more 'interesting' results. Addendum: another interesting variation to try is a small transformer network, and feeding the bits sequentially as symbols. This kind of architecture could compute bignum-sized integers.
A 32-bit adder is just 4 8-bit adders with carry connected. I don't see why it'd be significantly more difficult.
Edit: This wiki page has a good explanation of the concept: https://en.wikipedia.org/wiki/Carry-lookahead_adder