Software 2.0 (2017)
medium.com
Software 2.0 (2017)
1–10 of 23 posts
Re: Software 2.0 (2017)
#2A classical case of "when all you have is a hammer, everything looks like a nail".
Or rather, you're blind to everything that doesn't require a hammer.
Even in most of the examples he listed (speech/voice recognition software, translation, games, and databases), the vast majority of "code"/logic is not a neural network.
You don't "train" your UI, neural nets won't directly consume audio or spit out encoded audio, don't do HTTP, aren't an operating system and just having a game AI is pretty far from having a finished game.
Neural nets approximate and guess, but for the vast majority of problems in computing we want the exactness of code.
And coincidentally that's what makes neural networks great: They let us solve problems where stating the exact precise steps to solve them is impossible.
Re: Software 2.0 (2017)
#3This has not aged well. Not saying deep learning is not useful, but the hype finally caught up with reality.
Re: Software 2.0 (2017)
#4The author's boastful tweet should really be examined. Can a deep-learning stack write better code than a person?
Although, really, the more important question: Can we do studies on this without comparing expert computers to undergraduates who are just learning?
Re: Software 2.0 (2017)
#5Reminds me of those videos from the 1950s that predicted what life would look like in the year 2000.
He's right about cleaning datasets being an entire job itself. But that should have been a red flag for his conclusion even then. Scaling anything means scaling costs. So, the more you want your neurochip to do that it isn't capable of doing by design, you need to do before or after it's done its task(s). That's what's between the lines of what he mentioned of "silent fails." If you don't want bias in your output, and your design isn't capable of vetting bias, you need to do the work of vetting bias before you pass that dataset off. That means you need an entire model defining bias, predicting impact and constraints on outputs.
I've said it before and will continue saying it 'til I'm apparently blue in the face: The complexity of a solution is dependent on the complexity of the problem it solves. I get that simple things feel attractive because they require less effort. We don't really reduce effort so much as we shift it from wholly solutionizing to partly solutionizing and mostly trying to continue to partly solutionize.
Re: Software 2.0 (2017)
#6A classical case of "when all you have is a hammer, everything looks like a nail". Or rather, you're blind to everything that doesn't require a hammer. Even in most of the examples he listed (speech/voice recognition software , translation, games, and databases), the vast majority of "code"/logic is not a neural network. You don't "train" your UI, neural nets won't directly consume audio or spit out encoded audio, do…
> You don't "train" your UI, neural nets won't directly consume audio or spit out encoded audio
I'm not an expert, but I thought pictures and audio were one of the few places where you could feed raw inputs into a neural net and get good results? Or am I wrong and we instead feed in some pre-processed version?
Re: Software 2.0 (2017)
#7A classical case of "when all you have is a hammer, everything looks like a nail". Or rather, you're blind to everything that doesn't require a hammer. Even in most of the examples he listed (speech/voice recognition software , translation, games, and databases), the vast majority of "code"/logic is not a neural network. You don't "train" your UI, neural nets won't directly consume audio or spit out encoded audio, do…
> You don't "train" your UI, neural nets won't directly consume audio or spit out encoded audio I'm not an expert, but I thought pictures and audio were one of the few places where you could feed raw inputs into a neural net and get good results? Or am I wrong and we instead feed in some pre-processed version?
My understanding is that they took in RGB picture data, yes, but that they were required to be square (s.t. matrix transforms work on them, I think?).
But that's still different from taking in the actual JPEG data, which is sort of what the parent gets at: something has to decode that, and that software isn't a neural net.
(Further, when I worked w/ ML that dealt w/ image data, we had a host of non-ML code written around it to support it, dealing with the various facets of running in the cloud, where to get the data, where to store the results, who to notify about the results, and a bunch of preprocessing on the image — such as removing pointless borders that humans put around images.)
Re: Software 2.0 (2017)
#8A classical case of "when all you have is a hammer, everything looks like a nail". Or rather, you're blind to everything that doesn't require a hammer. Even in most of the examples he listed (speech/voice recognition software , translation, games, and databases), the vast majority of "code"/logic is not a neural network. You don't "train" your UI, neural nets won't directly consume audio or spit out encoded audio, do…
> You don't "train" your UI, neural nets won't directly consume audio or spit out encoded audio I'm not an expert, but I thought pictures and audio were one of the few places where you could feed raw inputs into a neural net and get good results? Or am I wrong and we instead feed in some pre-processed version?
I think the point is that the OS needs to interface with the hardware and read out the data, which is then passed to a program which works with it, part of which is a neural network in this case. Usually there's some marshalling and unmarshalling in there too.
Re: Software 2.0 (2017)
#9Earlier quoted context omitted.
> You don't "train" your UI, neural nets won't directly consume audio or spit out encoded audio I'm not an expert, but I thought pictures and audio were one of the few places where you could feed raw inputs into a neural net and get good results? Or am I wrong and we instead feed in some pre-processed version?
My understanding is that they took in RGB picture data, yes, but that they were required to be square (s.t. matrix transforms work on them, I think?). But that's still different from taking in the actual JPEG data, which is sort of what the parent gets at: something has to decode that, and that software isn't a neural net. (Further, when I worked w/ ML that dealt w/ image data, we had a host of non-ML code written ar…
That software can be generated and continuously optimized by a neutral net. But that would probably mean that we achieved AGI, so decoding JPEGs won’t be that exciting by itself.
Re: Software 2.0 (2017)
#10Ideas discussed in his post might seem too far from the reality and controversial, but first don't forget they have been doing Tesla's self-driving ML models for a few years now - so, he definitely has some material to generalize. It's one of the most advanced ML models in production, and without reflecting on the process it would be hard to develop it, would be hard to maintain it, etc.
Also, from my own experience building DVC - when you do any ML project you do have code indeed, but not doubt that data can be considered as important element as code, we need to take it seriously - track, review, etc, etc.