61–70 of 94 posts
Earlier quoted context omitted.
It's theoretically possible for a neural net to do this; the network just needs to have the explanation as an output. I agree that decision trees would be more reliable and easier to train, but I'm not sure if hardcoding every feature is scalable.
> the network just needs to have the explanation as an output And how would you evaluate whether the explanation was correct or not?
Earlier quoted context omitted.
Neural nets are infamous for not doing this. Learning algorithms that produce decision trees are usually used in this situation.
This might be a dumb question, but let's say that for whatever reason on a specific problem it's much easier to train a neural network that generalizes well than a decision tree. Why not train the network, then build an equivalent decision tree that just tries to reproduce the network's output? When building the tree from the network, overfitting would not be a concern. In fact, you'd want it to overfit. You could ev…
Take image classification as an example. CNNs can do it by finding nonlinear patterns that exist. A decision tree would have a very tough time doing it because the pixels have a complicated relationship with each other that defines what the image is of.
I think for something like the data transformations we're talking about a Neural Network would be pretty over kill. It looks like this feature in excel works by comparing the data to pre-defined formats, which is probably done by searching all known formats in a somewhat intelligent (not ai, just intelligent) way so that it's fast. Then it can output that type of data in whatever form you want.
Your comment gave me an interesting idea though: What if we put neural networks inside of decision trees?
You know what this reminds me of? Those trained neural-net things which, however many training examples you give it, always seem to find some way to “cheat” and not do what you want while still obeying all your training data correctly. Something like this: Suppose we have a table of strings of digits, some including spaces, and we’d like to remove the spaces. From 123 456 234567 345 678 to 123456 234567 345678 Now, w…
This is why I want any ML device to be able to explain itself. It could train on your before-and-after examples and come up with a list of what it thinks you want it to do. For your example, it could list: “Remove interior spaces from each item” or it could say: “Remove the middle character from any 7-character strings to make them 6 characters in length” You would be able to do something with that.
Earlier quoted context omitted.
Neural nets are infamous for not doing this. Learning algorithms that produce decision trees are usually used in this situation.
This might be a dumb question, but let's say that for whatever reason on a specific problem it's much easier to train a neural network that generalizes well than a decision tree. Why not train the network, then build an equivalent decision tree that just tries to reproduce the network's output? When building the tree from the network, overfitting would not be a concern. In fact, you'd want it to overfit. You could ev…
True
> tunable error threshold, which might give you leeway for finding more comprehensible and simpler trees
True
However, my guess is you'll wind up doing only one of (a) having more accurate tree model than training it directory (b) improve the understand ability of your model significantly.
Humans are really good at taking a vague description of a task and using a small number of examples to disambiguate it. For example, "sort all of the folders, so that it Alan goes before Amy, etc". The rule ("sort") is pretty ambiguous, but one simple example in the context gives enough information to realise you probably mean alphabetically by first name. Is there something like this example that could be combined w…
You know what this reminds me of? Those trained neural-net things which, however many training examples you give it, always seem to find some way to “cheat” and not do what you want while still obeying all your training data correctly. Something like this: Suppose we have a table of strings of digits, some including spaces, and we’d like to remove the spaces. From 123 456 234567 345 678 to 123456 234567 345678 Now, w…
MS had been experimenting with this for a while[1]. They even included this in Excel 2013 as "FlashFill". It does not use any NN/ML at all. It uses "program synthesis", which by definition can tell you exactly what "program" it has synthesized to convert you data. In fact in you example it would not cheat, rather leave the string unchanged as explained in the paper. [1] https://www.microsoft.com/en-us/research/public…
You know what this reminds me of? Those trained neural-net things which, however many training examples you give it, always seem to find some way to “cheat” and not do what you want while still obeying all your training data correctly. Something like this: Suppose we have a table of strings of digits, some including spaces, and we’d like to remove the spaces. From 123 456 234567 345 678 to 123456 234567 345678 Now, w…
You don't realise how big this is. This is the beginning of the automation of coding.
Earlier quoted context omitted.
IIRC you _can_ get this, but it's a huge algorithm that doesn't do things in a way that would probably make sense to a human. It would be amazing to be able to transform code into human language.
I was sloppy in my own examples. I’d be perfectly satisfied with an AST, or regex, or other non-English explanation. But something that I can audit is what I’m after. Otherwise this tool would never be trustworthy enough to let lose on billions of rows of data, with silent errors occurring throughout. (Well, I guess it depends on the nature and importance of the data. Cat photos, meh. Drug prescriptions? Ahh!)
- 123 456 = 123456 valid
- 1234567 = 123567 not valid (dropped 4)
Properties:
- output may not contain whitespace
- no number characters may be dropped
- characters may not be reordered
Earlier quoted context omitted.
MS had been experimenting with this for a while[1]. They even included this in Excel 2013 as "FlashFill". It does not use any NN/ML at all. It uses "program synthesis", which by definition can tell you exactly what "program" it has synthesized to convert you data. In fact in you example it would not cheat, rather leave the string unchanged as explained in the paper. [1] https://www.microsoft.com/en-us/research/public…
But maybe I did want it to remove the middle character! Using my training data, there’s no way for the system to actually know for sure what I meant. There is also no way (in general) for it to detect “outliers” and ask me about them, because there is no good way to know what is an outlier and what is not.