Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Transform Data by Example [video]

microsoft.com

Post reply on HN
Print view View original

31–40 of 94 posts

Re: Transform Data by Example [video]

#31

Post by nilkn »

paulddraper wrote: post #16
function_seven wrote: post #11

Earlier quoted context omitted.

This is why I want any ML device to be able to explain itself. It could train on your before-and-after examples and come up with a list of what it thinks you want it to do. For your example, it could list: “Remove interior spaces from each item” or it could say: “Remove the middle character from any 7-character strings to make them 6 characters in length” You would be able to do something with that.

Neural nets are infamous for not doing this. Learning algorithms that produce decision trees are usually used in this situation.

This might be a dumb question, but let's say that for whatever reason on a specific problem it's much easier to train a neural network that generalizes well than a decision tree. Why not train the network, then build an equivalent decision tree that just tries to reproduce the network's output? When building the tree from the network, overfitting would not be a concern. In fact, you'd want it to overfit.

You could even say that it only needs to approximately reproduce the output with some tunable error threshold, which might give you leeway for finding more comprehensible and simpler trees.

Re: Transform Data by Example [video]

#33

Post by snaky »

nebabyte wrote: post #26
paulddraper wrote: post #16

Earlier quoted context omitted.

Neural nets are infamous for not doing this. Learning algorithms that produce decision trees are usually used in this situation.

Which is why learning algorithms that produce decision trees are far smarter in the long run. Neural nets might eke out other benefits but there's a lot to be said about justifiable/accountable decisions.

Except the big (if not the main) part of modern economics is all about "don't care about long run - that's the only smart strategy".

Re: Transform Data by Example [video]

#34

Post by snaky »

function_seven wrote: post #28
allenz wrote: post #25

Earlier quoted context omitted.

It's theoretically possible for a neural net to do this; the network just needs to have the explanation as an output. I agree that decision trees would be more reliable and easier to train, but I'm not sure if hardcoding every feature is scalable.

How do you know that the explanation jives with the other outputs, though? It seems like a turtles-all-the-way-down situation, because now I want to see how it was properly introspective of its own decision making. Also seems like it’s another magnitude of complexity in the neural net to have it not only train and learn on your inputs, but also train and learn on its own training and learning.

>How do you know that the explanation jives with the other outputs, though?

The third neural net would do the checking, obviously.

Re: Transform Data by Example [video]

#35

Post by Cieplak »

I wonder if it uses Z3 under the hood for solving constraints. Very nice of MSFT to MIT license Z3. It's super useful for problems that result in circular dependencies when modeled in Excel, and require iterative solvers (e.g., goal seek). I use the python bindings, but unfortunately it's not as simple as `pip install` and requires a lengthy build/compilation. Well worth the effort, though.

https://github.com/Z3Prover/z3

https://github.com/Z3Prover/z3/issues/288

Re: Transform Data by Example [video]

#36

Post by Giroflex »

teddyh wrote: post #5

You know what this reminds me of? Those trained neural-net things which, however many training examples you give it, always seem to find some way to “cheat” and not do what you want while still obeying all your training data correctly. Something like this: Suppose we have a table of strings of digits, some including spaces, and we’d like to remove the spaces. From 123 456 234567 345 678 to 123456 234567 345678 Now, w…

This isn't the machine's fault, though; for a small number of linearly independent examples there exists an enormous number of possible functions that match the training data. It has no way of guessing, really.

Re: Transform Data by Example [video]

#37

Post by tonmoy »

teddyh wrote: post #5

You know what this reminds me of? Those trained neural-net things which, however many training examples you give it, always seem to find some way to “cheat” and not do what you want while still obeying all your training data correctly. Something like this: Suppose we have a table of strings of digits, some including spaces, and we’d like to remove the spaces. From 123 456 234567 345 678 to 123456 234567 345678 Now, w…

MS had been experimenting with this for a while[1]. They even included this in Excel 2013 as "FlashFill". It does not use any NN/ML at all. It uses "program synthesis", which by definition can tell you exactly what "program" it has synthesized to convert you data. In fact in you example it would not cheat, rather leave the string unchanged as explained in the paper.

[1] https://www.microsoft.com/en-us/research/publication/automat...

Re: Transform Data by Example [video]

#38

Post by tonmoy »

Cieplak wrote: post #35

I wonder if it uses Z3 under the hood for solving constraints. Very nice of MSFT to MIT license Z3. It's super useful for problems that result in circular dependencies when modeled in Excel, and require iterative solvers (e.g., goal seek). I use the python bindings, but unfortunately it's not as simple as `pip install` and requires a lengthy build/compilation. Well worth the effort, though. https://github.com/Z3Prove…

Love Z3. It is easy to use and very decent performance! I don't think MS is using Z3 on this product though, looks more like the smart enumeration based program synthesis

Re: Transform Data by Example [video]

#39

Post by pishpash »

function_seven wrote: post #11
teddyh wrote: post #5

You know what this reminds me of? Those trained neural-net things which, however many training examples you give it, always seem to find some way to “cheat” and not do what you want while still obeying all your training data correctly. Something like this: Suppose we have a table of strings of digits, some including spaces, and we’d like to remove the spaces. From 123 456 234567 345 678 to 123456 234567 345678 Now, w…

This is why I want any ML device to be able to explain itself. It could train on your before-and-after examples and come up with a list of what it thinks you want it to do. For your example, it could list: “Remove interior spaces from each item” or it could say: “Remove the middle character from any 7-character strings to make them 6 characters in length” You would be able to do something with that.

This add-on already does that. (Did nobody try it??) It shows in the pane a list of candidate transforms, seemingly ranked in some descending plausibility order. They have semi-readable names. You get to choose one to apply.

Re: Transform Data by Example [video]

#40

Post by gergoerdi »

Check out MagicHaskeller which figures out list processing functions from examples: http://nautilus.cs.miyazaki-u.ac.jp/~skata/MagicHaskeller.ht...

For example, given the rule `f "abcde" 2 == "aabbccddee"`, it even figures out the role of the parameter `2`, so `f "zq" 3` gives `"zzzqqq"`.

Post reply on HN