Transform Data by Example [video] • Hacker News Board

Re: Transform Data by Example [video]

#91

Post by allenz » Fri, May 19, 2017, 10:38 PM UTC

function_seven wrote: post #28

allenz wrote: post #25

Earlier quoted context omitted.

It's theoretically possible for a neural net to do this; the network just needs to have the explanation as an output. I agree that decision trees would be more reliable and easier to train, but I'm not sure if hardcoding every feature is scalable.

How do you know that the explanation jives with the other outputs, though? It seems like a turtles-all-the-way-down situation, because now I want to see how it was properly introspective of its own decision making. Also seems like it’s another magnitude of complexity in the neural net to have it not only train and learn on your inputs, but also train and learn on its own training and learning.

Neural networks don't do anything as sophisticated as self-referential introspection. They just fit the outputs you train them with. The training data you provide would have to include the desired explanations.

Consistency is enforced by the dataset, and also by the model. Both outputs would read from the same hidden layer--the one that encodes the desired transformation.

Re: Transform Data by Example [video]

#92

Post by tdbeteam » Fri, May 19, 2017, 10:47 PM UTC

Relationship to FlashFill feature in Excel: FlashFill is a popular feature in Excel that also uses the example-driven paradigm to automatically produce transformations. While FlashFill supports string-based transformations, Transform Data by Example can leverage sophisticated domain-specific functions to perform semantic transformations beyond string manipulations. For examples, see: https://www.microsoft.com/en-us/research/wp-content/uploads/...

Re: Transform Data by Example [video]

#93

Post by ke1vin » Sat, May 20, 2017, 6:43 AM UTC

teddyh wrote: post #5

You know what this reminds me of? Those trained neural-net things which, however many training examples you give it, always seem to find some way to “cheat” and not do what you want while still obeying all your training data correctly. Something like this: Suppose we have a table of strings of digits, some including spaces, and we’d like to remove the spaces. From 123 456 234567 345 678 to 123456 234567 345678 Now, w…

When you only provide the system with a few examples, there are many possible transformations which satisfy the examples.

One way to eliminate this ambiguity is to also provide a natural language description of what you want, e.g. "remove the spaces".

In the natural language processing community, we call this semantic parsing.

But sometimes the semantic parser can misinterpret the language too and generate a program which still "cheats" in the same manner as you described. We call these "spurious programs".

Shameless plug-- my group has been working on how to deal with these spurious programs:

From Language to Programs: Bridging Reinforcement Learning and Maximum Marginal Likelihood https://arxiv.org/abs/1704.07926

Re: Transform Data by Example [video]

#94

Post by [deleted] » Sun, May 21, 2017, 4:21 PM UTC

jmmcd wrote: post #89

Houshalter wrote: post #88

Earlier quoted context omitted.

>Any errors the NN is making are not worth learning about. But that's the whole point of this method! To understand what errors the NN might be making. It's also quite possible the NN's errors aren't really errors, if there are mistakes or noise in the labels. This technique has been called "dark knowledge" and is really interesting. See http://www.kdnuggets.com/2015/05/dark-knowledge-neural-netwo... They train much…

Ah, very interesting! I agree that would be useful. But I think this thread has ended up with a proposal very different from the one I started replying to.

[deleted]