Transform Data by Example [video]
microsoft.com
Transform Data by Example [video]
1–10 of 94 posts
Re: Transform Data by Example [video]
#2This is a great product idea. If you ask any Excel power users, by far the most time-consuming and hard-to-automate task is text and date manipulation.
The beauty of this product is that its adoption strategy is baked into the product itself: I'd share this with all Excel user friends of mine because I want the algorithm to get smarter, and I might even learn a bit of C# myself so that I can contribute and scratch my own itch. This in turn makes the product better (because of the larger training data), lending itself to more word of mouth.
One concern I have is security: I'd love to hear from folks who built this/more familiar with this about how to ensure the security of suggested transformations.
Re: Transform Data by Example [video]
#3This is a great product idea. If you ask any Excel power users, by far the most time-consuming and hard-to-automate task is text and date manipulation. The beauty of this product is that its adoption strategy is baked into the product itself: I'd share this with all Excel user friends of mine because I want the algorithm to get smarter, and I might even learn a bit of C# myself so that I can contribute and scratch my…
I, too, wonder about security. Just as important: performance/scaleability. What happens when this runs on 100K rows against a service some guy stood up as a weekend project? Now what happens with 100 people hitting that service?
Either way, this looks very useful. Having spent more than my fair share of time massaging data prior to import, this looks pretty great.
Re: Transform Data by Example [video]
#4I'm playing around with a client-side js implementation of this at https://www.robosheets.com/
It's not production ready / launched yet, but it's getting there.
I'd be interested to finds (or really doesn't find) this useful :)
Re: Transform Data by Example [video]
#5You know what this reminds me of? Those trained neural-net things which, however many training examples you give it, always seem to find some way to “cheat” and not do what you want while still obeying all your training data correctly.
Something like this: Suppose we have a table of strings of digits, some including spaces, and we’d like to remove the spaces. From
123 456
234567
345 678
123456
234567
345678
4567890
Re: Transform Data by Example [video]
#6Is this related/a commercial application of the 'Deep Learning for Program Synthesis' post[0][1] from Microsoft Research on HN a month ago?
[0]https://www.microsoft.com/en-us/research/blog/deep-learning-...
[1]HN Discussion: https://news.ycombinator.com/item?id=14168027
Re: Transform Data by Example [video]
#7There is a paper describing such a method (not sure if that is what was implemented):
"Zhongjun Jin, Michael R. Anderson, Michael J. Cafarella, H. V. Jagadish: Foofah: Transforming Data By Example. SIGMOD Conference 2017: 683-698"
Re: Transform Data by Example [video]
#8alas it is too late,
it transformed our genes to dates,
no sequence for Bill
Re: Transform Data by Example [video]
#9It would be nice if it indicated where it was making stuff up (in the zip code example, for the rows that were missing some data, it just makes it up - these rows are not distinguished visually from the rows where it did not add data not in the input.)
What I mean is if every row had a date like "12 May 2002" and you wanted it turned into 2002.05.12 then it would be nice if it indicated when it added data. For example if one of the rows just read "15 May" then, since there is no year, it would not be completely absurd if it transformed into 2017.05.15 - or if all of the other data is 2002, then adding that. But I really think silently adding data that was not in the input is going too far. A transform shouldn't ever silently inject plausible data with no indication that this is interpolated. Bad things can result.
Otherwise great demo!
Re: Transform Data by Example [video]
#10Humans are really good at taking a vague description of a task and using a small number of examples to disambiguate it.
For example, "sort all of the folders, so that it Alan goes before Amy, etc". The rule ("sort") is pretty ambiguous, but one simple example in the context gives enough information to realise you probably mean alphabetically by first name.
Is there something like this example that could be combined with NLP to make things like these "intelligent assistants" we have now much more useful for data processing tasks?
It would be great to describe data manipulation to a machine the way that I would describe it to a colleague: give an overview of an algorithm, watch how they interpret it, and correct with a couple of examples in a feedback loop. Currently describing such things for a machine requires writing the algorithm manually in a programming language.