Deep learning with text: Learning when to skim and when to read

Post by alrojo » Thu, Mar 16, 2017, 6:23 PM UTC

itschekkers wrote: post #19

sixhobbits wrote: post #14

Earlier quoted context omitted.

A 'batch' is how much of the data you put in memory at once while training the NN. To train even a small language model, you'll go through 1000s of batches, so the time difference is way bigger than it sounds. I agree a more practical example would have been nice -- maybe it'll come out in the paper.

my impression was that this was about the time taken to make each prediction, not to train the model? and yep, looking forward to the paper!

It was based on test time prediction, so given you have received a sentence, how fast does it take to compute the prediction with either a bag-of-words or an LSTM.

When you say practical example, would that be in the scenario that you have an API server running? So to consider such costs as latency, data transfer, API overhead etc.?

Thanks for your feedback!

Post by alrojo » Thu, Mar 16, 2017, 6:34 PM UTC

itschekkers wrote: post #9

cool paper, i enjoyed following the post (and really appreciated the effort put into the bokeh viz!) i wonder if this would have been improved by being clearer about the motivation. the authors frame it as though the ~60ms penalty for using the LSTM for prediction is a huge burden, and i can imagine situations where it is. however, it seems like if this is the case, we need some real life/"scaled out" examples of how…

In the Baseline section, you will find a sentence with the following: "However, there is a cost to using the strategies. We have to run all of the sentences through the bag-of-words model first, to determine if we should use the bag-of-words or the LSTM." followed by some math that takes the added cost into account, which is also what we based our plots/results on.

Re: Deep learning with text: Learning when to skim and when to read

Re: Deep learning with text: Learning when to skim and when to read