Live data from Hacker News

Machine Learning for Systems and Systems for Machine Learning [pdf]

learningsys.org

31–40 of 50 posts

Re: Machine Learning for Systems and Systems for Machine Learning [pdf]

#31
TPUs are only one part of this eye-opening presentation. Skip to page 28, where Jeff starts talking about:

* Using reinforcement learning so the computer can figure out how to parallelize code and models on its own. In experiments, the machine beats human-designed parallelization.

* Replacing B-tree indices, hash maps, and Bloom filters with data-driven indices learned by deep learning models. In experiments, the learned indices outperform the usual stalwarts by a large margin in both computing cost and performance, and are auto-tuning.

* Using reinforcement learning to manage datacenter power. Machine intelligence outperforms human-designed energy-management policies.

* Using machine intelligence to replace user-tunable performance options in all software systems, eliminating the need to tweak them with command line parameters like --num-threads=16, --max-memory-use=104876, etc. Machine intelligence outperforms hand-tuning.

* Using machine intelligence for all tasks currently managed with heuristics. For example, in compilers: instruction scheduling, register allocation, loop nest parallelization strategies, etc.; in networking: TCP window size decisions, backoff for retransmits, data compression, etc.; in operating systems: process scheduling, buffer cache insertion/replacement, file system prefetching, etc.; in job scheduling systems: which tasks/VMs to co-locate on same machine, which tasks to pre-empt, etc.; in ASIC design: physical circuit layout, test case selection, etc. Machine intelligence outperforms human heuristics.

IN SHORT: machine intelligence (today, that means deep learning and reinforcement learning) is going to penetrate and ultimately control EVERY layer of the software stack, replacing human engineering with auto-tuning, self-improving, better-performing code.

Eye-opening.

Re: Machine Learning for Systems and Systems for Machine Learning [pdf]

#32
post #31

TPUs are only one part of this eye-opening presentation. Skip to page 28, where Jeff starts talking about: * Using reinforcement learning so the computer can figure out how to parallelize code and models on its own. In experiments, the machine beats human-designed parallelization. * Replacing B-tree indices, hash maps, and Bloom filters with data-driven indices learned by deep learning models. In experiments, the lea…

I remember there was a joke that in google's code base, there are more Bayesian cases than if...else...

Re: Machine Learning for Systems and Systems for Machine Learning [pdf]

#33
post #31

TPUs are only one part of this eye-opening presentation. Skip to page 28, where Jeff starts talking about: * Using reinforcement learning so the computer can figure out how to parallelize code and models on its own. In experiments, the machine beats human-designed parallelization. * Replacing B-tree indices, hash maps, and Bloom filters with data-driven indices learned by deep learning models. In experiments, the lea…

Good summary. Some systems groups are already going in this direction. PeletonDB is trying to use DL to build a self-tuning DB https://github.com/cmu-db/peloton We have been trying to self-tune resource management decisions in Hadoop YARN using deep learning.

Re: Machine Learning for Systems and Systems for Machine Learning [pdf]

#34
post #31

TPUs are only one part of this eye-opening presentation. Skip to page 28, where Jeff starts talking about: * Using reinforcement learning so the computer can figure out how to parallelize code and models on its own. In experiments, the machine beats human-designed parallelization. * Replacing B-tree indices, hash maps, and Bloom filters with data-driven indices learned by deep learning models. In experiments, the lea…

Great summary. Thanks and agree this is a big deal.

Re: Machine Learning for Systems and Systems for Machine Learning [pdf]

#35

Nvidia Titan V can do 110 TFLOPS, 12GB of 1.7 Gb/s Memory [1] and sells for 3,000$. TPU v2 does 180 TFLOPS, 64GB of 19.2Gb/s Memory [2]. That's a heck of a performance boost for a chip that's likely costing google way less than the nvidia flagship. [1] http://www.tomshardware.com/news/nvidia-titan-v-110-teraflop...

The TFLOPS numbers are not directly comparable. The TPUs use reduced precision in some areas, whereas I am guessing the Titan V numbers are based on single precision operations.

If all you need is reduced precision, then it’s fair to compare the two. I’d also assume memory bandwidth matters just as much as TFLOPS for ml workloads.

Re: Machine Learning for Systems and Systems for Machine Learning [pdf]

#36
post #32
post #31

TPUs are only one part of this eye-opening presentation. Skip to page 28, where Jeff starts talking about: * Using reinforcement learning so the computer can figure out how to parallelize code and models on its own. In experiments, the machine beats human-designed parallelization. * Replacing B-tree indices, hash maps, and Bloom filters with data-driven indices learned by deep learning models. In experiments, the lea…

I remember there was a joke that in google's code base, there are more Bayesian cases than if...else...

It was a quote in Joel Spolsky's blog.

> A very senior Microsoft developer who moved to Google told me that Google works and thinks at a higher level of abstraction than Microsoft. “Google uses Bayesian filtering the way Microsoft uses the if statement,” he said

https://www.joelonsoftware.com/2005/10/17/news-121/

Re: Machine Learning for Systems and Systems for Machine Learning [pdf]

#37
post #31

TPUs are only one part of this eye-opening presentation. Skip to page 28, where Jeff starts talking about: * Using reinforcement learning so the computer can figure out how to parallelize code and models on its own. In experiments, the machine beats human-designed parallelization. * Replacing B-tree indices, hash maps, and Bloom filters with data-driven indices learned by deep learning models. In experiments, the lea…

Ok, I can understand how a bloom filter can be replaced by a neural network predictive model. You could actually train it while stuff gets added. This would make adding somewhat more expensive, but ...

Ah so it appears they're advocating using neural networks as index functions to sorted arrays (hashmaps are simply sorted by hash instead of by something in the data).

So what they do is they take a FIXED set of data that you want to quickly lookup in, already sorted, train a model (2 layer 32 width, relu activation is one architecture, but they also train sequences of models, HUGE changes to error (as the cost of max and min error are huge, you minimize max error rather than average error)).

They have the following brilliant insight : an index over a database (which gives the position of the data given the search key) is a CDF (cumulative distribution function) ! That's brilliant ! Of course it is !

And of course, this is Google. Once you have an index trained (which is a linear operation), you can translate the neural network model directly into C++, and compile it into machine instructions that don't depend on anything like tensorflow libraries. The resulting code can be pasted into anything you want. This may work fast, but seems less then entirely practical ... although I guess you could do the same in Java far easier and you could just include that code.

Paper here: http://learningsys.org/nips17/assets/slides/dean-nips17.pdf

Re: Machine Learning for Systems and Systems for Machine Learning [pdf]

#38
post #31

TPUs are only one part of this eye-opening presentation. Skip to page 28, where Jeff starts talking about: * Using reinforcement learning so the computer can figure out how to parallelize code and models on its own. In experiments, the machine beats human-designed parallelization. * Replacing B-tree indices, hash maps, and Bloom filters with data-driven indices learned by deep learning models. In experiments, the lea…

In case anyone wants to check out some pre-history, back in 2002 Manfred Warmuth et al.[0] were using learning (Weighted Majority) to drive systems components like cache replacement policy. I'm not sure where the work went from there, but add it to the pile of techniques.

[0]: https://users.soe.ucsc.edu/~sbrandt/papers/NIPS02.pdf

Re: Machine Learning for Systems and Systems for Machine Learning [pdf]

#39
post #31

TPUs are only one part of this eye-opening presentation. Skip to page 28, where Jeff starts talking about: * Using reinforcement learning so the computer can figure out how to parallelize code and models on its own. In experiments, the machine beats human-designed parallelization. * Replacing B-tree indices, hash maps, and Bloom filters with data-driven indices learned by deep learning models. In experiments, the lea…

So basically it will replace all heuristics/greedy optimization algorithms. I am wondering if ML can come up with better sorting algorithms, or I guess when you can use ML for end strategy of optimization you don't have to sort!

Re: Machine Learning for Systems and Systems for Machine Learning [pdf]

#40
post #31

TPUs are only one part of this eye-opening presentation. Skip to page 28, where Jeff starts talking about: * Using reinforcement learning so the computer can figure out how to parallelize code and models on its own. In experiments, the machine beats human-designed parallelization. * Replacing B-tree indices, hash maps, and Bloom filters with data-driven indices learned by deep learning models. In experiments, the lea…

Ok, I can understand how a bloom filter can be replaced by a neural network predictive model. You could actually train it while stuff gets added. This would make adding somewhat more expensive, but ... Ah so it appears they're advocating using neural networks as index functions to sorted arrays (hashmaps are simply sorted by hash instead of by something in the data). So what they do is they take a FIXED set of data t…

[deleted]
Post reply on HN