Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine Learning for Systems and Systems for Machine Learning [pdf]

learningsys.org

Post reply on HN
Print view View original

41–50 of 50 posts

Re: Machine Learning for Systems and Systems for Machine Learning [pdf]

#41

Post by killjoywashere »

31reasons wrote: post #39
cs702 wrote: post #31

TPUs are only one part of this eye-opening presentation. Skip to page 28, where Jeff starts talking about: * Using reinforcement learning so the computer can figure out how to parallelize code and models on its own. In experiments, the machine beats human-designed parallelization. * Replacing B-tree indices, hash maps, and Bloom filters with data-driven indices learned by deep learning models. In experiments, the lea…

So basically it will replace all heuristics/greedy optimization algorithms. I am wondering if ML can come up with better sorting algorithms, or I guess when you can use ML for end strategy of optimization you don't have to sort!

I think the genomics folks have been onboard with this for a couple years now.

Re: Machine Learning for Systems and Systems for Machine Learning [pdf]

#42

Post by quadrature »

novaRom wrote: post #28

I speculate that Google will sell TPUv2 for as less as 500 USD per PCIe card already in 2018. Nvidia's Volta TensorCores are essentially the same: 32-bit accumulators and 16-bit multipliers, but GPUs are more general-purpose which is not necessary for Deep Learning since most intensive operation is dot-product (y+=w*x).

I feel like the cloud play would be much stronger than entering the hardware market.

Re: Machine Learning for Systems and Systems for Machine Learning [pdf]

#43

Post by modeless »

argonaut wrote: post #21
cobookman wrote: post #7

Nvidia Titan V can do 110 TFLOPS, 12GB of 1.7 Gb/s Memory [1] and sells for 3,000$. TPU v2 does 180 TFLOPS, 64GB of 19.2Gb/s Memory [2]. That's a heck of a performance boost for a chip that's likely costing google way less than the nvidia flagship. [1] http://www.tomshardware.com/news/nvidia-titan-v-110-teraflop...

The TFLOPS numbers are not directly comparable. The TPUs use reduced precision in some areas, whereas I am guessing the Titan V numbers are based on single precision operations.

Titan V numbers are also reduced precision (16 bits), using their tensor cores.

Re: Machine Learning for Systems and Systems for Machine Learning [pdf]

#44

Post by nickpsecurity »

Great presentation. Far as application, I already thought this might be useful in lightweight, formal methods to spot problems and suggest corrections for failures in Rust's borrow checkers, separation logic on C programs, proof tactics, and static analysis tooling. For Rust example, the person might try to express a solution in the language that fails the borrow checker. If they can't understand why, they submit it to the system that attempts to spot where the problem is. The system might start with humans spotting it and restructuring the code to pass borrow checker. Every instance of those will feed into the learning system that might eventually do that on its own. There's also potential to use automated, equivalence checks/tests between user-submitted code and the AI's suggestions to help human-in-the-loop decide if it's worth review before passing onto the other person.

In hardware, both digital and analog designers seem to use lots of heuristics in how they design things. Certainly could help there. Might be especially useful in analog due to small number of experienced engineers available.

Re: Machine Learning for Systems and Systems for Machine Learning [pdf]

#45

Post by laythea »

cs702 wrote: post #31

TPUs are only one part of this eye-opening presentation. Skip to page 28, where Jeff starts talking about: * Using reinforcement learning so the computer can figure out how to parallelize code and models on its own. In experiments, the machine beats human-designed parallelization. * Replacing B-tree indices, hash maps, and Bloom filters with data-driven indices learned by deep learning models. In experiments, the lea…

"replacing human engineering"

Good summary, but someone still has to write the machine intelligence!

Re: Machine Learning for Systems and Systems for Machine Learning [pdf]

#47

Post by p1esk »

laythea wrote: post #45
cs702 wrote: post #31

TPUs are only one part of this eye-opening presentation. Skip to page 28, where Jeff starts talking about: * Using reinforcement learning so the computer can figure out how to parallelize code and models on its own. In experiments, the machine beats human-designed parallelization. * Replacing B-tree indices, hash maps, and Bloom filters with data-driven indices learned by deep learning models. In experiments, the lea…

"replacing human engineering" Good summary, but someone still has to write the machine intelligence!

Great comment. Fits most AGI discussions.

Re: Machine Learning for Systems and Systems for Machine Learning [pdf]

#48

Post by eternalban »

frankmcsherry wrote: post #38
cs702 wrote: post #31

TPUs are only one part of this eye-opening presentation. Skip to page 28, where Jeff starts talking about: * Using reinforcement learning so the computer can figure out how to parallelize code and models on its own. In experiments, the machine beats human-designed parallelization. * Replacing B-tree indices, hash maps, and Bloom filters with data-driven indices learned by deep learning models. In experiments, the lea…

In case anyone wants to check out some pre-history, back in 2002 Manfred Warmuth et al.[0] were using learning (Weighted Majority) to drive systems components like cache replacement policy. I'm not sure where the work went from there, but add it to the pile of techniques. [0]: https://users.soe.ucsc.edu/~sbrandt/papers/NIPS02.pdf

Thanks for the link. Very interesting. I found this [1] from 2015.

Reading your cite, the practical issue seems to me to be that the optimizer's memory footprint costs may in fact negate any benefit (e.g. ~40% over LRU) obtained in reducing cache misses.

My gut feeling is that this approach (for online systems) may work best with a hardware component (a card hosting the 'experts' and their virtual model e.g. the "virtual cache"). The distributed variant also seems worth exploring.

[1]: https://arxiv.org/pdf/1403.0388.pdf

Re: Machine Learning for Systems and Systems for Machine Learning [pdf]

#49

Post by shaklee3 »

EvgeniyZh wrote: post #13
shaklee3 wrote: post #8

Earlier quoted context omitted.

It's not clear to me how programmable the tpu is. I'm sure it's great at convolutions and matrix multiplies. Can it do anything else?

Neither do tensor cores

The tensor core is one part of the GPU. It has plenty of other capabilities.

Re: Machine Learning for Systems and Systems for Machine Learning [pdf]

#50

Post by eggie5 »

jamesblonde wrote: post #30
eggie5 wrote: post #22

Earlier quoted context omitted.

ballpark how much would it cost to train ImageNet (ILSVRC) on a std deep CNN arch (VGG or inception) on AWS using a p2 or p3?

Ballpark - 1100 dollars on AWS. 44hr 28min (from Dawnbench - http://dawn.cs.stanford.edu/benchmark/ ) on a DGX-1 (cost 24.48 dollars/hour on p3.16xlarge). https://aws.amazon.com/ec2/pricing/on-demand/ On a DL11 server, it will take about 60 hrs, and only cost you 15k upfront. The economics speak for themselves for fp32 training, at this moment in time.

I didn't know about the dawn project, thank you for the reference and figures.
Post reply on HN