Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Growing a Compiler: Getting to Machine Learning from a General Purpose Compiler

docs.google.com

Post reply on HN
Print view View original

11–20 of 40 posts

Re: Growing a Compiler: Getting to Machine Learning from a General Purpose Compiler

#11

Post by mlevental »

SilasX wrote: post #7
KenoFischer wrote: post #3

Earlier quoted context omitted.

This is a talk we gave at the "Compilers for Machine Learning" workshop at CGO this past Sunday. Obviously the slides are missing some of the context of the presentation, but the core argument is that one shouldn't build compilers for machine learning, but rather build general purpose compilers that are flexible and extensible. From there, you can easily get all the same benefits as dedicated machine learning compile…

How ... whatever was the reason for thinking that you needed a ML-specific compiler? Do people also think you need a timecard-tracking-specific compiler?

do you know how modern ml works? do you know that every functional unit in a net needs to be differentiable and so needs to carry around either dual numbers (forward mode) or adjoints (reverse mode)? it's not as simple as just writing a math library.

Re: Growing a Compiler: Getting to Machine Learning from a General Purpose Compiler

#12

Post by ViralBShah »

mlevental wrote: post #11
SilasX wrote: post #7

Earlier quoted context omitted.

How ... whatever was the reason for thinking that you needed a ML-specific compiler? Do people also think you need a timecard-tracking-specific compiler?

do you know how modern ml works? do you know that every functional unit in a net needs to be differentiable and so needs to carry around either dual numbers (forward mode) or adjoints (reverse mode)? it's not as simple as just writing a math library.

There's an extensive discussion of AD in the slides (the topic of this HN post), and how it is done in Julia. Precisely because it is not as simple as writing a math library is why you need language and compiler support.

Re: Growing a Compiler: Getting to Machine Learning from a General Purpose Compiler

#14

Post by chrisseaton »

SilasX wrote: post #7
KenoFischer wrote: post #3

Earlier quoted context omitted.

This is a talk we gave at the "Compilers for Machine Learning" workshop at CGO this past Sunday. Obviously the slides are missing some of the context of the presentation, but the core argument is that one shouldn't build compilers for machine learning, but rather build general purpose compilers that are flexible and extensible. From there, you can easily get all the same benefits as dedicated machine learning compile…

How ... whatever was the reason for thinking that you needed a ML-specific compiler? Do people also think you need a timecard-tracking-specific compiler?

> How ... whatever was the reason for thinking that you needed a ML-specific compiler?

As the slides say, you need the program to be differentiable. Do general purpose compilers make it easy to automatically differentiate a program? No, not until very recent work. So people thought a better idea until we figured out how to do that was ML-specific compilers and frameworks.

> Do people also think you need a timecard-tracking-specific compiler?

No, nobody thinks this, because timecard-tracking does not need any special properties such as being differentiable.

The answer isn't as crazy as your extremely snarky question makes it out to be, is it?

Re: Growing a Compiler: Getting to Machine Learning from a General Purpose Compiler

#15

Post by seanmcdirmid »

At first, I read this as "getting a general purpose compiler from machine learning"...but I guess we aren't there yet.

I do wonder, however, how full differentiable programming languages will be supported by compilers in the near future.

Re: Growing a Compiler: Getting to Machine Learning from a General Purpose Compiler

#16

Post by mlevental »

ViralBShah wrote: post #12
mlevental wrote: post #11

Earlier quoted context omitted.

do you know how modern ml works? do you know that every functional unit in a net needs to be differentiable and so needs to carry around either dual numbers (forward mode) or adjoints (reverse mode)? it's not as simple as just writing a math library.

There's an extensive discussion of AD in the slides (the topic of this HN post), and how it is done in Julia. Precisely because it is not as simple as writing a math library is why you need language and compiler support.

yes that's what i was pointing out the poster i responded to

Re: Growing a Compiler: Getting to Machine Learning from a General Purpose Compiler

#17

Post by xiaodai »

It's unfortunate but google docs are blocked at work. So why am I reading this at work? Cos it might be useful for my work.

I do wish more slides are hosted elsewhere not google docs. Because some companies are worried about the use of google docs especially if the data is sensitive e.g. banking and insurance.

Re: Growing a Compiler: Getting to Machine Learning from a General Purpose Compiler

#18

Post by samcodes »

SilasX wrote: post #7
KenoFischer wrote: post #3

Earlier quoted context omitted.

This is a talk we gave at the "Compilers for Machine Learning" workshop at CGO this past Sunday. Obviously the slides are missing some of the context of the presentation, but the core argument is that one shouldn't build compilers for machine learning, but rather build general purpose compilers that are flexible and extensible. From there, you can easily get all the same benefits as dedicated machine learning compile…

How ... whatever was the reason for thinking that you needed a ML-specific compiler? Do people also think you need a timecard-tracking-specific compiler?

Good point, timecard-tracking software and general purpose differentiable computing languages are roughly the same level of complexity.

Re: Growing a Compiler: Getting to Machine Learning from a General Purpose Compiler

#19

Post by ViralBShah »

xiaodai wrote: post #17

It's unfortunate but google docs are blocked at work. So why am I reading this at work? Cos it might be useful for my work. I do wish more slides are hosted elsewhere not google docs. Because some companies are worried about the use of google docs especially if the data is sensitive e.g. banking and insurance.

Here's a PDF hosted elsewhere for those who can't see the Google Slides at work: https://juliacomputing.com/assets/pdf/CGO_C4ML_talk.pdf

Re: Growing a Compiler: Getting to Machine Learning from a General Purpose Compiler

#20

Post by throwawayjava »

xiaodai wrote: post #17

It's unfortunate but google docs are blocked at work. So why am I reading this at work? Cos it might be useful for my work. I do wish more slides are hosted elsewhere not google docs. Because some companies are worried about the use of google docs especially if the data is sensitive e.g. banking and insurance.

Wow. Why?
Post reply on HN