Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unreasonable Ineffectiveness of Machine Learning in Computer Systems Research

sigarch.org

Post reply on HN
Print view View original

41–50 of 105 posts

Re: Unreasonable Ineffectiveness of Machine Learning in Computer Systems Research

#41

Post by bmh100 »

rattray wrote: post #40
austincheney wrote: post #34

Perhaps the biggest hurdle in this regard is the approach to machine learning. Nearly everything I have seen on machine learning is a primer on big data followed by a series of algorithms on making the best and smartest decision upon that mountain of data. This is completely the wrong approach. Machine learning can be done on a dime, provided the proper nurturing and environment, but you have to be willing to make so…

You're advocating an evolutionary approach, correct? Doesn't such an approach need lots of examples to trial, before it generalizes broadly? "Big data" is often shorthand for "lots of examples", no?

The poster is referring to control theory (often seen in ML as reinforcement learning), while also touching on the explore-exploit tradeoff in optimization more generally.

Re: Unreasonable Ineffectiveness of Machine Learning in Computer Systems Research

#42

Post by bmh100 »

dkarapetyan wrote: post #36

I think this is because the kinds of problems that arise in system design are logical and symbolic in nature and the current crop of "AI" has no symbolic reasoning capabilities. All the current hype is about pattern matching. Very good pattern matching but just pattern matching nonetheless. Whereas when constructing a compiler or a JIT it's more like what mathematicians do by setting down some axioms and exploring th…

It would actually be very straightforward to do so if the costs of testing solutions weren't so high. CPU architecture and JIT code can both be represented as unstructured (non-tabular) data. I even recall a circuit having been optimized by a genetic algorithm a while ago in an experiment. I also recall using LSTM to generate valid code from IIRC examples in Linux. Superiptimizatiom is also a relevant topic.

We just need better simulation tools or more resources.

Re: Unreasonable Ineffectiveness of Machine Learning in Computer Systems Research

#43

Post by visarga »

backpropaganda wrote: post #24
visarga wrote: post #8

Earlier quoted context omitted.

It imitates an old paper title: "The Unreasonable Effectiveness of Mathematics in the Natural Sciences" by Eugene Wigner (1960)

I thought Wigner was actually paying homage to Karpathy.

Few know that Karpathy was using a previous meme.

Re: Unreasonable Ineffectiveness of Machine Learning in Computer Systems Research

#44

Post by mturmon »

eellpp wrote: post #15
snackai wrote: post #10

Earlier quoted context omitted.

> but we are still decades (or more) away from full level-5 autonomy Decades? Are you really sure? This assumption seems quite uninformed to me. The Grand Challenge was 12 years ago, from then we evolved from Level 1 to Level 3. Back then there was no data, computing power wasn't available as today, chips (&sensors) can be designed an built within a few months or even less these days for far less money. Machine Learn…

It max 5 years or even less, if the problem is in engineering the solution. But if the problem is in underlying maths in building Level 5, then surely its decades

One of the lessons of these first steps toward autonomous driving is: if you have a big array of sensors, and plenty of on board computing, then many driving problems, especially in semi controlled circumstances, like open highways, are within our grasp now.

The sensors and computing are expensive now - especially lidar. But that's an engineering problem, as you suggest.

Re: Unreasonable Ineffectiveness of Machine Learning in Computer Systems Research

#45

Post by pm215 »

mcguire wrote: post #2

Branch predictions are an interesting use, although I'm wondering how expensive a misprediction really is. But this: " Another example is the use of regression techniques from machine learning to build models of program behavior. If I replace 64-bit arithmetic with 32-bit arithmetic in a program, how much does it change the output of the program and how much does it reduce energy consumption? For many programs, it is…

Wikipedia reckons 10-20 cycle penalty for a branch mispredict (which sounds plausible given it fills the pipeline with useless junk). Given that branches are quite common, that's painful enough to want to avoid, but definitely not so painful that you'd want to devote as much silicon to solving it as you do to, say, L1 cache.

I do recall a bit of research (published by a Nokia R&D team I think) that reckoned you could get a mostly-ok performance estimate by tracking about half a dozen indicators including instructions executed, cache misses, tlb misses and brancb mispredicts and weighting them appropriately. The trouble there is nobody wants a performance model that's right 90% of the time but significantly wrong 10% of the time with no way to tell if the workload you want to test is in the 10%. But it's an indication of the importance of branch prediction still, I think.

Re: Unreasonable Ineffectiveness of Machine Learning in Computer Systems Research

#46

Post by freeone3000 »

sroussey wrote: post #30
xfs wrote: post #25

Earlier quoted context omitted.

> none of us in the automobile or IT industries are close to achieving true Level 5 autonomy - Gill Pratt, Toyota Research Institute http://spectrum.ieee.org/cars-that-think/transportation/self... > It will be 25 years before self-driving cars take off in America - Bill Gurley, Uber investor http://www.cnbc.com/2017/04/06/bill-gurley-uber-investor-sel...

At the point of level 4, the roads and conditions will change to better accommodate, likely pushing cities and villages farther apart (particularly in America).

How would they be pushed father apart? Where would they be pushed to? America isn't exactly known for its density.

Re: Unreasonable Ineffectiveness of Machine Learning in Computer Systems Research

#47

Post by dkarapetyan »

bmh100 wrote: post #42
dkarapetyan wrote: post #36

I think this is because the kinds of problems that arise in system design are logical and symbolic in nature and the current crop of "AI" has no symbolic reasoning capabilities. All the current hype is about pattern matching. Very good pattern matching but just pattern matching nonetheless. Whereas when constructing a compiler or a JIT it's more like what mathematicians do by setting down some axioms and exploring th…

It would actually be very straightforward to do so if the costs of testing solutions weren't so high. CPU architecture and JIT code can both be represented as unstructured (non-tabular) data. I even recall a circuit having been optimized by a genetic algorithm a while ago in an experiment. I also recall using LSTM to generate valid code from IIRC examples in Linux. Superiptimizatiom is also a relevant topic. We just…

I have also seen genetic algorithms used for these kinds of optimization problems. In fact there is a module in postgres that uses genetic algorithms to optimize query plans (https://www.postgresql.org/docs/9.6/static/geqo-pg-intro.htm...).

But I don't put genetic algorithms in the same bucket. Genetic algorithms are a different breed of optimization algorithm compared to neural nets and gradient descent which is what the modern crop of AI is basically all about.

Re: Unreasonable Ineffectiveness of Machine Learning in Computer Systems Research

#48

Post by saosebastiao »

Indirectly they have helped quite a bit. Some of the most advanced mathematical and symbolic solvers (MIP, IP, LP, CP, SAT, SMT) have slowly been incorporating machine learning to advance their capabilities. Their use cases in these solvers include: branch prediction, branch selection, constraint evaluation order, solver type selection, search strategy selection, cost estimations for column generation strategies, problem classification, solve time estimation, etc.

And since advancements in our abilities at solving symbolic and mathematical problems have directly enabled the current research in PLT and formal systems, I see no reason to discount the impact ML has had in pushing that frontier.

Re: Unreasonable Ineffectiveness of Machine Learning in Computer Systems Research

#49

Post by jacquesm »

mturmon wrote: post #44
eellpp wrote: post #15

Earlier quoted context omitted.

It max 5 years or even less, if the problem is in engineering the solution. But if the problem is in underlying maths in building Level 5, then surely its decades

One of the lessons of these first steps toward autonomous driving is: if you have a big array of sensors, and plenty of on board computing, then many driving problems, especially in semi controlled circumstances, like open highways, are within our grasp now. The sensors and computing are expensive now - especially lidar. But that's an engineering problem, as you suggest.

The problems in automated driving are not in controlled and normal circumstances but in all the edge cases and exceptions.

Re: Unreasonable Ineffectiveness of Machine Learning in Computer Systems Research

#50

Post by jacquesm »

bmh100 wrote: post #42
dkarapetyan wrote: post #36

I think this is because the kinds of problems that arise in system design are logical and symbolic in nature and the current crop of "AI" has no symbolic reasoning capabilities. All the current hype is about pattern matching. Very good pattern matching but just pattern matching nonetheless. Whereas when constructing a compiler or a JIT it's more like what mathematicians do by setting down some axioms and exploring th…

It would actually be very straightforward to do so if the costs of testing solutions weren't so high. CPU architecture and JIT code can both be represented as unstructured (non-tabular) data. I even recall a circuit having been optimized by a genetic algorithm a while ago in an experiment. I also recall using LSTM to generate valid code from IIRC examples in Linux. Superiptimizatiom is also a relevant topic. We just…

Genetic algorithms are not neural nets though.
Post reply on HN