71–80 of 99 posts
Is anyone else at least a bit worried about a bunch of developers running around doing "machine learning" without much understanding of mathematics and probability? E.g. consider the creation of fragile models that overfit data being used in finance, infrastructure, medicine, etc.
I would love to see something like this in Elixir :)
Is anyone else at least a bit worried about a bunch of developers running around doing "machine learning" without much understanding of mathematics and probability? E.g. consider the creation of fragile models that overfit data being used in finance, infrastructure, medicine, etc.
A decent CS undergrad degree decade ago included abstract math concepts. I took Engineering math, Information Theory, Numerical analysis, Probability, Simulation in my sophomore and Junior years. NLP and AI were electives in Senior year. As a Junior, we were building toy programs that do Operations research type of work - solving linear equations via various matrix operations, design optimal queue processes based on…
E.g - no probability class that doesn't require analysis 1 and 2 is truly a probability class.
Earlier quoted context omitted.
The difference is that the software or the elevator will work but the statistical model is wrong and doesn't work. It is like the elevator only lift people above 120 and below 90 and for the others it just don't work or take you to the wrong floor.
> The difference is that the software... will work Lots of software doesn't work. Is there a substantial difference between putting an overfitting model in production, and putting a poorly tested program in production?
Is anyone else at least a bit worried about a bunch of developers running around doing "machine learning" without much understanding of mathematics and probability? E.g. consider the creation of fragile models that overfit data being used in finance, infrastructure, medicine, etc.
I'm a little bit worried. At least at the same level as when I see a bunch of developers compiling programs without much understanding of what an LL(k) parser does, or how a pushdown automaton works, or what a Turing machine is. I usually feel the same every time I see an elevator without a liftman, don't compute a square root by hand, or hear about Google self-driving cars.
Your comment is a bit obtuse, developers are not creating parsers and compilers. Elevators and calculators are very robust technologies that already work.
What I worry about is a new wave of engineers and developers thinking they understand statistical models and then proceeding to work at the big banks and have their models blow things up. If PhDs can make such disastrous non-robust models, how on earth is a random developer who took a summer course not going to do the same?
Now if the banks actually failed on their own, then by natural selection the less skilled would be out of jobs and people would stop trying to "short-cut" gaining this type of knowledge. But that's not what happens. Academics keep writing papers and hyping up specific techniques for which they can give conferences on, and the taxpayer bails out the idiots at the top.
Earlier quoted context omitted.
I think there might be. When ML fails the only individual capable of noticing is someone who understands the math. When code breaks often the "lay" user notices. The result is obvious to a novice. When ML fails it looks like a duck, quakes like a duck but after multiple years of study its immediately recognizable as an antelope. Though to disagree with my own point, security vulnerabilities have a similar profile. In…
> "When code breaks often the "lay" user notices. The result is obvious to a novice." That depends "how" it breaks. As a novice coder myself, I've had things go wrong that I don't notice or can't identify, and it looks like my program is running fine. I think that's the parent's point: it might be stupid to implement crappy macho learning models into production, but it isn't worrisome. It's expected.
Is anyone else at least a bit worried about a bunch of developers running around doing "machine learning" without much understanding of mathematics and probability? E.g. consider the creation of fragile models that overfit data being used in finance, infrastructure, medicine, etc.
Java and Scala? Who uses that in ML? Python has long been the best language for ML, with some competition from Matlab.
Earlier quoted context omitted.
I'm a little bit worried. At least at the same level as when I see a bunch of developers compiling programs without much understanding of what an LL(k) parser does, or how a pushdown automaton works, or what a Turing machine is. I usually feel the same every time I see an elevator without a liftman, don't compute a square root by hand, or hear about Google self-driving cars.
Well someone clearly has a horse in this race. Your comment is a bit obtuse, developers are not creating parsers and compilers. Elevators and calculators are very robust technologies that already work. What I worry about is a new wave of engineers and developers thinking they understand statistical models and then proceeding to work at the big banks and have their models blow things up. If PhDs can make such disastro…
I worked in the finance sector as a (lead) developer on a production use trading system for quite some years. None of the developers had formal math or statistics knowledge to the extent required to develop this system.
This didn't really bother anyone in the least despite the fact a mistake could cause a loss of millions of dollars in practically no time at all...
The reason for this wasn't because anyone was ignorant enough to think the programmers knew what was going on. It was because it was a finance company that also employed plenty of mathematicians, statisticians, physicists, and other's with the proper math/stats background. The programmers wrote the code, but the math/stats people wrote the business rules and the formulas and extensively tested that the system worked correctly against a large enough variety of models with expected outcomes that they were able to have sufficient confidence that the system reward/risk measurements were appropriate.
So my answer would be no; I don't find this all that troublesome. We can't be experts at everything and smart companies realize this so they should be creating teams with the correct skillset to be successful.