https://www.slideshare.net/macanbet88/agen-sbobet-terpercaya...
https://www.scribd.com/document/435656590/Agen-Sbobet-Macanb...
http://www.lulu.com/shop/sisil-natalia/agen-sbobet/ebook/pro...
https://app.box.com/s/4fkt9dgtqrcxyc12cxl4mrnjkpidexfy
https://www.4shared.com/web/preview/pdf/Ngf12-MPiq
https://show.zohopublic.com/publish/hj9wp501bfe9f5d294545b9f...
How to recognize AI snake oil [pdf]
121–130 of 364 posts
Re: How to recognize AI snake oil [pdf]
#122I've worked on data related to healthcare and security (and sometimes both) for quite a while now, and I think there are a couple of general contextual themes, where, if present, means that you have to be extremely careful about applying "AI" (some kind of ML in most cases):
(a) where there's a high cost for incorrect predictions (e.g. criminal recidivism, educational attainment, terrorist attacks, etc)
(b) where causation is important (e.g. drug efficacy and safety, educational attainment, almost all of healthcare)
(c) where you're in an adversarial domain (e.g. fraud, cybersecurity, security in general)
(d) where high technical performance (precision/recall/F1/etc) isn't correlated with predictiveness of what you're actually looking for (much of healthcare)
In healthcare and security, there's starting to be an awareness of the snake-oil that's out there, but I still run into people regularly who ask for a magic algorithm that predicts patient outcomes or a security breach.
Re: How to recognize AI snake oil [pdf]
#123Re: How to recognize AI snake oil [pdf]
#124Over the years my heuristic has turned into: "Did the team formulate their problem as a supervised learning problem?" - If not it's probably BS. In longform if anyone is interested https://medium.com/@marksaroufim/can-deep-learning-solve-my-... EDIT: I would consider autoencoders, word2vec, Reinforcement Learning examples of turning a different problem into a supervised learning problem EDIT 2: Social functions like…
Auto-encoders have been more successful in fraud and anomaly detection then supervised methods. For the uninitiated: the basic concept is to reduce the feature space (i.e. the things you know) to a lower dimensional space, then decode back into the original space. When enough differences arise between the original and reconstructed variables, the event may be flagged for a human to review (or some triage process).
I wonder if a similar approach can be used for a classification task where one or more classes have only few training examples (those would be similar to "anomalies", I suppose).
Re: How to recognize AI snake oil [pdf]
#125Over the years my heuristic has turned into: "Did the team formulate their problem as a supervised learning problem?" - If not it's probably BS. In longform if anyone is interested https://medium.com/@marksaroufim/can-deep-learning-solve-my-... EDIT: I would consider autoencoders, word2vec, Reinforcement Learning examples of turning a different problem into a supervised learning problem EDIT 2: Social functions like…
It's hard to verbalize this, most of it is "intuition" but I think it boils down to "supervised learning is BS."
Humans are smarter than computers. How can a human teach a computer how to do something when the human itself can't teach another human that something?
We haven't solved that problem. The snake is eating its tail.
You can't teach a human how to do something when the methodology to do that is the student trying something and the teacher saying "Yes" or "No".
Well.... why? Why is it yes or why is it no? What is the difference between what the human or the computer, or in general, the student, did and what is good or correct? And then you still have to define "good" and many times that means waiting, in the case of the PDF linked to above, perhaps many years to determine if the employee the AI picked, turned out to be a good employee or not.
And how do you determine that? How do you know if an employee is good or not? We haven't even figured that out yet.
How can we create an AI to pick good employees if human beings don't know how to do that?
Supervised learning isn't going to solve any problem, if that problem isn't solved or perhaps even solvable at all.
In other words, over the years, my heuristic has turned into, "Has a human being solved this problem?" If not, then AI software that claims to is BS.
Re: How to recognize AI snake oil [pdf]
#126Earlier quoted context omitted.
An old team I advised for a while also compared model performance to a randomised baseline model. What they didn't seem to get however was that a randomised baseline model would beat a randomised baseline model on a naive comparison 50% of the time, so their understanding of randomness/statistical significance/performance metrics was way off. So while they believed they were also testing their models before presentin…
I am confused, if a new model beats randomly selected randomised model 100% of time for each experiment why does it matter if randomised model beats other randomised models? Are they only comparing against the subset of worst randomised models?
I think he's saying something like the following:
1/ the team implemeted a naive baseline
2/ they implemeted a more sophisticated model that depended on some parameter p
3/ for 100 different values of p, they examined its performance, and picked the model with the best performance
Now they're not quite subject to the multiple comparisons problem there, since the models with different values of p aren't independent from one another. But they're not not suffering from it either. It mostly depends on the model. But it's a very easy mistake to make. I'd say many many academic papers make the same mistake.
Re: How to recognize AI snake oil [pdf]
#127Earlier quoted context omitted.
The San Jose Mercury News had a weather-forecasting contest. It was won one year by a 12-year-old, who's algorithm was "The weather tomorrow will be the same as the weather today". A kind of AI I guess.
Using a markov chain with your stochastic matrix set to I...
Using “a I”
Re: How to recognize AI snake oil [pdf]
#128I don't have time to read the entire paper but I would like to share an anecdote. I worked at a company with a well staffed/funded machine learning team. They were in charge of recommendation systems - think along the lines of youtube up next videos. My team wanted better recommendations (really, less editorial intensive) so the ML team spent weeks crafting 12 or more variants of their recommendation system for our c…
I have had a similar experience, but I really do think it points more to the team than it does to the efficacy of machine learning. I was on a team of extremely intelligent people, but they were very academic, with minimal practical coding skills, refusals to use version control etc. By academic, I mean graduates from the number one university in their respective fields, top publications, etc. They produced very little actual value. Great theoretical ideas, in depth understanding of different methods of optimization, etc.
The team I was on before that one was a bunch of scrappy engineers from Poland, India and the USA with no graduate degrees, but 20 years coding and distributed systems experience each. The difference in problem solving ability, the speed at which they moved, broke down problems, tried out different methods, was staggering.
I think ML is suffering from a prestige problem, and many companies are suffering for it. The wrong people are being hired and promoted, with business leaders calling the shots on who runs machine learning projects without fully understanding who can actually deliver.
Re: How to recognize AI snake oil [pdf]
#129I don't have time to read the entire paper but I would like to share an anecdote. I worked at a company with a well staffed/funded machine learning team. They were in charge of recommendation systems - think along the lines of youtube up next videos. My team wanted better recommendations (really, less editorial intensive) so the ML team spent weeks crafting 12 or more variants of their recommendation system for our c…
This reminds me of a job interview I was on. I was asked about how I would use AI/machine learning for their problem space. Since they seemed to be smart and level-headed, I answered honestly, "Pick something unimportant, use a machine learning algorithm just to get familiar with the tools, ignore the result unless it happens to work, then put machine learning in your marketing materials. But keep track of it, and if it is useful to you in 5 years, be sure to use it for real."
They said, "That's about what we concluded except that we didn't get around to actually doing that pilot project yet."
I got the job. :-)
Re: How to recognize AI snake oil [pdf]
#130I worked at a place that was selling ML powered science instrument output analysis. It did not work at all (fake it till you make it is normal, was told). So there was a person in the loop (machine output -> internet -> person doing it manually pretending to be machine -> internet -> report app). The joke was “organic neural net.” Theranos of the North! ML is a great and powerful pattern matcher (talking about NN not…
We won’t ever have an AI winter like in the 70s again. A lot of ML is already very useful across many domains (computer vision, NLP, advertising, etc). Back then, there was almost no personal computing, almost no internet, smol data, and so on. Stuff you need for ML to be useful and used. So what if some corporate hack calls linear regression “AI”? The results speak for themselves. The ML genie is too profitable to g…
Didn't linear regression used to be called "AI" as recently as a decade ago?