Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

IBM halting sales of Watson AI tool for drug discovery

statnews.com

Post reply on HN
Print view View original

161–170 of 178 posts

Re: IBM halting sales of Watson AI tool for drug discovery

#161

Post by joel_ms »

shshhdhs wrote: post #153
pjmorris wrote: post #128

Earlier quoted context omitted.

> I now think it’s madness for anyone to be in charge of anything if they don’t have relevant experience in the relevant domain. This. In support of that idea, here's C.A.R. Hoare, from 'The Emperor's Old Clothes', the 1980 Turing Award lecture: 'At last, there breezed into my office the most senior manager of all, a general manager of our parent company, Andrew St. Johnston. I was surprised that he had even heard of…

What was the solution here? To learn the Elliott 503 Mark II more deeply?

Given the speaker, I'm guessing the suggestion is that the incident sowed the seed for him to develop Hoare Logic[0]

[0] https://en.wikipedia.org/wiki/Hoare_logic

Re: IBM halting sales of Watson AI tool for drug discovery

#162

Post by snaky »

chillacy wrote: post #147
foobiekr wrote: post #50

Slowly, I am noticing that people are starting to question the abilities of machine learning systems and, in particular, seeing a lot of the transparent AI-washing that is now going on for systems that aren't AI at all. This is one of the most positive changes in the tech industry, honestly.

I've seen startups with zero AI (beyond rule based algorithms) call their product AI because it attracts investors. So I can vouch for that.

Like about half of startups do.

https://www.theverge.com/2019/3/5/18251326/ai-startups-europ...

Re: IBM halting sales of Watson AI tool for drug discovery

#163

Post by acdc4life »

randcraw wrote: post #158
acdc4life wrote: post #132

Earlier quoted context omitted.

The biggest issue I have is: "To do this we developed an entirely new system using convolutional neural networks to turn a few seconds of audio into a unique “fingerprint.” Why did you pick a neural network? What mathematical properties does a neural network have that makes it appealing to this problem? How were the networks trained? Back propagation? It doesn't converge, and worse learning weights for a new batch ca…

I agree with spaced-out. A neural net can capture all those smaller eigenvectors in the signal that are routinely thrown away during traditional feature engineering, like what you describe. When the number of training samples grows big enough, those factors with marginal contribution become significant and allow higher levels of accuracy in prediction or classification than are possible when curating features manuall…

> A neural net can capture all those smaller eigenvectors in the signal that are routinely thrown away during traditional feature engineering

What on earth are you talking about?

>Deep nets are here to stay.

Maybe in silicon valley for consumer products in things like snapchat and siri. They won't work for industrial problems.

Re: IBM halting sales of Watson AI tool for drug discovery

#164

Post by acdc4life »

spaced-out wrote: post #155
acdc4life wrote: post #132

Earlier quoted context omitted.

The biggest issue I have is: "To do this we developed an entirely new system using convolutional neural networks to turn a few seconds of audio into a unique “fingerprint.” Why did you pick a neural network? What mathematical properties does a neural network have that makes it appealing to this problem? How were the networks trained? Back propagation? It doesn't converge, and worse learning weights for a new batch ca…

You'll never be able to develop features with the heuristic methods you described that will work as well as the features learned by a neural net.

Huh, a quick Google search gave me: https://www.ee.columbia.edu/~dpwe/papers/Wang03-shazam.pdf

This was a paper from Shazam from 2003. This is essentially what I proposed, there is no training. Shazam works pretty well. It's not even going into the mathematical consideration I went into.

>You'll never be able to develop features with the heuristic methods you described that will work as well as the features learned by a neural net.

False.

Re: IBM halting sales of Watson AI tool for drug discovery

#165

Post by gbersac »

pmart123 wrote: post #30
tolstoy77 wrote: post #20

a lot of these AI companies products are really terrible. Has anyone ever tried the AI API models from clarifai ? Just so unaccurate. It seems like a scam. I've also had a really bad experience with watson's speech to text apis.

Watson’s is awful. Google’s default models are by far the best followed by Microsoft at a distance. Then, many of them are so bad that it is laughable.

Do you have any insight about AWS IA ?

Re: IBM halting sales of Watson AI tool for drug discovery

#166

Post by sonnyblarney »

acdc4life wrote: post #114
sonnyblarney wrote: post #98

Earlier quoted context omitted.

If you mean for medical diagnosis, maybe, but you do realize that NN/AI is totally state of the art for many tasks? NLP, image recognition etc. ? AI is in a hype bubble right now surely, but it's a 'very real' thing that's going to infiltrate a lot of areas.

>you do realize that NN/AI is totally state of the art for many tasks? Being state of the art doesn't imply that these things will solve these problems. In ML terms, how do you know that NN/AI isn't a local maxima that we need to jump out of? All NLP systems are joke. Sure replace Watson with DL, might perform better on Jeopardy. But in real conversations? Forget it. I wouldn't bet on these things. NN will win, but n…

"Being state of the art doesn't imply that these things will solve these problems. "

I suggest you are misinformed about the state of AI.

AI is currently ahead of all other approaches in many fields.

It's lead to quite a number of practical advances and breakthroughs.

The 'best examples' are those that I described, but there are many more.

Your comments indicate I think some ignorance on the issue - I think I see the point you are trying to make but I also submit that you're not aware of what AI is doing today.

'Self driving cars' would be impossible without AI today, for example. The vision systems depend on AI it's a breakthrough without which we simply wouldn't have the tech.

Re: IBM halting sales of Watson AI tool for drug discovery

#167

Post by pjmorris »

shshhdhs wrote: post #153
pjmorris wrote: post #128

Earlier quoted context omitted.

> I now think it’s madness for anyone to be in charge of anything if they don’t have relevant experience in the relevant domain. This. In support of that idea, here's C.A.R. Hoare, from 'The Emperor's Old Clothes', the 1980 Turing Award lecture: 'At last, there breezed into my office the most senior manager of all, a general manager of our parent company, Andrew St. Johnston. I was surprised that he had even heard of…

What was the solution here? To learn the Elliott 503 Mark II more deeply?

Oversimplifying, but quoting from the speech, that particular challenge was overcome as follows: "Thus we muddled through by common sense and compromise to something approaching success."

He then discusses how the problems there led to his thinking on CSP, and, eventually, formal methods.

But, go read the speech, it's much more wonderful than any summary thereof: http://zoo.cs.yale.edu/classes/cs422/2010/bib/hoare81emperor...

Re: IBM halting sales of Watson AI tool for drug discovery

#168

Post by PedroBatista »

jammygit wrote: post #151
PedroBatista wrote: post #23

Earlier quoted context omitted.

IBM will not fail, Oracle will fail first than IBM and that's unlikely to happen. You see, they are NOT tech companies. They have something you would call "tech", they sell something that looks like "tech" but they really solve problems people with access to large amounts of money have. In other words, they produce cushy and warm beds for C-level execs to have a good night of sleep, and everyone needs some bed to sle…

In the startup world there is a lot of customer obsession. What is it about these big companies that let's them de-prioritize that and still thrive? Honest question.

They have customer semi-obsession, but just a selected big customers ( most SP500 and governments ) and I can't think of a company that offers what IBM does ( the full spectrum, not just a subset).

So after getting in, they don't need to be good, they just need to not be too awful.

Re: IBM halting sales of Watson AI tool for drug discovery

#169

Post by Bartweiss »

ethbro wrote: post #13
euler_ wrote: post #11

Earlier quoted context omitted.

Does it make a difference if they preform better than human docs? The same thing happens now with human docs.

The difference is the expectation of fallibility with human docs. Which is why checks are built into the system. I have zero trust in IBM to market their ML products correctly so that proper checks are maintained. Especially since technically, explainability is still an active area of ML research.

> the expectation of fallibility with human docs

Not just the expectation but the understanding. A doctor might very well forget which leg to amputate, so we know to Sharpie "NOT THIS LEG" on the one being kept. But a doctor is very unlikely to see a patient with a broken wrist and prescribe antipsychotics, so we don't do much to prevent that error. Human fallibility happens along fairly predictable channels, and we've spent a very long time committing resources to controlling those channels.

Watson, though, thought Toronto is a city in the USA. Anyone who's dealt with ML output knows that the errors are often quite surprising, even before dealing with adversarial inputs. Even in a system where Watson's outputs are subject to checks, the checks we have today are human-specific and developed at a significant human cost. ML answers can't just outperform individual human doctors to add value, they need to either be gracefully integrated with them or be able to outperform the entire system which keeps those doctors on track.

Re: IBM halting sales of Watson AI tool for drug discovery

#170

Post by freehunter »

tellak wrote: post #68
freehunter wrote: post #25

Earlier quoted context omitted.

>I have zero trust in IBM to market their ML products correctly so that proper checks are maintained. That's a problem, yes, but it's not a new one. Vendor management has been around for decades, centuries, millennia maybe. If I contract out part of my job, it's still my responsibility to make sure the contractors are doing their job right. "But their marketing said..." or "but their sales guys said..." is not an exc…

> Doctors noticing that Watson is wrong is expected. Well as a doctor if I’m still ultimately responsible then nothing has fundamentally changed, this is just another tool, possibly one I’ll be forced to use by someone with not one day of medical training. And medicine in the US is in a precarious position. Software engineers are generally not licensed, and they’re not sued (it’s virtually unheard of). It’s different…

>Well as a doctor if I’m still ultimately responsible then nothing has fundamentally changed

If your stethoscope fails and you mistakenly pronounce the patient dead because of that, who is responsible? Do you blame the stethoscope salesman for claiming it's an accurate medical instrument that you never need to second-guess? Do you stop using stethoscopes altogether because they're "just another tool"?

If your x-ray machine fails and you say the patient's leg isn't broken because of that, who is responsible? It's "just another tool", do you stop using x-ray machines?

If your physician's assistant measures the patient's blood pressure wrong and you never double check their work, do you fire all of your PAs? And go back to seeing every patient for every procedure yourself?

Everything you have is "just another tool" and as with any tool, it's up to the human doctor to interpret the output. The idea is tools make you faster and more accurate, but everyone knows tools fail so you need to be able to double check their work. If the tool is consistently inaccurate, sure, throw it away. But if your argument is "if the tool can't completely replace me it's worthless" I think you're selling yourself a little short there.

Of course you're ultimately responsible. Your stethoscope didn't go to college for 8 years, it's just there to make your job easier.

Post reply on HN