Seems to me that the foundations are not crumbling, but there should be a way to formally determine how good is a model going to be in the wild before it is used, especially in certain industries. Which I think it's where research is focused on these days? White box models, Bayesian distributions etc.?
Machine learning’s crumbling foundations
41–50 of 106 posts
Re: Machine learning’s crumbling foundations
#42Earlier quoted context omitted.
Yeah, that was an odd claim about Han Chinese surnames. Many have been around for thousands of years ( https://www.chinadaily.com.cn/ezine/2007-07/20/content_54412... ) and almost all are single-character surnames based on a limited set of possible sounds (~400 in Mandarin, IIRC)
I don't know anything about Chinese surnames, but their paucity cannot be due to a limited set of possible sounds. First, I would interpret "sounds" as phonemes (including tones), and there are far fewer of those than 400. More likely what you mean is the number of combinations of phonemes into valid Chinese Mandarin monosyllables, of which I cannot imagine there being only 400. In any case, there are (from what litt…
> More likely what you mean is the number of combinations of phonemes into valid Chinese Mandarin monosyllables, of which I cannot imagine there being only 400.
That's your problem, not ilamont's. The limited syllable inventory of Mandarin Chinese is very well known. No need to stretch your imagination over it.
That said, surnames are not limited by the number of syllables for the obvious reason that the spelling is part of the surname.
Re: Machine learning’s crumbling foundations
#43Earlier quoted context omitted.
> Meanwhile, speech recognition seems to work extremely well by now (I am a little bit older, so I remember when it didn't work so well). *provided you speak English or Mandarin, the former preferably of a continental US variety It's astonishing how bad things get again once you mix in an accent, local dialect (e.g. Swiss German) or a less frequently spoken language (like Croatian).
Nevertheless, the huge jump is from "does not work at all" to "it works". It seems likely that the technology that worked for English will also work for many other languages. As for Chinese, it is also pretty amazing that you can visit a Chinese website, click "translate" in your browser's menu bar, and get a reasonably readable translated version. I wonder if people just take too many things for granted. Or internet…
> It seems likely that the technology that worked for English will also work for many other languages.
It won't for the foreseeable future. Not for technical reasons; it's just that other languages are usually not handled correctly because most companies think they can just use the exact same approach as in English and they're done.
Until they realise that non-English native speakers also use English words and abbreviations to some degree, both in IT-related contexts but also in everyday life. Now it doesn't just need to handle that one language but also English with an accent. If they're lucky it'll work reasonably well in most cases despite variations depending on the region.
Right now even keyboard completion suggestions struggle with mixing languages and become completely useless in some cases. As English words may be mixed in at any location (and in wildly different frequencies depending on the user) the software now has to guess the language for every single word. The results are not great.
> they say quality of Google searches have been declining, nevertheless we had a pretty good run for the past 20 years or so
As long as Google continues with blunders like showing wrong pictures of people in infoboxes they'll keep failing hard. Their amazing AI shows wrong pictures for serial killers, rape victims and more, which already led to consequences for those people. What makes it much worse is that when someone complains about such a case Google will just replace that picture with another wrong portrait - if they react at all. It would be helpful if those big tech companies would for once trust in human intelligence instead of throwing larger models at the problem.
Re: Machine learning’s crumbling foundations
#44My favourite example of bad data in for machine learning is the tragic tale of Scots Wikipedia: https://www.theguardian.com/uk-news/2020/aug/26/shock-an-aw-... It turned out an enthusiastic but misguided US teenager who didn't actually know the Scots language was responsible for most of the entries on it... and a bunch of natural language machine learning models had already been trained on it.
Scots pretty much is a dialect of English that is phonetically spelt out - it's not surprising that a US teenager could write it.
Re: Machine learning’s crumbling foundations
#45You could use this article's underlying thesis to explain why a lot of tech companies fail as well. Google Health is a good example of failing to appreciate specialization and domain-expertise. Trying to draw value from broad generic data collection when IRL it requires vertical-focused domain-oriented collection and analysis to really draw value. Funnelling everything into a giant pool of data only had so much value…
> Google Health is a good example of failing to appreciate specialization and domain-expertise. Trying to draw value from broad generic data collection when IRL it requires vertical-focused domain-oriented collection and analysis to really draw value.
I'm not sure I agree with this statement. From what I've heard, Google Health employed a huge team of doctors and they were included through the entire feature development lifecycle, similar to how the product org functions in other software companies.
Re: Machine learning’s crumbling foundations
#46I've always thought it was very sad and unfortunate that core data classes like sampling design and experimental design have fallen out of academic style.
Re: Machine learning’s crumbling foundations
#47My favourite example of bad data in for machine learning is the tragic tale of Scots Wikipedia: https://www.theguardian.com/uk-news/2020/aug/26/shock-an-aw-... It turned out an enthusiastic but misguided US teenager who didn't actually know the Scots language was responsible for most of the entries on it... and a bunch of natural language machine learning models had already been trained on it.
Scots pretty much is a dialect of English that is phonetically spelt out - it's not surprising that a US teenager could write it.
>"Scots pretty much is a dialect of English that is phonetically spelt out - it's not surprising that a US teenager could write it."
No, there are a number of distinct linguistic features of Scots, and it has its own regional dialects, e.g. Doric, Orcadian, Shetland (which is also in part based on the extinct Norn language). See e.g. https://dsl.ac.uk/about-scots/history-of-scots/ (and sub-pages such as https://dsl.ac.uk/about-scots/history-of-scots/grammar/ ) for further information. Simply doing a dictionary-lookup word-replacement completely misses all of this nuance.
Re: Machine learning’s crumbling foundations
#48You could use this article's underlying thesis to explain why a lot of tech companies fail as well. Google Health is a good example of failing to appreciate specialization and domain-expertise. Trying to draw value from broad generic data collection when IRL it requires vertical-focused domain-oriented collection and analysis to really draw value. Funnelling everything into a giant pool of data only had so much value…
> Google Health is a good example of failing to appreciate specialization and domain-expertise. Trying to draw value from broad generic data collection when IRL it requires vertical-focused domain-oriented collection and analysis to really draw value. I'm not sure I agree with this statement. From what I've heard, Google Health employed a huge team of doctors and they were included through the entire feature developm…
Hiring a broad set of domain experts != a domain/vertical focused business. ‘Doctors’ can cover a massive disparate field of study.
My point is they did it backwards, they should have found real world healthcare problems to solve then built the common ground between them. Building a generic API platform or cloud database turned out to not be the problem anyone needed help solving. Most companies who did the integration to Health did it for marketing, not because it was essential to any business value.
How many companies have done “AI” merely for marketing too?
Google search ranked websites better than anyone, they zeroed in on that one problem and removed all the cruft, while Yahoo and others were jamming as much crap into their ‘portals’ as possible. Google seemed to have forgot that lesson.
Waymo fell for this too. They built an entirely new type of car and gambled on a whole new taxi service (among other promises) that would entirely disrupt transportation - as the starting point. Innovation rarely ever jumps ten steps ahead like that. They chose to solve a thousand problems at once while the rest of the world with actual delivered products are struggling to solve even assisted highway driving in high-end luxury cars.. cars people were going to buy anyway.
Re: Machine learning’s crumbling foundations
#49Earlier quoted context omitted.
> programming's foundations are crumbling That's also correct, and has been for some time (it got worse on each tech boom). This may just be a special case of that.
What do you mean? At least from the point of view of the end user, apps seem to become better over time.
I was tempted to just downvote this, but I thought I'd reply instead:
No, they do not. An existing version of an app may get better over time, but unfortunately it then gets replaced with a different version, which starts from the position of extreme bugginess.
In the case of Microsoft Office apps, for instance, one could easily argue that they are steadily getting worse as more and more features are added.
Google Chrome is pretty clearly getting worse in terms of the amount of memory it uses. I could go on.
Re: Machine learning’s crumbling foundations
#50It's a structural issue caused by the way wealth creation works for majority of people in tech. Job hopping, trendy frameworks in CV, "high-impact" projects done ASAP, etc. No one wants to do boring, slow pace work with lots of planning, reflection and introspection. And why would they do it? These kind of jobs are usually worst paid. We, the practitioners, have every economic incentive to go the other route. The pro…
Sadly PCR tests for COVID also test positive for flu and half a dozen other causes. That's why CDC/FDA are seeking proposals for a new test that actually works!
https://www.cdc.gov/csels/dls/locs/2021/07-21-2021-lab-alert...