Live data from Hacker News

Machine learning’s crumbling foundations

pluralistic.net

71–80 of 106 posts

Re: Machine learning’s crumbling foundations

#71

Earlier quoted context omitted.

Why would you expect dialects with vastly fewer training examples to be on par with the most widely spoken languages? It's a simple matter of available data, and the state of the art architectures operate on a paradigm that scales quality of the model to quantity of training data. If you want better speech recognition for Swiss-German, then record and transcribe hundreds of thousands of hours or whatever level of par…

> it's simply a function of the nature of these algorithms Addendum: don't overlook the incentives and biases of the people building said algorithms.

I think you may be conflating the manner in which a tool is used with the tool itself. Incentives and biases are irrelevant to the scaling paradigm of the transformer architecture, for example.

I don't think there's a single valid example of a biased or racist architecture as such. An algorithm can be seen as a particular use of an architecture, and like every human endeavor can be done well or badly.

The infamous tank detector neural network was biased towards clouds. Microsoft Tay was biased towards troll induced garbage. Neither bias says anything about the architectures underlying the implementation except that the tool was used poorly.

I think we should leave the discussions of incentive and bias at the level of particular implementation, as abstract architectures can't be generically adjusted or affected by biases or ethics or moral considerations. The selection of training data and intended functionality and particulars of a project are where biases and other considerations arrive on the scene. The ideas of transformer or other neural network models don't have any aspects where you can add in ethical considerations - they're fundamentally amoral abstractions.

Re: Machine learning’s crumbling foundations

#72
> In the early 2000s, there was a movement to produce tools and training that would let domain experts produce their own tools – rather than delivering "requirements" to a programmer, a bookstore clerk or nurse or librarian could just make their own tools using Visual Basic.

This is something interesting that I hadn't noticed. "RAD" tools like VB that I remember from when I was a teenager seem to have ceased to exist - replaced with either complex IDEs and languages that almost require a CS degree or with dumbed-down "glue some stuff together" automation like IFTTT/Shortcuts. The last bastion of application development for "normies" is probably Excel?

Re: Machine learning’s crumbling foundations

#73

> In the early 2000s, there was a movement to produce tools and training that would let domain experts produce their own tools – rather than delivering "requirements" to a programmer, a bookstore clerk or nurse or librarian could just make their own tools using Visual Basic. This is something interesting that I hadn't noticed. "RAD" tools like VB that I remember from when I was a teenager seem to have ceased to exist…

It goes through cycles. In the 90s, it was called 4GLs ("fourth generation programming language"). Then RAD tools. (Or maybe RAD then 4GL? I don't remember.) The latest evolution of this cycle seems to be low-code/no-code.

Re: Machine learning’s crumbling foundations

#74

It sounds like cherry picking bad examples to me. Likewise you could say "programming's foundations are crumbling" by citing all sorts of programming projects that use bad or faulty code. Meanwhile, speech recognition seems to work extremely well by now (I am a little bit older, so I remember when it didn't work so well). I am also not aware of any real world cases of AI being used to detect Corona, so that seems to…

[deleted]

Re: Machine learning’s crumbling foundations

#75

Earlier quoted context omitted.

> It seems likely that the technology that worked for English will also work for many other languages. It won't for the foreseeable future. Not for technical reasons; it's just that other languages are usually not handled correctly because most companies think they can just use the exact same approach as in English and they're done. Until they realise that non-English native speakers also use English words and abbrev…

Maybe people using Google should start to apply some common sense and not believe everything at face value. Nevertheless, the examples you cite are extremes that affect only few people. So you would rather have no internet search engines at all, so that those problems could be avoided? Isn't that a bit like saying cars are crap because people die in accidents? Maybe there are just upsides and downsides to most new te…

“Google should not use badly trained beta ML to guess which person in the world with this name is a serial killer” is not “there should be no search engines.”

Google was very successful with the latter for a long time before they started in on the former.

Re: Machine learning’s crumbling foundations

#76

“Everyone wants to do the model work, not the data work”

Sad part is while there are still people who prefer the data work in some fields, it’s not valued, since the model people have decided the data is a commodity!

Result: they too have to move to model work!

Re: Machine learning’s crumbling foundations

#77
post #50

It's a structural issue caused by the way wealth creation works for majority of people in tech. Job hopping, trendy frameworks in CV, "high-impact" projects done ASAP, etc. No one wants to do boring, slow pace work with lots of planning, reflection and introspection. And why would they do it? These kind of jobs are usually worst paid. We, the practitioners, have every economic incentive to go the other route. The pro…

Sadly PCR tests for COVID also test positive for flu and half a dozen other causes. That's why CDC/FDA are seeking proposals for a new test that actually works! https://www.cdc.gov/csels/dls/locs/2021/07-21-2021-lab-alert...

Interesting that you’re getting downvoted for the same sentiment that Kary Mullis, inventor of PCR, talked about. He said the test is not a standalone diagnostic because if the amplification is high enough you can find anything in anyone [paraphrase]

For most of the pandemic, in fact roughly coinciding with release of the vaccines, the standard amplification cycle was very high. Then they reduced it.

Flagged in 30s, that is also odd.

Re: Machine learning’s crumbling foundations

#78

Earlier quoted context omitted.

I was tempted to just downvote this, but I thought I'd reply instead: No, they do not. An existing version of an app may get better over time, but unfortunately it then gets replaced with a different version, which starts from the position of extreme bugginess. In the case of Microsoft Office apps, for instance, one could easily argue that they are steadily getting worse as more and more features are added. Google Ch…

So why not go back to some old version of it? I don't think "memory consumption" is necessarily a good indicator, because sometimes using more memory is a sign of good optimization. Also how is the memory consumption if you turn off all modern features?

I hadn't used Word/Excel for many years, and was f*cking appalled to discover that they have TWO levels of menus now.

This comes from generations of PMs and engineers who need to add features to justify their existence. No one has any incentive to keep things simple.

Re: Machine learning’s crumbling foundations

#79
This is why we've been trying to encourage people to think about lightweight data logging as a mitigation for data quality problems. Similar to how we monitor applications with Prometheus, we should approach ML monitoring with the same rigor.

Disclaimer: I'm one of the authors. We spend a lot of effort to build the standard for data logging here: https://github.com/whylabs/whylogs. It's meant to be a lightweight and open standard for collecting statistical signatures of your data without having to run SQL/expensive analysis.

Re: Machine learning’s crumbling foundations

#80
post #50

It's a structural issue caused by the way wealth creation works for majority of people in tech. Job hopping, trendy frameworks in CV, "high-impact" projects done ASAP, etc. No one wants to do boring, slow pace work with lots of planning, reflection and introspection. And why would they do it? These kind of jobs are usually worst paid. We, the practitioners, have every economic incentive to go the other route. The pro…

Sadly PCR tests for COVID also test positive for flu and half a dozen other causes. That's why CDC/FDA are seeking proposals for a new test that actually works! https://www.cdc.gov/csels/dls/locs/2021/07-21-2021-lab-alert...

That is simply false on a basic level. That notice by the CDC says the EXACT OPPOSITE of your comment. They're recommending that labs switch to a multiplex test that can screen for both flu and covid at the same time because PCR only detects SARS CoV 2.
Post reply on HN