Past Performance is Not Indicative of Future Results (2020)

I agree that one would think that Science Fiction writers would have enough of an imagination to be able to consider alternate futures (Cory CYA's by saying such a scenario would make a good SF story) - but there are already promising approaches to AGI: Minsky's "Society of Mind", Jeff Hawkins' neuro-based approaches, the fairly new Hinton idea GLOM: https://www.technologyreview.com/2021/04/16/1021871/geoffrey... .

“By 2029, computers will have human-level intelligence,” Kurzweil said in an interview at SXSW 2017.

Time to get to work, eh? https://www.timeanddate.com/countdown/to?msg=Kurzweil%20AGI%...

So how would you quantify "good" or "bad"? You can't unless you also answer what "good" + "bad" should be. In psychology they just assume that mapping those onto 1 and 5 makes sense, so "good" + "bad" = 5 + 1 = 6, but that doesn't make sense since it would imply that "good" is the same as "bad" + "bad" + "bad" + "bad" + "bad". You get similar but different issues if you start including negative numbers, or if you just use relative measures and don't have a proper zero, no matter what you do numbers doesn't properly represent feelings as we know them.

It's key contributions are about the mainstream domination of quantitative vs. qualitative methods, especially in this paragraph:

> Quantitative disciplines are notorious for incinerating the qualitative ele­ments on the basis that they can’t be subjected to mathematical analysis. What’s left behind is a quantitative residue of dubious value… but at least you can do math with it. It’s the statistical equivalent to looking for your keys under a streetlight because it’s too dark where you dropped them.

and also of note is the "veneer of empirical facewash that provides plausible deniability", for discrimination, and for doing a poor job but continuing to be rewarded for it.

If I had to summarize it would be:

- The ML/AI community, which includes the researchers, practitioners, and the evangelists, are broadly utopian in what they think they can achieve. They are overconfident even in the domain of detecting the face of potential burglars in a home security camera, never mind in terms of creating new life with AGI. I think Doctorow's critique equally applies to "algorithms" even only as complex as a fancy Excel sheet, but he focuses on ML/AI as the most common source of this excess of optimism, that recording data and running it through a model is almost certainly the _most sensible thing to do_ for any given problem.

- If there is a manufactured consensus that the almost purely quantitative approach is the _most sensible thing to do_, then any failures or short-comings can be hand-waved away. Say sorry, "the model/algorithm did it", and just ignore the issue or apply a minor manual fix. This is a huge benefit for decision-makers wishing to maintain their status/livelihoods in both the public and private sector. Crucially, this excuse works if you're just ineffective, or if you're a bad actor.

Note that this is a critique of CEOs and government officials, more than of engineers -- we would only be complicit by association. If there is a critique for engineers, it's that we provide fodder for the excess of optimism in summary point 1 because we love playing with our tools, and that we allow ourselves to be the scapegoat for summary point 2.

The more you learn about neurobiology, the more apparent it is that there are so many levels of computation going on - everything from dendritic structure, to cellular metabolism, to epigenetics has an effect on information processing. The idea that we could reach some approximation of "general intelligence" by just scaling up some very large matrix operations just seemed like a complete joke.

However, as you say, that doesn't mean what we've done in ML is not worthwhile and interesting. We might have over-reached thinking ML is ready to drive a car without major fourth-coming advancements, but use-cases like style transfer and DLSS 2 are downright magical. Even if we just made marginal improvements in current ML, I'm sure there is a ton of untapped potential in terms of applying this tech to novel use-cases.