261–270 of 403 posts
Earlier quoted context omitted.
Yet, the darn thing still can't reason.
Five years ago we would have said "the darn thing still can't write a cohesive paragraph".
The trouble is that people have been sold this idea that ML/AI can do amazing things, without properly being told that really the things it can do are quite narrowly-scoped. They've been sold the Star Trek computer idea. For example, years ago I was working on a prototype/proof-of-concept thing for instrumenting industrial machinery with stick-on small computers. Simple stuff - attach accelerators, temperature, humid…
Architectures are also tweaked slightly although they continue to be largely based on the two architectures that started the current deep neural net boom, Long-Short Term Memory Networks (LSTMs) and Convolutional Neural Networks (CNNs), which are largely used for natural language processing and machine vision respectively.
Every once in a while there is a bit of excitement about results that come out of adding a new parameter which is then given a fancy name like "attention" and generates much breathless copy about how neural nets will soon become as intelligent as a dog/ your three year old/ your personal assitant etc. Usually though it's not clear whether these new parameters or architectural tweaks are really responsible for the success of the proposed techniques and when one looks at the field as a whole it becomes more and more apparent that the most successful work is backed by the most data and the most computing power.
To slightly er tweak your prediction, I think what'll happen is that, indeed, like you say, there will be libraries and APIs and so on (well, there already are) but all these will be controlled by large companies that can afford the ground work. So, I don't expect small outfits, research teams at universities or tiny startups, to make a big difference. Academia is fast losing the ability to produce work that beats the state-of-the-art anyway (that's from personal comms with other researches). So in terms of academia perhaps we should expect a shift away from neural nets, to something that's easier to do research on.
Earlier quoted context omitted.
The bias/variance trade-off is not really related to extrapolation. Think of a point cloud following a quadratic shape. A linear model will extrapolate terribly.
Well, "more predictive" doesn't mean it's a perfect fit. Every model has error. A line through a point cloud curving upwards will still represent some of the points in the cloud. So it will have high error, but it's still a representation of the data. And yes, the bias-variance tradeoff is about generalisation (i.e. the ability to extrapolate to unseen data). But this is more related to the fact that in the real worl…
The way I see it, the variance is the part of the error that you can reduce by collecting more data from your distribution and increasing model complexity if needed.
The bias part is what will not get better no matter how much you sample your distribution, and extrapolation problems fall into that category.
Earlier quoted context omitted.
There has been next to ZERO progress towards genuine AGI despite a never-ending deluge of AI articles; that's normally the cause of scepticism. After several decades and a much-hyped last few years we have fake cleverness - impressively so in both cases - but nothing more.
It's even hard to say how "genuine" cleverness in human beings works - and we've had a lot more time to study ourselves - granted that most of that time we've not had the tools to understand how our brains work at the algorithmic level. I am not claiming that achieving some version of "genuine" AGI necessarily involves understanding how human intelligence works, but it is reasonable to expect that knowing more about…
Earlier quoted context omitted.
Linear models have more bias, so they represent current data less well and are more predictive of future, unseen data (think of a straight line through a point cloud). Non-linear models have more variance so they represent current data better and are less predictive of future, unseen data (think of a line snaking around a point cloud). An added complication is that deep neural net models are, in practice, vectors (or…
I'm not good at math, but I'm confused by the association of AI with non-linear stuff, setting aside the association of non-linear with "bad". I thought ML involved linear algebra or something (says xkcd!) which would presumably be...linear?
- NN can absolutely represent non-linear functions, and they are based on solving system of linear equations.
- The non-linear function here has nothing to do with the linearity of the system of linear equations used to construct it.
- The two main sources of non-linearity are, (a) the inputs (e.g., an image, or a series of images varying a non-linear fashion), and (b) the activation functions.
An understanding of AI's limitations is as far from sinking in as the average MBA is from comprehending Finnegan's Wake. An understanding that machine learning is not the entire 60-plus-year-old field of artificial intelligence would be nice to see for once from an institution that supposedly prides itself on precision of language and accuracy of reporting.
An understanding of Crichton's Gell-Mann amnesia effect is slowly starting to sink in, though, to at least one former subscriber.
Earlier quoted context omitted.
Things may have changed over the past 5 years or so. Things may have changed over the past 5 weeks or so with GPT-3.
Until GPT-3 can write something meaningful, it's really just a showcase of the technology and a gimmick of a product. Sure it's cool, but what problem is it solving? As far as I can tell the only useful function it has is polluting the internet with pseudo-intellectual comments to promote some agenda (likely political). So now that I think about it, it actually would be incredibly valuable for things like subverting…
This is a completely wrong reading of the situation. Tesla engineers clearly understood it is impossible to determine whether a stationary object is actually on the street or just an overhead sign with the equipment that is available on current Tesla cars and therefore they simply ignore all stationary objects. Because Autopilot is not a self driving technology this is not considered a problem. If a Tesla with enabled Autopilot ever runs into a stationary object it is clearly the driver's fault. It's extremely predictable and since it's easy to blame the driver there is no need to fix the problem by adding the required sensors to Tesla cars.
Earlier quoted context omitted.
The thing is, if we could write a specific, closed-end, prescriptive definition of "meaningful" or "understanding" or whatever, then we'd be able to program it. And we can't, so we have to settle for something else, usually how a thing fails to be what we (indeed subjectively) consider meaningful. Still, it's not arbitrary. The way that something like GPT-3 tends to fail basically is that you 2-3 paragraphs where par…
the thing has no fixed world-model What is "world-model"? What makes you think GPT-3 does not have some kind of a world model? It's clearly not a very good one, but at the same time it does not mean it can't get better. A 3 year old also does not have a very good world model, what's the difference between his world model and one of GPT-3? Again, clearly there's a big difference, I'm just not sure we know enough about…
Earlier quoted context omitted.
Your comment is a little hand-wavy and strongly worded ("revolution", "immeasurable", "limitless", "all domains"). Many things have exponential growth - bacterial reproduction, compound interest, certain chemical reactions. It's important to understand that this does not automatically result in miraculous universal transformation, but must be considered in the context of the world we live in. A little humility is alw…
Your analytical thinking misses the human element. To give an example: Steve Jobs didn't have humility, nor did he shy away from hyperbole. You would have had the same response to his words, but look at what he - literally, in the physical world - achieved. Human motivation - no matter how deluded - is not actually something not to dismiss. It has effects on others, which translates to action. The inputs are not all…
Without the Alan Turings, John von Neumanns, Edgar Dijkstras, Donald Knuths, the researchers at XPARC and Bell Labs, etc., there would never have been a Steve Jobs. He does play a role, but not nearly one as big as all those people that came before him.
Some people like to push for progress for progress's sake; others mostly to make money out of it (not saying this is bad!). The latter get more rich but it doesn't mean they're more important.