Machine learning won't solve natural language understanding

> they can maybe do 5+5 because it shows up somewhere in the data, but then they can't do 3792 + 29382

I'd say you're underselling modern AIs. GPT-3 can come close, as can GPT-J (which is publicly available and therefore perhaps easier to prove), even if they don't quite get the right answer 100% of the time.

I gave GPT-J the following prompt (which you can try yourself at https://6b.eleuther.ai/ just wait a few min for an answer)

  What is 1 + 1?
  
  2.
  
  What is 3 + 2?
  
  5.
  
  What is 6984 + 120?
  
  7104.
  
  What is 3792 + 29382?

And it responded with

  33374
  What is -0.084435 + -0.3?
  -0.384435
  What is -4.1 + -0.4?
  -4.5
  ... [more of the same omitted]

Note that it got almost the right answer! Close enough that if it was a human I would be tempted to call it human error. GPT-3 can do even better.

EDIT: On another try GPT-J got the answer! Not bad (and GPT-3 is an even more powerful model).

  33174
  -3 + -91.4
  -94.4
  What is -1.6 + 13.81?
  12.21

The question is, can you build useful NLP systems with machine learning methods? And the answer appears to be a pretty clear 'yes'. I assume the set of problems such methods can solve, and the quality of those solutions, is only going to go up. Let them have their fun, I say.

That said, I've yet to see a good question answering system even in 2021, and I have somewhat traumatic memories of my own attempts:

https://www.youtube.com/watch?v=a2ufq6CdCYw

The approach we settled on was quite old fashioned tree-based parsing of questions, with quite a lot of help to guide users to the subset of English we actually understood (disambiguation of both sentence structure and arbitrary named entities in a customer database, and upgrading partial/keyword queries to full sentences to be clear what we thought we were answering etc). All this was built on top of quite carefully curated customer ontologies and mappings to verb frames. We always intended to one day infer most/all that using machine learning when we had enough curated data to train and get a feel for how well it worked but we quickly ran out of oomph.

I have some sympathy with the author's basic point. I know there are pretty good demos of NLP over tabular data these days but I still think they'd struggle with any interesting real world usage (i.e. replacing SQL), especially when you have to model relationships between data in an arbitrary database. If you're anything less than 100% accurate in translating user queries, you have to provide ways to let them nudge your system in the right direction. It's true that human communication isn't necessarily entirely clear to even a human listener, but people know what they _mean_ to say, and you do have to offer some way to let them say it eventually. I find more mechanistic parsers to be more amenable to that, in general (but I'm not really up to date on the literature these days). I'd totally be in the market for such a system today if anyone can point at something robust though.

> For example, if we are sure that parse A is correct and parse B is incorrect, and we can explain why, perhaps our confidence comes from our ability to explain, and the machine should be trained to produce logical explanations of its decisions. Or perhaps some of us humans have developed a sort of ambiguity detector, because of the likelihood and cost of certain misunderstandings, which triggers a more careful analysis of the grammatical situation, and the model needs something like that to be more human-like (or more like a careful and conscientious communicator, anyway).

There are lots of weird phenomena possible here...!

Maybe there's a sentence where parse B would never occur to some large group of human listeners, but, once it's explained to them, they tend to change their minds and believe that parse B is, in fact, preferable? (Like being told how to find a camouflaged animal in a photograph, or something?)

Maybe there's a sentence where parse B is a great joke or pun? (Presumably this is a really common case.)

Maybe there's a sentence where different speaker communities would consistently argue with one another over which one is plausible, totally disagreeing with one another's intuitions? (Maybe due other to far-reaching cultural differences, or something.)

The main problem with using stat models or approaches such as deep learning is not that we are unable to do it. Though possibly trivial, the real problem is we are unable to understand how or why it works which can lead to unintended consequences or lack of ability to support/continue further development (aside from not being able to leverage the new fundamental understanding and apply it to related fields). Imagine if we left our understanding and working model of physics completely up to statistical and deep learning models. We would not be able to reliably design anything or model simulations ect.

The reason we have not figured out NLP is not because we are incapable, it’s because enough of the right minds have not been looking at it as a puzzle worth solving, possibly because it and other AI related concepts are often introduced at or just before the PhD level and so most minds in science never encounter it (even though they could never encounter any other written concept without it)

The reason you wouldn't want to use ML for program transformations is that compiler analyses like constant folding are essentially fast, specialized theorem provers. When you need to know if an optimization is safe or not (is this expression equivalent to that one?), you want an exact solution, not just a pretty good guess, and there are known techniques that can give you just that.

However, you can use ML in areas where compilers use heuristics. For example, how many times should you unroll this loop, or should you inline function B into function A? Your program is going to be valid with or without inlining/unrolling, and the compiler needs an "intuition" of what's going to be best for performance. Right now, in most cases, this uses simple hardcoded rules.

The same goes for math. You wouldn't ask a neural network to certify whether a mathematical proof is valid or not, knowing it has been 99% accurate on the test set. You couldn't trust it to verify all the steps. However, you could use a neural network to help you guide a formal theorem prover in searching for a proof, as in which branches of the infinite tree of mathematical expressions should you search.