Earlier quoted context omitted.
So am I correct in assuming that you are looking for a machine that can do the mapping of a symbol into a form that the function can learn from? Your visual and auditory systems do this, and since they are physical systems this mapping is computable. By the way, for the addition example , the neural network needs only one example, e.g. 1+1=2, applied repeatedly to discover the addition of any two inputs. There is no…
"Extrapolation is the process of constructing new data points outside a discrete set of known data points."
I have rethought what you have said, and I think I am now closer to understanding what your question is. You are posing the question of what to do with an input, not the operation itself. For instance, if I feed the network an image of 5 and 3, with the output being the image 8, the network would not perform addition but some other functional mapping. Is that the feature you are suggesting that it should 'understand'?