A visual proof that neural nets can approximate any function
neuralnetworksanddeeplearning.com
A visual proof that neural nets can approximate any function
1–10 of 138 posts
Re: A visual proof that neural nets can approximate any function
#2Can it compute y=sin x?
Re: A visual proof that neural nets can approximate any function
#3Can it compute y=sin x?
> First, this doesn't mean that a network can be used to exactly compute any function. Rather, we can get an approximation that is as good as we want. By increasing the number of hidden neurons we can improve the approximation.
Re: A visual proof that neural nets can approximate any function
#4Any deeplearning expert here. Why Neural network can't compute a linear function Celsius to Fahrenheit 100% accurately.
Is it data or is it something can be optimised.
```
celsius_q = np.array([-40, -10, 0, 8, 15, 22, 38], dtype=float)
fahrenheit_a = np.array([-40, 14, 32, 46, 59, 72, 100], dtype=float)
for i,c in enumerate(celsius_q): print("{} degrees Celsius = {} degrees Fahrenheit".format(c, fahrenheit_a[i]))
l0 = tf.keras.layers.Dense(units=1, input_shape=[1])
model = tf.keras.Sequential([l0])
model.compile(loss='mean_squared_error', optimizer=tf.keras.optimizers.Adam(0.1)) history = model.fit(celsius_q, fahrenheit_a, epochs=500, verbose=False)
print("Finished training the model")
print(model.predict([100.0])) // it results 211.874 which is not 100% accurate (100×1.8+32=212)
```
What can be done to make this NN 100% accurate for simple linear equation 𝑓=1.8𝑐+32
https://colab.research.google.com/github/tensorflow/examples...
Re: A visual proof that neural nets can approximate any function
#5Any deeplearning expert here. Why Neural network can't compute a linear function Celsius to Fahrenheit 100% accurately. Is it data or is it something can be optimised. ``` celsius_q = np.array([-40, -10, 0, 8, 15, 22, 38], dtype=float) fahrenheit_a = np.array([-40, 14, 32, 46, 59, 72, 100], dtype=float) for i,c in enumerate(celsius_q): print("{} degrees Celsius = {} degrees Fahrenheit".format(c, fahrenheit_a[i])) l0…
It's the optimization strategy. Optimizers for neural nets are designed to perform well for non-linear problems, which comes at the expense of not providing exact solutions for linear regressions. A general exact solution is not possible for nonlinear, over-specified problems (which is what neural nets are good at), so strategies different than those used in linear regression are necessary.
Re: A visual proof that neural nets can approximate any function
#6Any deeplearning expert here. Why Neural network can't compute a linear function Celsius to Fahrenheit 100% accurately. Is it data or is it something can be optimised. ``` celsius_q = np.array([-40, -10, 0, 8, 15, 22, 38], dtype=float) fahrenheit_a = np.array([-40, 14, 32, 46, 59, 72, 100], dtype=float) for i,c in enumerate(celsius_q): print("{} degrees Celsius = {} degrees Fahrenheit".format(c, fahrenheit_a[i])) l0…
[deleted]
Re: A visual proof that neural nets can approximate any function
#7Any deeplearning expert here. Why Neural network can't compute a linear function Celsius to Fahrenheit 100% accurately. Is it data or is it something can be optimised. ``` celsius_q = np.array([-40, -10, 0, 8, 15, 22, 38], dtype=float) fahrenheit_a = np.array([-40, 14, 32, 46, 59, 72, 100], dtype=float) for i,c in enumerate(celsius_q): print("{} degrees Celsius = {} degrees Fahrenheit".format(c, fahrenheit_a[i])) l0…
Your data is suspiciously rounded off. Just doing linear regression on that data isn't going to give you a perfect fit either.
Re: A visual proof that neural nets can approximate any function
#8Any deeplearning expert here. Why Neural network can't compute a linear function Celsius to Fahrenheit 100% accurately. Is it data or is it something can be optimised. ``` celsius_q = np.array([-40, -10, 0, 8, 15, 22, 38], dtype=float) fahrenheit_a = np.array([-40, 14, 32, 46, 59, 72, 100], dtype=float) for i,c in enumerate(celsius_q): print("{} degrees Celsius = {} degrees Fahrenheit".format(c, fahrenheit_a[i])) l0…
If you wanted 100% accuracy, technically infinite data points and training time would be required. You could likely get >99% accuracy within a few epochs of training though. With such a simple function you're trying to emulate, your model will very quickly converge.
Neural networks are much better suited for distilling down and compressing very complex high dimensional data though anyways, and you really don't need to be using them for problems like this. It's completely overkill in addition to being very computationally inefficient. There's nothing wrong with just simply using linear regression. In many cases it's the right choice.
In your toy problem case you coded above, you are effectively just doing linear regression, except you added in an Adam gradient descent optimizer instead of just doing least squares, which by the way would have been infinitely faster and immediately given you an answer.
Re: A visual proof that neural nets can approximate any function
#9Any orthonormal ~set~ basis of functions can represent any function. So what?
Re: A visual proof that neural nets can approximate any function
#10*any continuous function