And here I thought that the Haskell style FizzBuzz was clever.
Fizz Buzz in Tensorflow
11–20 of 85 posts
Re: Fizz Buzz in Tensorflow
#12Earlier quoted context omitted.
Stupid question: We are trying to train the network how to calculate modulus of division. Of a binary encoding. Why should we expect this to be learnable in only two layers?
Yeah, secretly I was surprised that it worked as well as it did!
What architecture do you need to get 100% accuracy?
Don't forget to use a validation set for the model and hyperparameter selection though!
Re: Fizz Buzz in Tensorflow
#13No wonder you didn't get the job, using a shallow ann when they were probably looking for a pointer network.
Re: Fizz Buzz in Tensorflow
#14He should have used a Recurrent Neural Network with Long Short Term Memory so that the neural network won't have been dependent of the maximum number given NUM_DIGITS. Scalability! No wonders he didn't got the job.
Joke aside, it's funny how simple machine learning problems can reveal people who think you can just give neural networks anything and output anything, and it will work like magic.
Re: Fizz Buzz in Tensorflow
#15It would be much more interesting to see a TensorFlow program actually learn to solve FizzBuzz from examples, instead of hardcoding in the logic:
if i % 15 == 0: return np.array([0, 0, 0, 1])
elif i % 5 == 0: return np.array([0, 0, 1, 0])
elif i % 3 == 0: return np.array([0, 1, 0, 0])
else: return np.array([1, 0, 0, 0])Re: Fizz Buzz in Tensorflow
#16It would be much more interesting to see a TensorFlow program actually learn to solve FizzBuzz from examples, instead of hardcoding in the logic: if i % 15 == 0: return np.array([0, 0, 0, 1]) elif i % 5 == 0: return np.array([0, 0, 1, 0]) elif i % 3 == 0: return np.array([0, 1, 0, 0]) else: return np.array([1, 0, 0, 0])
The TF program does learn to sovle FizzBuzz. The above code was used to generate data with which to train the network.
Re: Fizz Buzz in Tensorflow
#17It would be much more interesting to see a TensorFlow program actually learn to solve FizzBuzz from examples, instead of hardcoding in the logic: if i % 15 == 0: return np.array([0, 0, 0, 1]) elif i % 5 == 0: return np.array([0, 0, 1, 0]) elif i % 3 == 0: return np.array([0, 1, 0, 0]) else: return np.array([1, 0, 0, 0])
It's really easy even just hand generate a "neural network" if you assume that the inputs can be preprocessed into base-3 and base-5 representations
Re: Fizz Buzz in Tensorflow
#18It would be much more interesting to see a TensorFlow program actually learn to solve FizzBuzz from examples, instead of hardcoding in the logic: if i % 15 == 0: return np.array([0, 0, 0, 1]) elif i % 5 == 0: return np.array([0, 0, 1, 0]) elif i % 3 == 0: return np.array([0, 1, 0, 0]) else: return np.array([1, 0, 0, 0])
As far as I can see, that function was used only to generate training data. How would you like to see it trained instead? Should it output a string and only be told whether that was the correct output or not for the given input number?
Re: Fizz Buzz in Tensorflow
#19He should have used a Recurrent Neural Network with Long Short Term Memory so that the neural network won't have been dependent of the maximum number given NUM_DIGITS. Scalability! No wonders he didn't got the job. Joke aside, it's funny how simple machine learning problems can reveal people who think you can just give neural networks anything and output anything, and it will work like magic.
> people who think you can just give neural networks anything and output anything, and it will work like magic.
What _is_ the proper method?
Re: Fizz Buzz in Tensorflow
#20Earlier quoted context omitted.
Yeah, secretly I was surprised that it worked as well as it did!
What architecture do you need to get 100% accuracy? Don't forget to use a validation set for the model and hyperparameter selection though!
I suspect that there is no way to predict in advance the smallest model which can represent any given function. However, we should be able to at least develop an intuition for depth and breadth that is sufficient on any given problem. Classically, division circuits are large and complex, so I naively expect a NN built out of add, mul, and Heaviside(x)*x to also be large and complex.
As with many models, I suspect that this network really learned some other property that has almost-but-not-quite the same pattern as divisible-by-N.