I am 15 years into this computers thing and this blog post made me feel like "those guys are doing black magic". Neural networks and deep learning are truly awesome technologies.
Of course they are
How Google Translate squeezes deep learning onto a phone
61–70 of 99 posts
Re: How Google Translate squeezes deep learning onto a phone
#62Given the reliability of closed captions on YouTube and the frequency of errors in plaintext Google translate, I wouldn't be surprised if this service fails often, and often when you need it most.
Re: How Google Translate squeezes deep learning onto a phone
#63I am 15 years into this computers thing and this blog post made me feel like "those guys are doing black magic". Neural networks and deep learning are truly awesome technologies.
they are awesome, but not that difficult to implement
Re: How Google Translate squeezes deep learning onto a phone
#64This is great. I particularly like that they also automatically generated dirty versions for their training set, because that's exactly what I ended up doing for my dissertation project (a computer vision system [1] that automatically referees Scrabble boards). I also used dictionary analysis and the classifier's own confusion matrix to boost its accuracy. If you're also interested in real time OCR like this, I did a…
Glad there's prior art on that. I had a small project where I iterated all the fonts on the system and used them to generate glyph training images. The next step was to dirty them up, but I never continued the project.
More generally, I really like the idea of generating controlled synthetic images and then messing them up for regularization.
Re: How Google Translate squeezes deep learning onto a phone
#65This is great. I particularly like that they also automatically generated dirty versions for their training set, because that's exactly what I ended up doing for my dissertation project (a computer vision system [1] that automatically referees Scrabble boards). I also used dictionary analysis and the classifier's own confusion matrix to boost its accuracy. If you're also interested in real time OCR like this, I did a…
It seems your dissertation paper is behind something password protected [1]. It would be nice to see that too.
Can't get [1]https://www.dcs.shef.ac.uk/intranet/teaching/campus/projects...
Re: How Google Translate squeezes deep learning onto a phone
#66Earlier quoted context omitted.
They are, but once you start learning about them, you realize the "black magic" part comes mostly from their mathematical nature and very little from them being "inteligent computers". A neural net is a graph, in which a subset of nodes are "inputs" (that's where the net gets information), some are outputs, and there are other nodes which are called "hidden neurons". The nodes are interconnected between each other in…
You could say very much the same about the brain... > [...] the "black magic" part comes mostly from their mathematical nature and very little from them being "inteligent computers". A brain is a graph, in which a subset of neurons are "inputs", some are outputs, and others are "hidden". The nodes are interconnected between each other in a fashion, which is called the "topology" or sometimes "architecture" of the net…
It's not really that deep, imo: a typical deep net these days has O(10^8) parameters (e.g. http://stackoverflow.com/questions/28232235/how-to-calculate...). You can store a hell of a lot of patterns in that many parameters, making them the best pattern matchers the world has ever seen. (Un)fortunately, pattern matching != intelligence. More interesting deep questions for which there is precious little theory revolve around the design of the networks themselves.
Re: How Google Translate squeezes deep learning onto a phone
#67Earlier quoted context omitted.
They are, but once you start learning about them, you realize the "black magic" part comes mostly from their mathematical nature and very little from them being "inteligent computers". A neural net is a graph, in which a subset of nodes are "inputs" (that's where the net gets information), some are outputs, and there are other nodes which are called "hidden neurons". The nodes are interconnected between each other in…
You could say very much the same about the brain... > [...] the "black magic" part comes mostly from their mathematical nature and very little from them being "inteligent computers". A brain is a graph, in which a subset of neurons are "inputs", some are outputs, and others are "hidden". The nodes are interconnected between each other in a fashion, which is called the "topology" or sometimes "architecture" of the net…
They are effective because:
- They use more parameters (and fewer computations per parameter.)
- They are hierarchical (convolutions are apparently useful at different levels of abstraction of data).
- They are distributed (word2vec, thought-vectors). Not restricted to a small set of artificial classes such as parts-of-speech or parts of visual objects.
- They are recurrent (RNN).
etc.
Re: How Google Translate squeezes deep learning onto a phone
#68Earlier quoted context omitted.
You could say very much the same about the brain... > [...] the "black magic" part comes mostly from their mathematical nature and very little from them being "inteligent computers". A brain is a graph, in which a subset of neurons are "inputs", some are outputs, and others are "hidden". The nodes are interconnected between each other in a fashion, which is called the "topology" or sometimes "architecture" of the net…
If there is magic to be found, it may be in that question. Why about graphs (namely the subset that are deep neural networks) allow them to not only contain such powerful heuristics, but also allow them to be created from scratch with barely any knowledge of the problem domain. As a side note, I was playing a board game last night (Terra Mystica I believe) and wondering if you could get 5 different neural networks to…
The best Go bot approach (as of some years ago, but it's not like neural networks are a new idea) uses a very different strategy. Specifically, the strategy of "identify a few possible moves, simulate the game for several steps after each move using a very stupid move-making heuristic instead of using this actual strategy recursively, and then pick the move that yielded the best simulated board state".
Re: How Google Translate squeezes deep learning onto a phone
#69I used this in Brazil this last March to read menus. It works extremely well. The mistranslations make it even more fun. Much faster than learning Portuguese! I took a few screen shots. Aligning the phone, focus, light, shadows on the small menu font was difficult. You must keep steady. Sadly, I ended up hitting the volume control on this best example. Tasty cockroaches! Ha! http://imgur.com/j9iRaY0
I had some Brazilian roomates who didn't speak english (and I don't speak portugues). We used a combination of my poor spanish and google translate off my phone to comunicate. It worked ok (much better than nothing.) However there were a number of times when there were very large issues in the translations that created some pretty big misunderstandings. Luckily we had a friend who had fluent English and Portuguese wh…
> Please write words with just one meaning.
Those words are very rare and tend to only be useful in very technical contexts.
Re: How Google Translate squeezes deep learning onto a phone
#70The most awesome and surprising thing about this is that the whole thing runs locally on your smartphone! You don't need network connection. All dictionaries, grammar processing, image processing, DNN - the whole stack runs on phone. I used this on my trip to Moscow and it was truely god send because it didn't need expensive international data plans (assuming you have connectivity!). English usage is fairly rare in Russia and it was just fun to learn Russian this way by pointing at interesting things.