Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Google Translate squeezes deep learning onto a phone

googleresearch.blogspot.com

Post reply on HN
Print view View original

71–80 of 99 posts

Re: How Google Translate squeezes deep learning onto a phone

#71

Post by shkkmo »

thaumasiotes wrote: post #69
shkkmo wrote: post #59

Earlier quoted context omitted.

I had some Brazilian roomates who didn't speak english (and I don't speak portugues). We used a combination of my poor spanish and google translate off my phone to comunicate. It worked ok (much better than nothing.) However there were a number of times when there were very large issues in the translations that created some pretty big misunderstandings. Luckily we had a friend who had fluent English and Portuguese wh…

> Please write words with just one meaning. Those words are very rare and tend to only be useful in very technical contexts.

Fair enough. The idea that is intended to express is 'unambiguous'. I tend to try to avoid more obscure words when writing text for automatic translation, often at the expense of explicit accuracy.

Re: How Google Translate squeezes deep learning onto a phone

#72

Post by pschanely »

Doesn't this article seem to say that the size of the training set is related to the size of the resulting network? It should be proportional to the number of nodes/layers that the network is configured for, not proportional to the number of training instances. Am I missing something?

Re: How Google Translate squeezes deep learning onto a phone

#73

Post by liabru »

joe_the_user wrote: post #65
liabru wrote: post #51

This is great. I particularly like that they also automatically generated dirty versions for their training set, because that's exactly what I ended up doing for my dissertation project (a computer vision system [1] that automatically referees Scrabble boards). I also used dictionary analysis and the classifier's own confusion matrix to boost its accuracy. If you're also interested in real time OCR like this, I did a…

It seems your dissertation paper is behind something password protected [1]. It would be nice to see that too. Can't get [1] https://www.dcs.shef.ac.uk/intranet/teaching/campus/projects...

Hmm looks like they have, well here's another link to it: https://dl.dropboxusercontent.com/u/1672291/scrabble-referee...

Re: How Google Translate squeezes deep learning onto a phone

#75

Post by JeremyBanks »

ocdtrekkie wrote: post #16
teraflop wrote: post #12

Earlier quoted context omitted.

> What makes one kind of neural net 'deep' and are all the other ones suddenly 'shallow' ? If this is a serious question, then googling "what is a deep neural network" would take you to any number of explanations. But to summarize very briefly, it's not a buzzword; it's a technical term referring to a network with multiple nonlinear layers that are chained together in sequence. Deep networks have been talked about fo…

So was Word Lens doing this before Google even bought them? Because Word Lens worked fine, locally on a phone, long before Google was doing it's whole deep learning thing.

It's not entirely clear to me, but this sentence from the article:

In the end, we were able to get our networks to give us significantly better results while running about as fast as our old system—great for translating what you see around you on the fly.

Suggests that they previously were not using neural networks, or were using less powerful ones.

Re: How Google Translate squeezes deep learning onto a phone

#76

Post by arielby »

thaumasiotes wrote: post #68
Lawtonfogle wrote: post #36

Earlier quoted context omitted.

If there is magic to be found, it may be in that question. Why about graphs (namely the subset that are deep neural networks) allow them to not only contain such powerful heuristics, but also allow them to be created from scratch with barely any knowledge of the problem domain. As a side note, I was playing a board game last night (Terra Mystica I believe) and wondering if you could get 5 different neural networks to…

The best Go bot approach (as of some years ago, but it's not like neural networks are a new idea) uses a very different strategy. Specifically, the strategy of "identify a few possible moves, simulate the game for several steps after each move using a very stupid move-making heuristic instead of using this actual strategy recursively , and then pick the move that yielded the best simulated board state".

The "use a stupid heuristic as part of the evaluation function" is is, in fact, also an important part of Chess AI's mode (as Quiescence Search), through for different reasons.

Re: How Google Translate squeezes deep learning onto a phone

#78

Post by shostack »

hyperbovine wrote: post #66
beambot wrote: post #34

Earlier quoted context omitted.

You could say very much the same about the brain... > [...] the "black magic" part comes mostly from their mathematical nature and very little from them being "inteligent computers". A brain is a graph, in which a subset of neurons are "inputs", some are outputs, and others are "hidden". The nodes are interconnected between each other in a fashion, which is called the "topology" or sometimes "architecture" of the net…

It's not really that deep, imo: a typical deep net these days has O(10^8) parameters (e.g. http://stackoverflow.com/questions/28232235/how-to-calculate... ). You can store a hell of a lot of patterns in that many parameters, making them the best pattern matchers the world has ever seen. (Un)fortunately, pattern matching != intelligence. More interesting deep questions for which there is precious little theory revolve…

Is "pattern matching != intelligence" what occurred when the Google image recognition stuff in the news recently was shown to recognize the pattern of a "dumbbell" as always having a large muscular arm attached to it?

Seemed like a great way to highlight the limitations of patterns.

Re: How Google Translate squeezes deep learning onto a phone

#79

Post by aylons »

raverbashing wrote: post #60
eosrei wrote: post #37

I used this in Brazil this last March to read menus. It works extremely well. The mistranslations make it even more fun. Much faster than learning Portuguese! I took a few screen shots. Aligning the phone, focus, light, shadows on the small menu font was difficult. You must keep steady. Sadly, I ended up hitting the volume control on this best example. Tasty cockroaches! Ha! http://imgur.com/j9iRaY0

Interesting It seems it can't really handle context, so 'cockroaches' may have been a mistranslation of 'cheap' in some contexts, as the 'it had stopped chestnut' may have simply been 'brazil nuts'

Most probably the OCR read "batata" (potato) as "barata" (cockroach).

Re: How Google Translate squeezes deep learning onto a phone

#80

Post by deepnet »

thaumasiotes wrote: post #68
Lawtonfogle wrote: post #36

Earlier quoted context omitted.

If there is magic to be found, it may be in that question. Why about graphs (namely the subset that are deep neural networks) allow them to not only contain such powerful heuristics, but also allow them to be created from scratch with barely any knowledge of the problem domain. As a side note, I was playing a board game last night (Terra Mystica I believe) and wondering if you could get 5 different neural networks to…

The best Go bot approach (as of some years ago, but it's not like neural networks are a new idea) uses a very different strategy. Specifically, the strategy of "identify a few possible moves, simulate the game for several steps after each move using a very stupid move-making heuristic instead of using this actual strategy recursively , and then pick the move that yielded the best simulated board state".

Monte Carlo Tree Search ( Random playout ) is currently the best computer strategy for evaluating a Go position.

This is likely due to the way Go works , random playout provides a rough estimate of who controls what territory ( this is how Go is scored ).

Recently two deep-learning papers showed very impressive results.

http://arxiv.org/abs/1412.3409

http://arxiv.org/abs/1412.6564

The neural networks were tasked with predicting what move an expert would make given a position.

The MCTS takes a long time 100,000 playouts are typical - once trained the neural nets are orders of magnitude faster.

The neural nets output a probability for each move ( that an expert would make that move ) - all positions are evauluated in a single forward pass.

Current work centers around combining the two approaches, MCTS evaluates the best suggestions from the neural net.

Expert Human players are still unbeatable by computer Go.

Post reply on HN