Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Introduction to Recurrent Neural Networks

victorzhou.com

Post reply on HN
Print view View original

21–26 of 26 posts

Re: An Introduction to Recurrent Neural Networks

#21

Post by MiroF »

probably_wrong wrote: post #5
ape4 wrote: post #4

I doubt Google Translate uses RNN. They use Statistical Machine Translation. Oops, I see they switched to NN in 2016. https://en.wikipedia.org/wiki/Google_Translate

I feel I should point out that those two things are not mutually exclusive. RNNs are, after all, a mechanism for learning conditional probabilities. I think the confusion comes from Google itself, who used the term "Statistical Machine Translation" (SMT) to refer to "Rule-based SMT". Both methods are statistical.

SMT as a term of art in the translation field means rule based SMT... it's not a google particularity, I see the same usage in both industry and academia

Re: An Introduction to Recurrent Neural Networks

#23

Post by mrfox321 »

stereolambda wrote: post #14

It's worth noting that apparently (as I learned lately) RNNs are going slightly out of fashion because they are hard to parallelize and have trouble remembering important stuff at larger distances. Transformers are proposed as a possible solution - very roughly speaking, they use attention mechanisms instead of recurrent memory and can run in parallel. I have to say that while I understand the problems with recurrent…

It depends on the problem domain. Transformers are useful for NLP (language modeling, machine translation), but RNNs (and CNNs) are still being used in speech-to-text. When the input/output relationships are monotonic, transformers are probably too general.

Re: An Introduction to Recurrent Neural Networks

#24

Post by cheez »

vzhou842 wrote: post #12
cheez wrote: post #9

This kind of article is absolutely the thing everyone new to deep learning/neural networks should read. I wish there was one for each type of algorithm.

I do have similar articles for Neural Networks (MLP) and CNNs: --- NN: https://victorzhou.com/blog/intro-to-neural-networks/ NN HN discussion: https://news.ycombinator.com/item?id=19320217 --- CNN: https://victorzhou.com/blog/intro-to-cnns-part-1/ https://victorzhou.com/blog/intro-to-cnns-part-2/ CNN HN discussions: https://news.ycombinator.com/item?id=19981736 https://news.ycombinator.com/item?id=20064900

Awesome!
Post reply on HN