Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CS224d: Deep Learning for Natural Language Processing

cs224d.stanford.edu

Post reply on HN
Print view View original

21–30 of 33 posts

Re: CS224d: Deep Learning for Natural Language Processing

#21

Post by agibsonccc »

xigency wrote: post #20
aabajian wrote: post #18

I was fortunate to take this class the first time it was offered. I found it a great introduction to the material, but a bit over my head. Deep learning requires a strong grasp of linear algebra - and particularly at the "Stanford" level. My undergrad didn't prepare me well for visualizing outer products and matrix / tensor derivatives. Once you get over those hurdles, deep learning is quite fun . It often works like…

That example is just a simple Markov model. Using the 'T9' method of completing text is more of a novelty than something useful. I also have trouble with 'complete the sentence' type of programs because they don't actually create new ideas, they just rehash data. (It does have use in OCR, voice recognition, and typing/texting.) I agree that the math can be complex, but I think it boils down to probability and the not…

I think a common example along the same vein is the analogies trick you always see. It's been demonstrated to death at this point but the great thing here is word2vec more or learns to predict the next word using hierarchical softmax so he's not technically "wrong" since this is the training objective. It's good to clarify it though.

Re: CS224d: Deep Learning for Natural Language Processing

#22

Post by xigency »

agibsonccc wrote: post #21
xigency wrote: post #20

Earlier quoted context omitted.

That example is just a simple Markov model. Using the 'T9' method of completing text is more of a novelty than something useful. I also have trouble with 'complete the sentence' type of programs because they don't actually create new ideas, they just rehash data. (It does have use in OCR, voice recognition, and typing/texting.) I agree that the math can be complex, but I think it boils down to probability and the not…

I think a common example along the same vein is the analogies trick you always see. It's been demonstrated to death at this point but the great thing here is word2vec more or learns to predict the next word using hierarchical softmax so he's not technically "wrong" since this is the training objective. It's good to clarify it though.

Yes, and I guess that goes along with the black box idea. What function you are training for depends on your needs, and that can be achieved with deep learning or soft AI.

Re: CS224d: Deep Learning for Natural Language Processing

#23

Post by akhilcacharya »

aabajian wrote: post #18

I was fortunate to take this class the first time it was offered. I found it a great introduction to the material, but a bit over my head. Deep learning requires a strong grasp of linear algebra - and particularly at the "Stanford" level. My undergrad didn't prepare me well for visualizing outer products and matrix / tensor derivatives. Once you get over those hurdles, deep learning is quite fun . It often works like…

Where'd you do your undergrad?

Re: CS224d: Deep Learning for Natural Language Processing

#24

Post by viksit »

It's interesting that this is trending at the same time as a RNN based NLP powered assistant that I've just posted on HN.

It uses a lot of the same concepts - recurrent nets and word embeddings. If you guys want to play around with it in a real life scenario, head over there to check it out. Discussion here [1]. Link here. [2]

[1] https://news.ycombinator.com/item?id=10060074

[2] http://getmyra.co

Edit: Update wrong link.

Re: CS224d: Deep Learning for Natural Language Processing

#25

Post by mirumirai »

napolesmarble wrote: post #11
pkaye wrote: post #3

I'm still not clear on the difference between deep learning and machine learning. Also are there good primer books on machine learning fundamentals?

"The Nature of Code" is a book I've heard repeated is pretty good. It's free to browse online if your interested: http://natureofcode.com/book/chapter-10-neural-networks/

I've gone through much of this book. It's really good, but it's definitely not a book on machine learning fundamentals - more on complexity and simulations.

That chapter however is a nice intro to neural networks, and the previous chapter is a nice intro to genetic algorithms.

Re: CS224d: Deep Learning for Natural Language Processing

#26

Post by gherkin0 »

xigency wrote: post #20
aabajian wrote: post #18

I was fortunate to take this class the first time it was offered. I found it a great introduction to the material, but a bit over my head. Deep learning requires a strong grasp of linear algebra - and particularly at the "Stanford" level. My undergrad didn't prepare me well for visualizing outer products and matrix / tensor derivatives. Once you get over those hurdles, deep learning is quite fun . It often works like…

That example is just a simple Markov model. Using the 'T9' method of completing text is more of a novelty than something useful. I also have trouble with 'complete the sentence' type of programs because they don't actually create new ideas, they just rehash data. (It does have use in OCR, voice recognition, and typing/texting.) I agree that the math can be complex, but I think it boils down to probability and the not…

> and one I found interesting was a model for summarizing text, including chapters, books, and other writing.

Do you have a cite for that?

Re: CS224d: Deep Learning for Natural Language Processing

#27

Post by aabajian »

aadamson wrote: post #19
aabajian wrote: post #18

I was fortunate to take this class the first time it was offered. I found it a great introduction to the material, but a bit over my head. Deep learning requires a strong grasp of linear algebra - and particularly at the "Stanford" level. My undergrad didn't prepare me well for visualizing outer products and matrix / tensor derivatives. Once you get over those hurdles, deep learning is quite fun . It often works like…

Your paper is the only paper listed above mine on the reports page! Solid last name optimization

lol and my first name is "Aaron" thank my parents.

Re: CS224d: Deep Learning for Natural Language Processing

#28

Post by aabajian »

akhilcacharya wrote: post #23
aabajian wrote: post #18

I was fortunate to take this class the first time it was offered. I found it a great introduction to the material, but a bit over my head. Deep learning requires a strong grasp of linear algebra - and particularly at the "Stanford" level. My undergrad didn't prepare me well for visualizing outer products and matrix / tensor derivatives. Once you get over those hurdles, deep learning is quite fun . It often works like…

Where'd you do your undergrad?

I'd rather not bash my undergrad, but suffice to say I tutored intro linear algebra and was very comfortable with eigenvalues, eigenvectors, Gaussian elimination, and that kind of stuff. What was tricky in 224d was taking the gradients with respect to specific components of a matrix. In the end you get comfortable with what the result should look like, but if you actually write the matrix indices down, it's quite hairy (mostly tensor product(s) that can be rewritten as matrix outer products).

Re: CS224d: Deep Learning for Natural Language Processing

#29

Post by xigency »

gherkin0 wrote: post #26
xigency wrote: post #20

Earlier quoted context omitted.

That example is just a simple Markov model. Using the 'T9' method of completing text is more of a novelty than something useful. I also have trouble with 'complete the sentence' type of programs because they don't actually create new ideas, they just rehash data. (It does have use in OCR, voice recognition, and typing/texting.) I agree that the math can be complex, but I think it boils down to probability and the not…

> and one I found interesting was a model for summarizing text, including chapters, books, and other writing. Do you have a cite for that?

Yes, I just found it actually. The article targets short stories specifically.

A. Kazantseva and S. Szpakowicz, "Summarizing Short Stories." Assoc. for Computational Linguistics, vol. 36, no. 1, pp. 71-109, Mar. 2010. [Online]. Available: http://www.mitpressjournals.org/doi/abs/10.1162/coli.2010.36...

There is a PDF available. It's about 40 pages long.

Post reply on HN