Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Oxford Deep NLP – An advanced course on natural language processing

github.com

Post reply on HN
Print view View original

51–60 of 68 posts

Re: Oxford Deep NLP – An advanced course on natural language processing

#51

Post by demonshalo »

traek wrote: post #42
demonshalo wrote: post #39

here is what I don't understand about deep NLP (please keep in mind that I just began exploring this field): I am currently working on an algorithm that uses elementary text cues in combination with large data-table lookups to determine things like relevant keywords of news articles scraped from various sites. I have given my results to hundreds of people independently to provide me with some feedback regarding the q…

> 2. if deep nlp can provide us with more efficiency than what is stated above then wouldn't it be safe to assume that is UNREASONABLY efficient? Why? Neural nets can already detect skin cancer as well as human dermatologists [1]. Why would you assume that your algorithm is the peak of efficiency and anything that performs better is "unreasonable"? [1] https://news.ycombinator.com/item?id=13484372

I didn't say that I assume mine to be the peak. My intention was to point out that any efficiency that could be reached beyond what is statistically possible (i.e. if you only rely on statistical metrics and parsing), could be considered unreasonably efficient. At least, there is an argument for that to be made.

My method should in no way, shape or form be considered as a "peak".

Re: Oxford Deep NLP – An advanced course on natural language processing

#52

Post by demonshalo »

ma2rten wrote: post #46
demonshalo wrote: post #39

here is what I don't understand about deep NLP (please keep in mind that I just began exploring this field): I am currently working on an algorithm that uses elementary text cues in combination with large data-table lookups to determine things like relevant keywords of news articles scraped from various sites. I have given my results to hundreds of people independently to provide me with some feedback regarding the q…

When you are doing Machine Learning you should always have a simple baseline and only use more complicated algorithms when they improve over your baseline.

That's my #3 question in a nutshell. And this seems to be a rather good strategy imo. However, the way everyone is talking about ML makes it seem as a diamond bullet to solve all that is holy and sacred!

Re: Oxford Deep NLP – An advanced course on natural language processing

#54

Post by ska »

traek wrote: post #42
demonshalo wrote: post #39

here is what I don't understand about deep NLP (please keep in mind that I just began exploring this field): I am currently working on an algorithm that uses elementary text cues in combination with large data-table lookups to determine things like relevant keywords of news articles scraped from various sites. I have given my results to hundreds of people independently to provide me with some feedback regarding the q…

> 2. if deep nlp can provide us with more efficiency than what is stated above then wouldn't it be safe to assume that is UNREASONABLY efficient? Why? Neural nets can already detect skin cancer as well as human dermatologists [1]. Why would you assume that your algorithm is the peak of efficiency and anything that performs better is "unreasonable"? [1] https://news.ycombinator.com/item?id=13484372

   Neural nets can already detect skin cancer as well as human dermatologists
For what it is worth, that statement is way too strong for what the linked article and paper show.

Re: Oxford Deep NLP – An advanced course on natural language processing

#55

Post by sprobertson »

option_greek wrote: post #20
visarga wrote: post #18

Earlier quoted context omitted.

Language encoding and generation (sequence to sequence).

Could you expand a bit on that. I have come across examples generating poems from Shakespeare works and realistic looking css/javascript but still trying to find out a more realistic usecase.

Chat bots, image to caption, and question answering are some more realistic use cases of the generator side. In those cases there is some input (previous chat message, an image, a question) which is encoded into a vector sometimes referred to as a context or thought vector. The decoder/generator unfolds that vector into a series of words.

Re: Oxford Deep NLP – An advanced course on natural language processing

#56

Post by jkbschwarz »

snissn wrote: post #36
jkbschwarz wrote: post #16

Im taking this course at Oxford and they have been working through the practicals 1-3 (further ones will be posted) For anyone considering working through this outside of Oxford: I think the practicals are the real gems here and should be doable without the practical lab sessions that you get when attending the course. With that being said, they use a dataset a bit closer to a real world assignment. Therefore, it req…

Would you be able to post your own solutions for the non sense spewing ted bot? I'd like to jump into hacking on something that already works while also working through the theory from the lectures

Yes happy to share once the evaluation period is over, but sadly I dont think Im allowed to share beforehand. Will have to check when but I expect it should be in approximately 2 weeks.

In the meantime, you might want to check out this excellent blog post http://r2rt.com/recurrent-neural-networks-in-tensorflow-ii.h.... This will provide you with skeleton code to implement a character level generative model (similar to Practical 3, Task 2 with the exception that there you will generate words and not characters). Andrej Karpathy's blog post on LSTMs is also excellent and I believe he also provides the code in his repository as well http://karpathy.github.io/2015/05/21/rnn-effectiveness/

Re: Oxford Deep NLP – An advanced course on natural language processing

#57

Post by jamra »

jventura wrote: post #49
demonshalo wrote: post #39

here is what I don't understand about deep NLP (please keep in mind that I just began exploring this field): I am currently working on an algorithm that uses elementary text cues in combination with large data-table lookups to determine things like relevant keywords of news articles scraped from various sites. I have given my results to hundreds of people independently to provide me with some feedback regarding the q…

I've worked in the NLP research area (mostly with statistical metrics), and I can safely say that 80% (precision) is the empirical threshold that most metrics are able to reach quite easily. The threshold between 80% and 90% starts to get difficult and above that you have to do some tweaking to adapt to the specifics of your problem. With that said, your values do seem to be in line with what I consider to be easily…

I believe that NN provide the logical relations for you so perhaps they can be used to train and then inspected to understand the data better.

Re: Oxford Deep NLP – An advanced course on natural language processing

#58

Post by julien_c »

wlrd wrote: post #43
melqdusy wrote: post #10

Stanford's version https://web.stanford.edu/class/cs224n/ Note: the videos will be available later.

Any idea when the lectures will be up?

Last spring's session's videos (so 9 months ago) are up on Youtube: https://www.youtube.com/playlist?list=PLmImxx8Char9Ig0ZHSyTq...

Re: Oxford Deep NLP – An advanced course on natural language processing

#59

Post by mcguire »

rcpt wrote: post #48
demonshalo wrote: post #39

here is what I don't understand about deep NLP (please keep in mind that I just began exploring this field): I am currently working on an algorithm that uses elementary text cues in combination with large data-table lookups to determine things like relevant keywords of news articles scraped from various sites. I have given my results to hundreds of people independently to provide me with some feedback regarding the q…

You don't have to now. But what if you were to change the problem a bit, you'd need to reinvent those "elementary text cues", right? With deep learning (or, more generally, representation learning) you can simply change the training data and reuse the rest of your algorithm. Jure Leskovec has a paper, node2vec, which describes this well: > A typical solution in- volves hand-engineering domain-specific features based…

What is the difference between feature engineering and defining an objective function?
Post reply on HN