Live data from Hacker News

Oxford Deep NLP – An advanced course on natural language processing

github.com

41–50 of 68 posts

Re: Oxford Deep NLP – An advanced course on natural language processing

#41

How does the course content and rigor compare to the Stanford deep learning for NLP course? From a cursory glance at the practicals, it seems like the Stanford version has more variety and depth of problems.

If you're well-versed in DL + nuances of text-mining, then the Stanford one might be great. But, if you're a beginner like me, you'll find it a bit hard to catch up [1]. The Stanford assignments + Tensorflow class were very helpful. The assignments especially really get you thinking.

[1] I took the Stanford one and went through the videos (both 2015,2016).

Re: Oxford Deep NLP – An advanced course on natural language processing

#42

here is what I don't understand about deep NLP (please keep in mind that I just began exploring this field): I am currently working on an algorithm that uses elementary text cues in combination with large data-table lookups to determine things like relevant keywords of news articles scraped from various sites. I have given my results to hundreds of people independently to provide me with some feedback regarding the q…

> 2. if deep nlp can provide us with more efficiency than what is stated above then wouldn't it be safe to assume that is UNREASONABLY efficient?

Why? Neural nets can already detect skin cancer as well as human dermatologists [1]. Why would you assume that your algorithm is the peak of efficiency and anything that performs better is "unreasonable"?

[1] https://news.ycombinator.com/item?id=13484372

Re: Oxford Deep NLP – An advanced course on natural language processing

#44
Hi, i am an absolute beginner in this field or precisely just a person who finds it very interesting. I wonder if the principles of NLP taught in this course are language agnostic/ can they be applied to languages apart of English/German etc. especially to the languages in other language families?

Re: Oxford Deep NLP – An advanced course on natural language processing

#45
post #26

What are the practical uses of language modelling RNNs ? (apart from writing grammar/syntax checkers)

I wonder if they could be used to improve speech recognition accuracy. So you'd have two models running when someone utters a sentence: the first would generate the x most likely phrases that it thinks were spoken, and the second (the RNN) would select the highest ranked 'plausible' sentence (i.e. a sentence it would have been able to generate itself). I guess that's a bit indirect, but these RNNs are essentially lea…

Language models are a standard component of speech recognition. Although N-gram LMs are used much more often than RNN LMs.

Re: Oxford Deep NLP – An advanced course on natural language processing

#46

here is what I don't understand about deep NLP (please keep in mind that I just began exploring this field): I am currently working on an algorithm that uses elementary text cues in combination with large data-table lookups to determine things like relevant keywords of news articles scraped from various sites. I have given my results to hundreds of people independently to provide me with some feedback regarding the q…

When you are doing Machine Learning you should always have a simple baseline and only use more complicated algorithms when they improve over your baseline.

Re: Oxford Deep NLP – An advanced course on natural language processing

#47
post #26

What are the practical uses of language modelling RNNs ? (apart from writing grammar/syntax checkers)

I wonder if they could be used to improve speech recognition accuracy. So you'd have two models running when someone utters a sentence: the first would generate the x most likely phrases that it thinks were spoken, and the second (the RNN) would select the highest ranked 'plausible' sentence (i.e. a sentence it would have been able to generate itself). I guess that's a bit indirect, but these RNNs are essentially lea…

Standard neural network-based speech recognition pipelines (i.e. RNN + CTC) always use a language model. Unlike a seq2seq model (or any autoregressive model, or a structured prediction output), CTC models output timesteps as conditionally independent. Hence, everyone uses an RNN LM or n-gram LM or both when retrieving probable sequences from a CTC model (e.g. with beam search).

Re: Oxford Deep NLP – An advanced course on natural language processing

#48

here is what I don't understand about deep NLP (please keep in mind that I just began exploring this field): I am currently working on an algorithm that uses elementary text cues in combination with large data-table lookups to determine things like relevant keywords of news articles scraped from various sites. I have given my results to hundreds of people independently to provide me with some feedback regarding the q…

You don't have to now. But what if you were to change the problem a bit, you'd need to reinvent those "elementary text cues", right? With deep learning (or, more generally, representation learning) you can simply change the training data and reuse the rest of your algorithm. Jure Leskovec has a paper, node2vec, which describes this well:

> A typical solution in- volves hand-engineering domain-specific features based on expert knowledge. Even if one discounts the tedious effort required for feature engineering, such features are usually designed for specific tasks and do not generalize across different prediction tasks. An alternative approach is to learn feature representations by solving an optimization problem [4]. The challenge in feature learn- ing is defining an objective function, which involves a trade-off in balancing computational efficiency and predictive accura

Re: Oxford Deep NLP – An advanced course on natural language processing

#49

here is what I don't understand about deep NLP (please keep in mind that I just began exploring this field): I am currently working on an algorithm that uses elementary text cues in combination with large data-table lookups to determine things like relevant keywords of news articles scraped from various sites. I have given my results to hundreds of people independently to provide me with some feedback regarding the q…

I've worked in the NLP research area (mostly with statistical metrics), and I can safely say that 80% (precision) is the empirical threshold that most metrics are able to reach quite easily. The threshold between 80% and 90% starts to get difficult and above that you have to do some tweaking to adapt to the specifics of your problem.

With that said, your values do seem to be in line with what I consider to be easily reachable, so it kind-of depends on how much work you need to do with the neural networks to extract those keywords. I'm not very knowledgeable on how NN are applied to this field, but I'm assuming that a drawback of that approach is that it may resemble a black-box in the sense that it may be hard to tweak the internals.

I prefer statistical metrics because they seem more simple to derive. For instance, you can think of things like "a relevant keyword is usually related with (or closer to) other relevant keywords" and you can test that hypothesis only by counting distances between words. This is what I've done in 2012 with quite good values, you can check the paper here: http://www.sciencedirect.com/science/article/pii/S1877050912...

Re: Oxford Deep NLP – An advanced course on natural language processing

#50

here is what I don't understand about deep NLP (please keep in mind that I just began exploring this field): I am currently working on an algorithm that uses elementary text cues in combination with large data-table lookups to determine things like relevant keywords of news articles scraped from various sites. I have given my results to hundreds of people independently to provide me with some feedback regarding the q…

I've worked in the NLP research area (mostly with statistical metrics), and I can safely say that 80% (precision) is the empirical threshold that most metrics are able to reach quite easily. The threshold between 80% and 90% starts to get difficult and above that you have to do some tweaking to adapt to the specifics of your problem. With that said, your values do seem to be in line with what I consider to be easily…

> I prefer statistical metrics because they seem more simple to derive.

That's exactly how I view it as well. My goal for this project is to reach a 90% "perfect" score. And in that case, ML seems to not even be needed. Perhaps the gap between 90 and 95-100% is where ML can help add value. But that in itself is what #3 is about in my original post.

Thank you for confirming my suspicions regarding the threshold though!

Post reply on HN