Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Natural Language Processing Demystified (Part One)

nlpdemystified.org

Post reply on HN
Print view View original

21–30 of 45 posts

Re: Show HN: Natural Language Processing Demystified (Part One)

#21

Post by sam_lowry_ »

PainfullyNormal wrote: post #4
jll29 wrote: post #3

NLP researcher here. It's great to see many offerings for courses and tutorials, and NLP has made a lot of progress, in terms of both its science as well as its re-usable software artifacts (ibraries & notebooks, standalone tools). But what saddens me is too many people are trying to dive into NLP without trying to understand language & linguistics first. For example, you can run a part of speech (POS) tagger in thre…

> I recommend people spend a bit of time to read an(y) introduction to linguistics textbook Do you have a favorite you can recommend?

Elements by Tesnière. I am not kidding, there is a shitload of knowledge there, largely forgotten by the time NLP merged with CompSci.

Jurafsky and Martin, Manning and Schutze are great books for computer scientists but these do not teach about the language.

Re: Show HN: Natural Language Processing Demystified (Part One)

#22

Post by [deleted] »

sam_lowry_ wrote: post #21
PainfullyNormal wrote: post #4

Earlier quoted context omitted.

> I recommend people spend a bit of time to read an(y) introduction to linguistics textbook Do you have a favorite you can recommend?

Elements by Tesnière. I am not kidding, there is a shitload of knowledge there, largely forgotten by the time NLP merged with CompSci. Jurafsky and Martin, Manning and Schutze are great books for computer scientists but these do not teach about the language.

[deleted]

Re: Show HN: Natural Language Processing Demystified (Part One)

#23

Post by adamsmith143 »

jll29 wrote: post #3

NLP researcher here. It's great to see many offerings for courses and tutorials, and NLP has made a lot of progress, in terms of both its science as well as its re-usable software artifacts (ibraries & notebooks, standalone tools). But what saddens me is too many people are trying to dive into NLP without trying to understand language & linguistics first. For example, you can run a part of speech (POS) tagger in thre…

I'm not at all sympathetic to this viewpoint. The Deep Learning revolution has shown us time and time again that Deep Learning experts universally outperform SME on modelling performance. I an almost 100% certain that the teams building big Transformers which are now by far the best NLP models (OpenAI, Meta, Google Brain, Deepmind, etc) are not made up of linguistic experts but Deep Learning experts.

Re: Show HN: Natural Language Processing Demystified (Part One)

#24

Post by mothcamp »

irln wrote: post #19
mothcamp wrote: post #2

Hi HN: I published part one of my free NLP course. The course is intended to help anyone who knows Python and a bit of math go from the very basics all the way to today's mainstream models and frameworks. I strive to balance theory and practice and so every module consists of detailed explanations and slides along with a Colab notebook (in most modules) putting the theory into practice. In part one, we cover text pre…

The interface is great. Did you create the front-end/back-end from scratch?

Thank you. Yep. It's all statically-generated pages using Next.js with a single Next.js API route for the subscription. All hosted on Netlify.

Re: Show HN: Natural Language Processing Demystified (Part One)

#25

Post by mothcamp »

gattilorenz wrote: post #14
LunaSea wrote: post #10

Earlier quoted context omitted.

Is this still true in an era where most NLP problems use language models as a solution?

I think so. First of all, knowing some linguistics will teach you terms and concepts (e.g. parse tree, phrase, morpheme, phoneme, etc) that will both help you find relevant literature and avoid reinventing terms for stuff that is widely known (so others will more readily find your work). Language models are currently the best solution for many problems, but it's hard to predict how we will move forward from here. May…

I agree. I think having linguistics knowledge can help especially in applied situations. Linguistics knowledge can help create fallback systems when an ML system fails, or help build rules to amplify or dampen the confidence of a response from an ML system, or aid in the engineering of a system (all that comes before or after the ML blackbox).

Sort of like an algorithmic trader knowing market microstructure intimately (versus only pure statistics).

Re: Show HN: Natural Language Processing Demystified (Part One)

#26

Post by amitport »

adamsmith143 wrote: post #23
jll29 wrote: post #3

NLP researcher here. It's great to see many offerings for courses and tutorials, and NLP has made a lot of progress, in terms of both its science as well as its re-usable software artifacts (ibraries & notebooks, standalone tools). But what saddens me is too many people are trying to dive into NLP without trying to understand language & linguistics first. For example, you can run a part of speech (POS) tagger in thre…

I'm not at all sympathetic to this viewpoint. The Deep Learning revolution has shown us time and time again that Deep Learning experts universally outperform SME on modelling performance. I an almost 100% certain that the teams building big Transformers which are now by far the best NLP models (OpenAI, Meta, Google Brain, Deepmind, etc) are not made up of linguistic experts but Deep Learning experts.

These groups are not mutually exclusive.

Re: Show HN: Natural Language Processing Demystified (Part One)

#27

Post by sam_lowry_ »

amitport wrote: post #26
adamsmith143 wrote: post #23

Earlier quoted context omitted.

I'm not at all sympathetic to this viewpoint. The Deep Learning revolution has shown us time and time again that Deep Learning experts universally outperform SME on modelling performance. I an almost 100% certain that the teams building big Transformers which are now by far the best NLP models (OpenAI, Meta, Google Brain, Deepmind, etc) are not made up of linguistic experts but Deep Learning experts.

These groups are not mutually exclusive.

In practice, they are, AFAIK.

Re: Show HN: Natural Language Processing Demystified (Part One)

#28

Post by nmstoker »

mirker wrote: post #16
scarface74 wrote: post #12

Earlier quoted context omitted.

Everything eventually gets boiled down to libraries. The purpose of technology is to get things done. I could say the same that it makes me sad that today’s developers use high level languages without ever knowing assembly. A chip designer could say that assembly language developers are saddened that the assembly language programmer never had to learn how processors are created.

It’s fine when the library is a tight abstraction. Unfortunately, ML libraries are leaky. Example: take a classification model and change the output dimensions without understanding the model.

Yes, the challenge people then face is that if they lack too much intuition for the subject, they can't spot obvious issues.

We've all seen how ML people don't necessarily have to have the skills to solve a problem (ie i don't need to speak Vietnamese to make a "passable" ML translator) but it's not long before the lack of knowledge starts to show up embarrassing shortfalls - being too arms length about any topic is a recipe for disaster!

Post reply on HN