Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Get into Natural Language Processing

blog.ycombinator.com

Post reply on HN
Print view View original

51–60 of 80 posts

Re: How to Get into Natural Language Processing

#51

Post by ktRolster »

hiddencost wrote: post #49
ktRolster wrote: post #44

Earlier quoted context omitted.

there's no such thing as grammar I'm not sure what you mean by this. Certainly adjectives are a thing, and if I learn a new adjective, "anguilliform" for example, I have never heard that in context, but I know exactly how to use it. That is grammar, right?

I'm being hyperbolic, but what I'm suggesting is that grammar is a convenient fiction. The part where you speak of "how to use it" points to the break down in your thinking. (Side note: look into the difference between prescriptive and descriptive linguistics for a sense of where I'm coming from on that point.) So, you learn a new adjective. Surely you can use it like any other adjective, right? Sure. But someone can…

what I'm suggesting is that grammar is a convenient fiction.

I don't think that's right either. It's not fiction, it's definitely something real.

Re: How to Get into Natural Language Processing

#52

Post by amelius »

visarga wrote: post #48

If you want to play with NLP, then just try Gensim, sklearn and Keras. If you're serious about NLP, it's hard stuff, you need a PHD in the field. In a way, vision is easier. Instead of discrete symbols (words) it's continuous signal which are much easier to interpret and generate from neural networks. By comparison, best language models are behind best image generation models (2-3 years behind, in my estimation). For…

So what you are saying is that a computer would be able to understand sign-language more easily than spoken language?

Re: How to Get into Natural Language Processing

#54

Post by [deleted] »

hiddencost wrote: post #40
throwaway91111 wrote: post #37

Earlier quoted context omitted.

This seems like a great argument for automatic grammar learning. How far has current research taken us in that direction? I'd much rather feed it edge cases to accommodate than rules to follow.

They tried that in the 70s and 80s until they realized they were wasting their time. Probabilistic context free grammars were a thing, also. Long story short, there's no such thing as grammar but they're a nice fiction for talking about communication. The deeper you get into NLP the more you (1) see what jelinek was talking about when he said "every time I fire a linguist, accuracy goes up" (he was hiring physicists…

[deleted]

Re: How to Get into Natural Language Processing

#55

Post by mercer »

chch wrote: post #33
ice109 wrote: post #31

Earlier quoted context omitted.

re the council people sentences: I don't understand the problem. they're ill-defined sentences. we use heuristics to parse them but those heuristics can fail (the council denied the demonstrators permit because they feared violence... and the council was obliging). just teach the computer the heuristics like we learn them.

That's exactly the issue. The way we learn them is through world experience, which is sometimes hard to figure out how to transfer into a computer. Example: "I dropped the egg on my glass living room table and it broke!" "I dropped my hammer on my glass living room table and it broke!" These are both ill-defined semantically, but if you asked most native English speakers "what broke" for each sentence, they'd probabl…

I wonder if at least in the context of chat bots these kinds of problems are mitigated or avoided by the conversational nature of these bots.

In the glass table example, the bot could be explicitly 'dumb' or pedantic and ask for clarification. Or perhaps even better, the bot could simulate natural conversation: assume one or the other (with maybe a dash of built-in knowledge about the world), make that assumption explicit to the user, and allow the user to correct it. This might even make the bot more human and pleasant to interact with.

Re: How to Get into Natural Language Processing

#56

Post by jventura »

demonshalo wrote: post #2

How? just get started working on a fun problem. A good place to start is keyword extraction. You don't need a PhD or expensive tools. All you need is some free time and willingness to read some cool stuff. Copy a few articles into text files and get working on implementing some of these methods until you have enough of an understanding to construct your own methods for the fun of it. Here's some good reading material…

I would say that a good example for starting in this field would be to implement something like Tf-Idf [0] for identifying keywords on a set of documents. I don't know where one can find current datasets for this, but I made WikiCorpusExtractor [1] to build sets of documents from the Wikipedia.

The only thing one really needs is to count the frequency of words in each document and do very simple math. Tf-Idf is still very relevant today and provides you with a very good idea on how statistics is used on text-mining.

[0] https://en.wikipedia.org/wiki/Tf%E2%80%93idf

[1] https://github.com/joaoventura/WikiCorpusExtractor

Re: How to Get into Natural Language Processing

#57

Post by demonshalo »

bosie wrote: post #7
demonshalo wrote: post #2

How? just get started working on a fun problem. A good place to start is keyword extraction. You don't need a PhD or expensive tools. All you need is some free time and willingness to read some cool stuff. Copy a few articles into text files and get working on implementing some of these methods until you have enough of an understanding to construct your own methods for the fun of it. Here's some good reading material…

Thank you very much for the reading material

u welcome!

Re: How to Get into Natural Language Processing

#58

Post by demonshalo »

jventura wrote: post #56
demonshalo wrote: post #2

How? just get started working on a fun problem. A good place to start is keyword extraction. You don't need a PhD or expensive tools. All you need is some free time and willingness to read some cool stuff. Copy a few articles into text files and get working on implementing some of these methods until you have enough of an understanding to construct your own methods for the fun of it. Here's some good reading material…

I would say that a good example for starting in this field would be to implement something like Tf-Idf [0] for identifying keywords on a set of documents. I don't know where one can find current datasets for this, but I made WikiCorpusExtractor [1] to build sets of documents from the Wikipedia. The only thing one really needs is to count the frequency of words in each document and do very simple math. Tf-Idf is still…

I started even simpler than that. I started by just eliminating stopwords and count the frequency in each word in the document itself. I did not use a set of documents as the goal was for the algorithm to be used on the spot for a single block of text.

A few months later and after many iterations + a whole lot of testing, the algorithm now can extract super relevant keywords 90%+ of the time!

I wish I knew about the WikiCorpusExtractor. Thanks for the link!

Re: How to Get into Natural Language Processing

#59

Post by demonshalo »

garysieling wrote: post #5
demonshalo wrote: post #2

How? just get started working on a fun problem. A good place to start is keyword extraction. You don't need a PhD or expensive tools. All you need is some free time and willingness to read some cool stuff. Copy a few articles into text files and get working on implementing some of these methods until you have enough of an understanding to construct your own methods for the fun of it. Here's some good reading material…

Another fun thing is to paste article text into some API, like the Watson demo, so you can see what kinds of things are possible: https://alchemy-language-demo.mybluemix.net/ I played around with this a bit to develop https://www.findlectures.com , so knowing what works/doesn't work there I'm developing some NLP scripts to support my use cases.

I never thought about this particular use-case. The subtitle for TED talks should be an ocean of info for you to extract keywords from :D Pretty neat site you got there. I will be using it. Thanks!

Re: How to Get into Natural Language Processing

#60

Post by demonshalo »

p1esk wrote: post #12

NLP right now looks like the computer vision 5 years ago: DL methods are starting to work really well, so a lot of "traditional" methods to process text might soon become obsolete. The goal is to just feed gigabytes of raw text to a huge, complex neural network, and hope it will extract relevant features.

The problem is datasets. How can you distinguish a good result from a bad result? In some cases, depending on the user, it could be both at the same time.

Most advancements in ML is not accomplished by some new super algorithm. Rather, advancements are reached when new datasets are presented!

Post reply on HN