Natural Language Processing for the Working Programmer
1–10 of 48 posts
Re: Natural Language Processing for the Working Programmer
#2I was working with NLP, and its various toolkits (python-nltk I'm looking at you). The thing about NLP is, there just isn't enough libraries (for humans) to simply plug NLP into use. Even nltk, the premier python library for NLP, seems to be an NLP core-library for building NLP solutions, rather than for building NLP-powered apps. It also seems to extremely unpythonic.
Is there a missing link there? I don't know. So I took a few days and built an extremely simple NER (named entity recognition) engine and made it extremely easy for any programmer to begin using.
See http://blog.nerily.com/howto-train-your-own-modelset-for-you... . We'll see how NLP becomes more easily accessible with better tools to come over time.
Re: Natural Language Processing for the Working Programmer
#3I was working with NLP, and its various toolkits (python-nltk I'm looking at you). The thing about NLP is, there just isn't enough libraries (for humans) to simply plug NLP into use. Even nltk, the premier python library for NLP, seems to be an NLP core-library for building NLP solutions, rather than for building NLP-powered apps. It also seems to extremely unpythonic. Is there a missing link there? I don't know. So…
yeah, for the vast majority of uses, most people really want to do just a fairly small set of things fairly well.
NER comes to mind, lots and lots and lots of toolkits for building up to NER, but very few that let submit English text and get back a list of people, places and things without having to virtually build my own NER system from scratch anyways.
Give me NER, Entity relationships (ER) and a couple kinds of sentiment analysis scoring (SA) (which can be jump started with decent NER) and I've pretty much exhausted 95% of what I'd ever want to do.
I really really really don't need yet another library to do sentence tokenization, term tokenization, tf counting and stemming. If I was building a free text indexer or bayesian filter or some such it might be useful, but I'm probably not, there are far better solutions to those domain than I'm likely to come up with, but there aren't for NER, ER and SA.
Re: Natural Language Processing for the Working Programmer
#4I wonder if PDF or Epub version is available? couldn't find it on the site.
Re: Natural Language Processing for the Working Programmer
#5I wonder if PDF or Epub version is available? couldn't find it on the site.
Here you go
http://nlpwp.org/nlpwp.pdf
Re: Natural Language Processing for the Working Programmer
#6Re: Natural Language Processing for the Working Programmer
#7I was working with NLP, and its various toolkits (python-nltk I'm looking at you). The thing about NLP is, there just isn't enough libraries (for humans) to simply plug NLP into use. Even nltk, the premier python library for NLP, seems to be an NLP core-library for building NLP solutions, rather than for building NLP-powered apps. It also seems to extremely unpythonic. Is there a missing link there? I don't know. So…
yeah, for the vast majority of uses, most people really want to do just a fairly small set of things fairly well. NER comes to mind, lots and lots and lots of toolkits for building up to NER, but very few that let submit English text and get back a list of people, places and things without having to virtually build my own NER system from scratch anyways. Give me NER, Entity relationships (ER) and a couple kinds of se…
I can't recommend any libraries "for humans" for this, there are APIs out there for it.
The main problem with many NLP libs (and data mining applications in general) has a lot to do with how much memory good models can take up when doing it in order for it to be accurate at all. Here are a few APIs and libs that might be useful though:
(Disclaimer: publisher of this one here)
https://www.mashape.com/agibsonccc/semantic-analytic
There are other text processing APIs on there as well. As for libraries, I primarily come from the JVM camp for NLP, but I would recommend the following libraries:
http://nlp.stanford.edu/software/index.shtml (Comprehensive) http://code.google.com/p/clearnlp/ (Fairly simple)
My favorite is cleartk (http://code.google.com/p/cleartk/ ) mainly due to the fact it's a consistent interface, but UIMA itself can be a difficult toolchain to pick up, and I could understand most of these being overkill for many simple applications people may have in mind.
Re: Natural Language Processing for the Working Programmer
#8Earlier quoted context omitted.
yeah, for the vast majority of uses, most people really want to do just a fairly small set of things fairly well. NER comes to mind, lots and lots and lots of toolkits for building up to NER, but very few that let submit English text and get back a list of people, places and things without having to virtually build my own NER system from scratch anyways. Give me NER, Entity relationships (ER) and a couple kinds of se…
I can't recommend any libraries "for humans" for this, there are APIs out there for it. The main problem with many NLP libs (and data mining applications in general) has a lot to do with how much memory good models can take up when doing it in order for it to be accurate at all. Here are a few APIs and libs that might be useful though: (Disclaimer: publisher of this one here) https://www.mashape.com/agibsonccc/semant…
I've heard some good things about OpenNLP as well
http://opennlp.apache.org/
but haven't had the time to look at it with any detail.
Re: Natural Language Processing for the Working Programmer
#9Earlier quoted context omitted.
I can't recommend any libraries "for humans" for this, there are APIs out there for it. The main problem with many NLP libs (and data mining applications in general) has a lot to do with how much memory good models can take up when doing it in order for it to be accurate at all. Here are a few APIs and libs that might be useful though: (Disclaimer: publisher of this one here) https://www.mashape.com/agibsonccc/semant…
I've heard some good things about OpenNLP as well http://opennlp.apache.org/ but haven't had the time to look at it with any detail.
OpenNLP is great. I've used it for a lot of subtasks, but nothing that produces end results as described earlier. It's an amazing library for building NLP systems, but doesn't produce anything directly (named entity recognition, etc) Typically it's coupled with other libraries.
The big problem I think with NLP in general is typically to do anything, you need a pipeline. (Sentence segmentation, tokenization, part of speech tagging) usually at a bare minimum. Then from there you can do named entity recognition or other tasks that produce actual usable results.
Re: Natural Language Processing for the Working Programmer
#10I was working with NLP, and its various toolkits (python-nltk I'm looking at you). The thing about NLP is, there just isn't enough libraries (for humans) to simply plug NLP into use. Even nltk, the premier python library for NLP, seems to be an NLP core-library for building NLP solutions, rather than for building NLP-powered apps. It also seems to extremely unpythonic. Is there a missing link there? I don't know. So…
While it's possible that some parts of NLTK may be unpythonic, I disagree that it's somehow inaccessible or hard to use. The NLTK project is fairly well-documented, and I've never had any difficulty using it. Also, all the NLTK code I've ever examined looks Pythonic to me, unlike the Java-style Python that I sometimes see coming out of academia; they follow PEP8 and have a developer style guide. There may be pockets of unpythonic code, but I've not come across any.
NLTK is also free and open-source, with a liberal license (Apache), which I appreciate greatly.
Also, I don't understand what you mean by nltk being a library for building NLP solutions, rather than NLP-powered apps. Can you expand on that?
Re: NER, I found this gist (not my own) for a basic example of entity extraction with nltk: https://gist.github.com/322906/90dea659c04570757cccf0ce1e6d2... This looks pretty straight-forward to me. What NLP toolkit are you using for the NER service your Chrome extension calls, if not NLTK?