[deleted]
Making Text Mining Accessible to Any Developer & Non-Expert
31–33 of 33 posts
Re: Making Text Mining Accessible to Any Developer & Non-Expert
#32Text mining is one of my specialties and I have had similar ideas for a business. One thing that has stopped me is the awesome (and free for about 50K API calls a day) Open Calais service that does entity extraction and identifies some relationships between entities in input text. For document clustering there are many good open source tools that people and companies can use. The commercial Ling Pipe product does a g…
True, but problems are there to be solved. :)
We've done a lot of work around data normalization/scrubbing from a multitude of sources as part of a sister project, so I'm fairly confident about this aspect.
Curation and classification is another issue, but we have a few ideas.
As for business, you never know, just let me try to get off this Ramen based diet first. ;)
Re: Making Text Mining Accessible to Any Developer & Non-Expert
#33I'm really happy to see more people moving into this space.
I've used a number of different systems (openCalais, AlchemyAPI, Zemanta...) in a variety of projects (Sentiment analysis, document classification...), and what I've found thus far is that while each system works extremely well within some restricted application classes, none come close to being general purpose APIs for the myriad applications developers try to throw at them.
A couple of pain points I've encountered are requiring a larger than expected corpus to generate meaningful data based on overly broad scope of the platform's analysis, or the lack of ability to apply negative signals from external sources. I find there tends to remain a large quantity of logic sitting rather redundantly on the application end to post-filter what's generated.
I don't pretend to understand the level of complexity involved or what's being worked on currently (not an NLP guy), but I do think there's a huge space to create publicly available text mining which can more effectively be applied to narrow domains.