Ask HN: Natural language processing to identify grammar in a text?

As someone else alluded to, this is a task for multiple models. Fortunately, there are a lot of great NLP libraries that combine multiple pre-trained language models into a single pipeline you can interface with, like Stanza. From their docs, their vanilla pipeline breaks down the sentence "Barack Obama was born in Hawaii. He was elected president in 2008." as :

('Barack', '4', 'nsubj:pass')

('Obama', '1', 'flat')

('was', '4', 'aux:pass')

('born', '0', 'root')

('in', '6', 'case')

('Hawaii', '4', 'obl')

('.', '4', 'punct')

It should be very easy to deploy Stanza's pipeline as an API endpoint. Here is an example of such a NLP-library-as-API endpoint, albeit with Hugging Face's Transformers, deployed via Cortex: https://github.com/cortexlabs/cortex/blob/master/examples/py...

Example of passive voice detection using spacy[1]: https://gist.github.com/armsp/30c2c1e19a0f1660944303cf079f83...

[1] https://spacy.io/

A language model is a model that predicts the probability of a given text, that is all. It should not be conflated with other types of NLP tasks like part of speech tagging. I'm guessing the popularity of transformer based models, which are built around the LM task and then adapted to other donations, is leading to this confusion.

I actually (sort of) wrote one of these a while back (though I don't think I ever got to implementing tenses - possibly this would be somewhat easy to implement on top of whatever I already built there, but maybe not idk). In any case:

> copulae verbs, linking verbs, terms that are often filtered (i.e. stop terms), question terms, time sensitive nouns, amplifiers, clauses, coordinating conjunctions, negations, conditionals (ORs), and contractions

https://github.com/nyxtom/salient/

Some tough love:

There will always be a gap between "your judgement" and the "judgement baked into a model" -- worse yet, if the model is very general and oriented towards cheap computation and away from expensive people it will have vague and contradictory judgements inside it that make the results meaningless.

That is the language of failure: the structure of success looks like the following.

(1) The system works like a "magic magic marker", that is, you mark up a lot of text (say 20,000 sentences) the way you think it should be marked up. This might be a character-at-a-time or word-at-a-time. Character-at-a-time is real and eternal, word-at-a-time is not real because there is not really such a thing as a "word". (e.g. "red ball" can fill slots that take "ball", you can smash together subwords to make words, for that matter people violate punctuation rules "Amazon.com announced that...", people call themselves n3pg34r, ...) So if you segment the text up front and segment it the wrong way you may throw out essential information and choose to fail.

(2) You need some system to mark up the text manually and efficiently. It is a lot of work. A typical person can make about 2000 or so up/down judgements a day; if a sentence counts for 10 decisions then maybe you can annotate 200 sentences a day. If you can get students to do it and get teachers to review it you might make short work of it.

This annotator

http://brat.nlplab.org/

ticks the requirements, but most people find it terribly hard to use and wind up building "easy to use" systems that don't align things right at level (1) and... fail.

Assuming you do (1) and (2) the odds are in your favor, but you have to now

(3) build models; it does not matter if the model is a bunch of rules you cobbled together, or hidden markov, or LSTM, or convolutional. Off the top I would train an LSTM to 'predict the next character' on maybe 100M characters of text, then I would stick a simple model that takes the LSTM state as an input and labels characters at the output (could be SVM, Random Forest, Logit, or 3 layer on NN)

(4) Accept that the system is not going to be perfect, but have the ability to manually patch wrong results, improve the training data over time. I'd say this is a more important practice than any particular approach to (3)

Some tool could give you (1-4) tied up in a bow

https://www.tagtog.net/

claims to. But (2) involves elbow grease that 90% of people aren't going to do. Some of the 10% of people who do that elbow grease will succeed, the other 90% will fail.