Live data from Hacker News

Proselint

proselint.com

61–70 of 141 posts

Re: Proselint

#61
post #39

Earlier quoted context omitted.

A linter doesn't prevent breaking its rules, it just notifies the writer of which rules are being broken. I was writing some C earlier and my linter warned me about "incrementing a void pointer". However, I understood the context better than my linter, knew that I'd be compiling with gcc (which allows void pointer arithmetic), so I ignored the warning and carried on. My code compiled and ran nicely. When it comes to…

Sounds like your system is in US English rather than the variant of English you are used to? http://grammarist.com/spelling/color-colour/

Sorry, I guess that example was too terse.

I was referring to a hypothetical American creative writer, writing a scene in which a British character writes a letter. In this hypothetical work, written in US English, there would then be a section of text that used UK English spellings. The naive spell-checker would not understand the context, and would flag these as misspellings.

This was meant to be analogous to my "incrementing a void pointer" example; the static analysis tool produces warnings which the author knows to ignore. In the C programming case, my function was passed the size of the objects comprising the array pointed to by the void pointer, so the linter was wrong to tell me I was making a mistake. Similarly, the spell-checker was wrong to say "change this instance of 'colour' to 'color'".

Similar considerations apply to prose linters.

Polonius would be a lesser character if shed of cliches, and a good writer would know to ignore the linter's opinions on the matter.

Re: Proselint

#62
post #60
post #58

What kinds of NLP technique does this system use? Is it possible to specify new rules in a high-level way? Can it learn from examples? Does it work on a sentence-by-sentence basis only, or does it "grasp" complete paragraphs?

Rules are defined in Python scripts which can have arbitrary complexity. However, it seems like most rules are just string or regex matching: https://github.com/amperser/proselint/blob/master/proselint/... https://github.com/amperser/proselint/blob/master/proselint/...

[deleted]

Re: Proselint

#63
post #40
post #34

This sounds interesting. As a bit of constructive criticism, please put some examples high up. You tell me it does cool things. Great, show me. I've looked about on the various pages and can see only one example and I don't understand it: text.md:0:10: wallace.uncomparables Comparison of an uncomparable: 'unique' can not be compared. What's the context of this, what's the error it would have caught in my writing? The…

This would catch something like "even more unique". In fact, looking at the code ( https://github.com/amperser/proselint/blob/master/proselint/... ) it would even catch something like "extremely unique", which I've been guilty of using. But yes, there should be examples on the front page.

So this program is like having some insufferable pedants arguing over your language? Great!

Does it accept 'nearly unique' ?

Re: Proselint

#64
post #52
post #14

I can see a lot of value for this sort of tool, and might even play with it myself, for sake of evaluating whether or not to incorporate its suggestions into my writing. At the same time, however, I have some wariness that its widespread use could actually have a shaping, and, specifically homogenizing , effect on language. For me, a large part of the beauty of language is how facile it is, how judiciously breaking i…

I agree! But still, it corrects incorrect things that my spell checker doesn't see, like inconsistent spacing and 'goofy approximations' like (R) for ®. (Depends on your definition of incorrect, but I personally would not mind at all if these things were homogenized for everyone, it would not take any richness out of the English language). What I'd like (--help doesn't list such an option) would be to be able to enab…

That's cool but it sounds like this tool is way oversold. It namedrops DFW and other great authors then shows examples of it correcting spacing and "brb." This isn't stylistic revising that takes you closer to those writers, it's just simple corrections.

Re: Proselint

#65
post #57

Earlier quoted context omitted.

Some people feel you should never ever say things like "more unique", "most unique" etc Which I think is equally as misguided as trying to force "data" to be plural, and that "less than 3" is wrong

In a mathematical context, something is either "unique" or it is not. There is no in-between state. But you can easily define it to mean something else. And you can even make "uniqueness" comparable.

It's a good thing then that most people don't use it in its technical sense

An easy fix for this not problem is to use distinct instead of unique

Re: Proselint

#66
post #58

What kinds of NLP technique does this system use? Is it possible to specify new rules in a high-level way? Can it learn from examples? Does it work on a sentence-by-sentence basis only, or does it "grasp" complete paragraphs?

> What kinds of NLP technique does this system use?

It depends on your interpretation of NLP. In a sense, all of the rules are hard coded, and so it does string token processing that happens to be informed by contributed interpretations of style guides' rules for usage. Thus, most of the NLP has been performed by the human programmers interpreting those rules.

Though we are interested in extensions in the direction of robust machine NLP approaches able to meet the other goals of proselint, that presents many challenges (including some I mention in response to your third question). Nonetheless, this is an active area of research.

> Is it possible to specify new rules in a high-level way?

In short, no, but it is an area of active research on our part to develop a rule-templating engine for exactly this purpose. "High-level" is subjective though, so there may always be someone who intends to ask about a level higher than the interface that we provide at the time that this question is asked.

> Can it learn from examples?

In a sense, yes, all of the rules have been learned by people from the example text in guides and translated to linting rules. But I do not think that was your intended question.

If instead you mean: you would provide it a set of examples of your writing and it would induce a rule, no it does not do that currently, and may not for quite some time.

Stylistic rule induction is a difficult – though interesting – problem (as is rule induction more generally). It is not something we are intrinsically opposed to, but the simplest version of learning from examples would violate two core principles of the design of proselint.

First, our rules are taken from and organised around the advice provided by respected authors in their writing on linguistic style.

Second, any inductive method will be intrinsically uncertain about the rules that it induces. This uncertainty will always be opposed to our aim of having a low false alarm rate, making inductive methods possible but subject to extensive tuning and testing. This suggests that further development of a test set outside of the examples provided would be needed, to ensure coverage of any of the rules that the examples would suggest inducing.

Additionally, almost all state-of-the-art machine learning systems would require a set of relevant labeled examples of usage errors and non-errors that would somehow generalise to the examples that you would like to provide it. Even specifying the data format would be difficult; if you have any insights as to how this would be done, please develop them below, it can only be helpful and aid progress in this direction.

> Does it work on a sentence-by-sentence basis only, or does it "grasp" complete paragraphs?

I think the easiest way for you to answer this question is for you to see it in action at this website: http://proselint.com/write/

I should mention that longer range dependencies require greater computational power which brushes up against another aim of proselint, to be fast enough to run on reasonably large files as a real-time linter. This may not always be the case in all instantiations of proselint, but for now this is true.

If you have paragraph level rules that you might want to suggest (like the issue I just created when writing this response: https://github.com/amperser/proselint/issues/310), please do! It is even more helpful if you can find an authoritative reference to include as part of your issue, because that will be needed to incorporate the rule into proselint.

Re: Proselint

#67
Probably a stupid nitpick, but this bothers me:

> detecting grammatical errors is AI-complete, requiring human-level intelligence to get things right.

(emphasis mine)

First, there's a problem of usage. When in CS we say that a problem is class-complete (like NP-complete), we mean that the problem belongs to the class (which in this case is true, because human-level intelligence can check grammar), but also that it is class-hard, which informally means "at least as hard as the hardest problems in class", and more formally means that any other problem in class can be cheaply reduced to the problem, and so finding a suitable solution to the problem is identical to finding a suitable solution to all other problems in class. Not only checking grammar not known to be "AI-complete" then, we don't even know that human-level intelligence is necessary to solve it.

But the reason this bothers me even though I fully understand the statement was made informally, is a little deeper than that: we don't even know what "human-level intelligence" (or intelligence in general) is, let alone what AI means. That people refer to AI as if it's a thing rather than a very vague notion, clouds how people think of AI research as well as intelligence. I would have simply said "we don't know of good algorithms to dependably check grammar, and this appears to be a very hard problem that may require intelligence".

Re: Proselint

#69
This is very cool and needed, thank you.

Could you include a sample .proselintrc? rc files tend to have very different opinions on how to be formatted: dictionaries, JSON, bash-argument syntax, and so on. (EDIT: Ah, found one: https://github.com/amperser/proselint/blob/cd428bb0ecc5530c1.... Can’t quite get it to ignore butterick, though.)

I find it a little curious that you use a Markdown example and lint for curly quotes and unicode ellipses by default (butterick), since Markdown discourages such pre-formatting in its syntax, but that’s just hairsplitting, of which I can tell by your swelling Issues count that you have plenty of as it is. :)

Looking forward to some formatting/syntax highlighting in the CLI output, but I know you have your hands full as it is.

Post reply on HN