Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nitra, JetBrains’ research project for language tooling, goes open-source

blog.jetbrains.com

Post reply on HN
Print view View original

31–40 of 42 posts

Re: Nitra, JetBrains’ research project for language tooling, goes open-source

#31

Post by citizenmatt »

mythz wrote: post #29
citizenmatt wrote: post #10

Earlier quoted context omitted.

We do have a lot of future plans for Nitra, but for now, it's worth pointing out that the products that are cross platform are Java based, while Nitra is based on .net. The project is currently Windows only.

Interesting, what made you decide on .NET (i.e. given you're predominantly JVM based)? Was it just based on the Nermerle's team preference?

I wouldn't say we're predominantly JVM based. We've also got ReSharper, dotTrace, dotMemory, dotCover and dotPeek, which are all .net based.

But basically, it's because Nitra is an extension and evolution of work done on Nemerle, and Nemerle is a .net language.

Re: Nitra, JetBrains’ research project for language tooling, goes open-source

#32

Post by drdaeman »

Just curious, how do they handle broken code? (Like when you start writing a line in the middle of the file, not yet done with it, but already need all the goodness like highlighting and code completion to work.)

A common approach with libraries I've encountered is that parser just stops with error - but that's almost unacceptable for use in a proper code editor, which should really try its best to recover and continue processing, even if some chunk in the middle is failing.

Re: Nitra, JetBrains’ research project for language tooling, goes open-source

#33

Post by quotemstr »

sparkie wrote: post #25
Igglyboo wrote: post #21

Having a minimal idea of how these work, how similar is this to antlr? "It is also a build tool to compile the grammars into parsers" this line specifically caught my attention.

The parsing part is similar conceptually and syntactically, but their implementation is very different. Antlr parses LL grammars - an unambiguous subset of context-free-grammars which are quite restrictive in the production rules they allow. This tool on the other hand uses PEGs, which parse a different (but overlapping) set of grammars, which aren't necessarily limited to CFGs, but are always guaranteed to be unambi…

Note that PEGs are not context-free grammars. They're both more and less powerful than traditional CFGs, and they're tricky to use: because PEG choice is ordered and traditional CFG choice is unordered, it's hard to translate standard language grammars to a PEG recognizer system. That's why, for my forever-project, I've oped to use scannerless GLR instead of PEGs. Both PEGs and GLR recognize languages that are closed under composition (the property that gives you extensibility), but the formalisms for GLR parsers are much better.

The Harmonia project is the best whack at the problem I've seen. See http://harmonia.cs.berkeley.edu/papers/twagner-parsing.pdf.

As others have mentioned, for an IDE, you also want strong error recovery. Doing that in a general way when using tools based on declarative grammars is, well, very hard, especially when you want to recover from brace mismatch problems. The best approach is "island and reef parsing", where you actually parse your buffer twice: you first build a map of all the "reefs" (parenthesis) using a simple recursive descent parser, pair up mismatched parenthesis using an ad-hoc algorithm, insert corrections for mismatches, then apply your fully general parser to the result. (The word "parenthesis" here refers to any balanced construct, even "begin" and "end". You can actually infer what the "parenthesis" for a given language are by examining the grammar!)

See also http://fileadmin.cs.lth.se/cs/Personal/Emma_Soderberg/docs/S...

Re: Nitra, JetBrains’ research project for language tooling, goes open-source

#34

Post by mythz »

citizenmatt wrote: post #31
mythz wrote: post #29

Earlier quoted context omitted.

Interesting, what made you decide on .NET (i.e. given you're predominantly JVM based)? Was it just based on the Nermerle's team preference?

I wouldn't say we're predominantly JVM based. We've also got ReSharper, dotTrace, dotMemory, dotCover and dotPeek, which are all .net based. But basically, it's because Nitra is an extension and evolution of work done on Nemerle, and Nemerle is a .net language.

Is ReSharper written with C#? I thought I heard it was written with C++.

Re: Nitra, JetBrains’ research project for language tooling, goes open-source

#35

Post by sitkack »

bodski wrote: post #11

This looks very much in the spirit of Yegges' 'Grok' project at Google [1][2]. Does anyone know if that project is still alive? [1] http://bsumm.net/2012/08/11/steve-yegge-and-grok.html [2] https://www.youtube.com/watch?v=KTJs-0EInW8

You might be interested in https://github.com/yinwang0/pysonar2

Re: Nitra, JetBrains’ research project for language tooling, goes open-source

#36

Post by citizenmatt »

mythz wrote: post #34
citizenmatt wrote: post #31

Earlier quoted context omitted.

I wouldn't say we're predominantly JVM based. We've also got ReSharper, dotTrace, dotMemory, dotCover and dotPeek, which are all .net based. But basically, it's because Nitra is an extension and evolution of work done on Nemerle, and Nemerle is a .net language.

Is ReSharper written with C#? I thought I heard it was written with C++.

It's written in C#, with some of the Visual Basic support written in VB. Parts of ReSharper for C++ are written in C++ and C++/CLI.

Re: Nitra, JetBrains’ research project for language tooling, goes open-source

#37

Post by mwsherman »

drdaeman wrote: post #32

Just curious, how do they handle broken code? (Like when you start writing a line in the middle of the file, not yet done with it, but already need all the goodness like highlighting and code completion to work.) A common approach with libraries I've encountered is that parser just stops with error - but that's almost unacceptable for use in a proper code editor, which should really try its best to recover and contin…

That part of Visual Studio impresses me a lot, though it’s not obvious to a user. I haven’t (say) closed the brace yet, so the code is invalid and therefore a correct AST can’t be parsed.

It has to be heuristic? Or a given (say) line falls back to last known good state?

Re: Nitra, JetBrains’ research project for language tooling, goes open-source

#39

Post by lobster_johnson »

mwsherman wrote: post #37
drdaeman wrote: post #32

Just curious, how do they handle broken code? (Like when you start writing a line in the middle of the file, not yet done with it, but already need all the goodness like highlighting and code completion to work.) A common approach with libraries I've encountered is that parser just stops with error - but that's almost unacceptable for use in a proper code editor, which should really try its best to recover and contin…

That part of Visual Studio impresses me a lot, though it’s not obvious to a user. I haven’t (say) closed the brace yet, so the code is invalid and therefore a correct AST can’t be parsed. It has to be heuristic? Or a given (say) line falls back to last known good state?

I suspect the latter, helped by an ability to recover by inserting missing characters such as braces and quotes. (Xcode actually offers to fix trivial errors such as those, I'm sure VS does the same.)

It's an interesting problem. I suppose that as it knows the point of breakage, it can annotate the AST to indicate breakage, but preserve the subsequent node; breakage itself becomes a kind of AST node. It's possible that in such a situation, any subsequent AST nodes probably have to point to their pre-breakage nodes as parents in order to stay sane. Thus the AST tree becomes a kind of Git-like revision history that stays fragmented until the next time the AST fully parses. It could easily be something even simpler, however.

Re: Nitra, JetBrains’ research project for language tooling, goes open-source

#40

Post by edwinyzh »

mwsherman wrote: post #37
drdaeman wrote: post #32

Just curious, how do they handle broken code? (Like when you start writing a line in the middle of the file, not yet done with it, but already need all the goodness like highlighting and code completion to work.) A common approach with libraries I've encountered is that parser just stops with error - but that's almost unacceptable for use in a proper code editor, which should really try its best to recover and contin…

That part of Visual Studio impresses me a lot, though it’s not obvious to a user. I haven’t (say) closed the brace yet, so the code is invalid and therefore a correct AST can’t be parsed. It has to be heuristic? Or a given (say) line falls back to last known good state?

In my case, the later, I use a very tolerated css and html tokenizer/parser to provide basic html code-completion in LIVEditor - the real-time css and html tweaker/editor (http://liveditor.com)
Post reply on HN