Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ANTLR 4.6 Released

antlr.org

Post reply on HN
Print view View original

21–28 of 28 posts

Re: ANTLR 4.6 Released

#21

Post by verdex_blarg »

chrisseaton wrote: post #16
chubot wrote: post #5

I'd be interested see pointers to ANTLR-based languages people are developing. I found it to be a powerful tool, but also limiting. As far as I know, ANTLR v4 is even more of a "framework" than v3, in that it dictates your program structure and always outputs a full parse tree. It appears to have lost the ability to verify the value of k for LL(k) grammars, due to a more powerful underlying algorithm. I think that si…

I work on implementing languages and I often need new parsers. Either for full languages or little embedded languages or expression syntaxes of various kinds. I often think 'right, this time I'm going to write a really nice little Antlr grammar and do it properly'. But every single time I regret it because there's always some complexity that means things don't fit into the Antlr model, and making things work the way…

This is exactly the sort of thing I was looking for. I keep on thinking that my life would be easier if I just adopted something like ANTLR, but every time I look into it it looks like it's more work than it's worth.

Good to hear that this has been a similar experience to people who have actually gone and actually used ANTLR.

Re: ANTLR 4.6 Released

#22

Post by doublerebel »

chubot wrote: post #5

I'd be interested see pointers to ANTLR-based languages people are developing. I found it to be a powerful tool, but also limiting. As far as I know, ANTLR v4 is even more of a "framework" than v3, in that it dictates your program structure and always outputs a full parse tree. It appears to have lost the ability to verify the value of k for LL(k) grammars, due to a more powerful underlying algorithm. I think that si…

The query language for Riemann uses a parser made with ANTLR. ANTLR has many compile targets, so it was reasonably easy to port the parser from Clojure to Nodejs.

https://github.com/riemann/riemann/blob/master/resources/que...

https://github.com/nextorigin/riemann-query-parser/blob/mast...

I'm using it with Godot2, a Riemann-like streaming monitoring system in Nodejs.

https://github.com/nextorigin/godot2-dash

Re: ANTLR 4.6 Released

#23

Post by chubot »

chrisseaton wrote: post #16
chubot wrote: post #5

I'd be interested see pointers to ANTLR-based languages people are developing. I found it to be a powerful tool, but also limiting. As far as I know, ANTLR v4 is even more of a "framework" than v3, in that it dictates your program structure and always outputs a full parse tree. It appears to have lost the ability to verify the value of k for LL(k) grammars, due to a more powerful underlying algorithm. I think that si…

I work on implementing languages and I often need new parsers. Either for full languages or little embedded languages or expression syntaxes of various kinds. I often think 'right, this time I'm going to write a really nice little Antlr grammar and do it properly'. But every single time I regret it because there's always some complexity that means things don't fit into the Antlr model, and making things work the way…

I have this "meta-language" rant brewing for my blog[1], and I feel like you took the words out of my mouth!

Are you saying that ANTLR makes some unwarranted assumptions about Unicode? Does that just depend on the generated Java code, and would it be different for generated parsers in C++? I don't know the JVM very well, but my understanding was that the JVM makes some Unicode assumptions that aren't always appropriate.

The last "rant" was here, about meta-languages only being suitable for toys: https://news.ycombinator.com/item?id=13040682

vidarh also responds with his problems parsing Ruby (a "real" language): http://hokstad.com/compiler

The blog is based on my experience in writing a parser for bash. I ported the POSIX shell grammar to ANTLR, but it fell ridiculously short of being a production quality parser.

I believe it's essentially impossible to write a bash parser with ANTLR. But I don't hear about any alternatives. All I heard is yacc/bison for bottom up parsers, and ANTLR for top-down. Shell needs a top-down parser because it's an interactive language (the PS2 prompt) and for completion. But it doesn't appear there is any other "serious" meta-language for top-down parsers other than ANTLR? It certainly is the best documented and longest-lived, but I am surprised how far short it falls for many tasks.

Part of the rant will be a survey of "real" language parsers (i.e. take the top 20 TIOBE language implementations) and see that they almost all use hand-written parsers, or bespoke parser generators. For example, Python has its own top-down parser generator in the tree, pgen.c.

[1] http://www.oilshell.org/blog/2016/11/20.html

Re: ANTLR 4.6 Released

#24

Post by chrisseaton »

chubot wrote: post #23
chrisseaton wrote: post #16

Earlier quoted context omitted.

I work on implementing languages and I often need new parsers. Either for full languages or little embedded languages or expression syntaxes of various kinds. I often think 'right, this time I'm going to write a really nice little Antlr grammar and do it properly'. But every single time I regret it because there's always some complexity that means things don't fit into the Antlr model, and making things work the way…

I have this "meta-language" rant brewing for my blog[1], and I feel like you took the words out of my mouth! Are you saying that ANTLR makes some unwarranted assumptions about Unicode? Does that just depend on the generated Java code, and would it be different for generated parsers in C++? I don't know the JVM very well, but my understanding was that the JVM makes some Unicode assumptions that aren't always appropria…

To be fair to ANTLR I think it might have actually wanted a CharSequence, which is a bit more pragmatic, and I could probably have wrapped my byte[] in an object implementing that interface.

The real problem I should mentioned was that I seemed to be using very little of the real ANTLR parsing algorithm, as there was so much logic in actions to parse a real language like Ruby. These languages aren't designed with a formal grammar model in mind. They just do what they do and it's your problem if ANTLR isn't designed for what they want.

Actions are the grammar equivalent of unsafe blocks in languages like Rust or Haskell. If you are using them in an application it's a red flag, and I never managed to write an ANTLR grammar without them. That then makes the red flag on ANTLR, not on my programming skills.

Re: ANTLR 4.6 Released

#25

Post by [deleted] »

chubot wrote: post #23
chrisseaton wrote: post #16

Earlier quoted context omitted.

I work on implementing languages and I often need new parsers. Either for full languages or little embedded languages or expression syntaxes of various kinds. I often think 'right, this time I'm going to write a really nice little Antlr grammar and do it properly'. But every single time I regret it because there's always some complexity that means things don't fit into the Antlr model, and making things work the way…

I have this "meta-language" rant brewing for my blog[1], and I feel like you took the words out of my mouth! Are you saying that ANTLR makes some unwarranted assumptions about Unicode? Does that just depend on the generated Java code, and would it be different for generated parsers in C++? I don't know the JVM very well, but my understanding was that the JVM makes some Unicode assumptions that aren't always appropria…

[deleted]

Re: ANTLR 4.6 Released

#26

Post by raiph »

chubot wrote: post #23
chrisseaton wrote: post #16

Earlier quoted context omitted.

I work on implementing languages and I often need new parsers. Either for full languages or little embedded languages or expression syntaxes of various kinds. I often think 'right, this time I'm going to write a really nice little Antlr grammar and do it properly'. But every single time I regret it because there's always some complexity that means things don't fit into the Antlr model, and making things work the way…

I have this "meta-language" rant brewing for my blog[1], and I feel like you took the words out of my mouth! Are you saying that ANTLR makes some unwarranted assumptions about Unicode? Does that just depend on the generated Java code, and would it be different for generated parsers in C++? I don't know the JVM very well, but my understanding was that the JVM makes some Unicode assumptions that aren't always appropria…

> it doesn't appear there is any other "serious" meta-language for top-down parsers other than ANTLR?

Have you explored Perl 6 and its Rules sub-language and engine?[1] The Rakudo Perl 6 compiler is written in Perl 6 using Rules.[2]

[1] https://en.wikipedia.org/wiki/Perl_6_rules

[2] https://github.com/rakudo/rakudo/blob/nom/src/Perl6/Grammar.....

Re: ANTLR 4.6 Released

#27

Post by chubot »

raiph wrote: post #26
chubot wrote: post #23

Earlier quoted context omitted.

I have this "meta-language" rant brewing for my blog[1], and I feel like you took the words out of my mouth! Are you saying that ANTLR makes some unwarranted assumptions about Unicode? Does that just depend on the generated Java code, and would it be different for generated parsers in C++? I don't know the JVM very well, but my understanding was that the JVM makes some Unicode assumptions that aren't always appropria…

> it doesn't appear there is any other "serious" meta-language for top-down parsers other than ANTLR? Have you explored Perl 6 and its Rules sub-language and engine?[1] The Rakudo Perl 6 compiler is written in Perl 6 using Rules.[2] [1] https://en.wikipedia.org/wiki/Perl_6_rules [2] https://github.com/rakudo/rakudo/blob/nom/src/Perl6/Grammar.... .

I have heard of this feature of Perl 6, but not used it. One obvious difference is that ANTLR is language-agnostic in that you can generate parsers in C++ or Python too, while I assume Perl 6 doesn't have that functionality. So that limits its appeal.

(Although honestly ANTLR is pretty skewed towards Java; the generated code is kind of like Java-in-Python or Java-in-C++.)

I'm curious what algorithm Perl uses to match grammars. ANTLR uses a few algorithms, like the one to generate lookahead tables for LL(k), and then the LL() algorithm, and apparently a new one with ALL() in ANTLR v4.

Re: ANTLR 4.6 Released

#28

Post by vbezhenar »

verdex_blarg wrote: post #21
chrisseaton wrote: post #16

Earlier quoted context omitted.

I work on implementing languages and I often need new parsers. Either for full languages or little embedded languages or expression syntaxes of various kinds. I often think 'right, this time I'm going to write a really nice little Antlr grammar and do it properly'. But every single time I regret it because there's always some complexity that means things don't fit into the Antlr model, and making things work the way…

This is exactly the sort of thing I was looking for. I keep on thinking that my life would be easier if I just adopted something like ANTLR, but every time I look into it it looks like it's more work than it's worth. Good to hear that this has been a similar experience to people who have actually gone and actually used ANTLR.

I needed to parse a tiny subset of SQL. I did it with ANTLR very quickly. It would take much more time doing it by hand, and I know quite a bit about parsing. For complex language it might be the other way, of course. Actually I never saw a single real programming language without hand-crafted parser. But if you need a tiny DSL, ANTLR is fine.
Post reply on HN