Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why you should not use (f)lex, yacc and bison

tomassetti.me

Post reply on HN
Print view View original

31–40 of 59 posts

Re: Why you should not use (f)lex, yacc and bison

#31

Post by mateo411 »

1vuio0pswjnm7 wrote: post #24
stephc_int13 wrote: post #6

I learned about the existence of Lex, Yacc, and Bison a bit more than 20 years while discussing the difficulties of debugging handwritten parsers with a friend. But I was quickly discouraged by the complexity of using these tools, they are more like a new language that you have to learn. Today I prefer writing parsers by hand, in C, like Fabrice Bellard, it is not super easy but manageable and I never encountered maj…

How easy is it to make quick changes to hand-written parsers. Imagine, for example, that the input being parsed is not under the authors control and it suddenly changes. The parser must be changed.

The input changing shouldn't matter. Maybe you could see a performance issue if you had to parse a really large file/files. The issue would be if the grammar to what you are parsing changes. If that changes, then no parser will work, correctly until it supports the new grammar.

As a practical matter the grammar can act like a contract between the two systems. You don't have control of the input, but the input must follow the same grammar.

Re: Why you should not use (f)lex, yacc and bison

#32

Post by professoretc »

AshamedCaptain wrote: post #7

Glad to hear the ANTLR vs Yacc holy way (a.k.a. LL vs LR) is still going strong. This one holy war may actually be older than emacs vs vi. Have the ANTLR guys dropped their Java credo or they're still thinking they are going to convince any C developer with a .jar?

I wrote a parser for a DSL using ANTLR about ten-ish years ago, and it was an incredibly painful experience. There's very little freely-available documentation for ANTLR, most of it is just "buy my book". ANTLR doesn't know anything about the language it's targeting (not even Java), so if you name a rule e.g., `while` the generated code will be filled with errors, because it names the parser functions after the rules. The graphical tools for building parsers are cool, except that many advanced things can only be done by adding actual code to the parser rules, which the graphical tool does not support (and how could it?) so you're back at square one.

Re: Why you should not use (f)lex, yacc and bison

#33

Post by robocat »

WesolyKubeczek wrote: post #19

> flex uses a BSD license, while Bison uses the GPL. Bison waive the license for most generated parsers, but it can be a problem Now, this is one of the most atrocious handwavy kind of FUD that I've had a displeasure to see. If the parsers generated by bison are GPL-free, where is the problem? Can you even name the problem? Or you just see the word "GPL" and it's scaring you somehow?

I am guessing that the author is referencing organisations that are pathologically allergic to GPL, where the legal department might not accept the exception.

https://www.google.com/search?q=bison+gpl+exception

Re: Why you should not use (f)lex, yacc and bison

#34

Post by UncleEntity »

I've found re2c + lemon to be a pretty good combination.

re2c has good unicode support -- takes a while to compile if attempting TR31 "Unicode Identifier And Pattern Syntax" though.

Haven't used lemon on anything too complicated, I basically just play with grammars for fun, but it works really well for my MiniScheme fork I like to poke at every once in a while.

My new shiny is Coco/R which is a dream to use (once you get up the learning curve) but hasn't really been maintained for ages.

Re: Why you should not use (f)lex, yacc and bison

#35

Post by WesolyKubeczek »

robocat wrote: post #33
WesolyKubeczek wrote: post #19

> flex uses a BSD license, while Bison uses the GPL. Bison waive the license for most generated parsers, but it can be a problem Now, this is one of the most atrocious handwavy kind of FUD that I've had a displeasure to see. If the parsers generated by bison are GPL-free, where is the problem? Can you even name the problem? Or you just see the word "GPL" and it's scaring you somehow?

I am guessing that the author is referencing organisations that are pathologically allergic to GPL, where the legal department might not accept the exception. https://www.google.com/search?q=bison+gpl+exception

The problem is that you have to guess what the problem is, exactly.

The author might just as well put up a comparison chart:

    bison: PROBLEMATIC AND GPL
    ANTLR: does not have a problem*
    ----
    * With yearly consulting contract from our company
The message would be exactly the same.

Re: Why you should not use (f)lex, yacc and bison

#36

Post by 1vuio0pswjnm7 »

mateo411 wrote: post #31
1vuio0pswjnm7 wrote: post #24

Earlier quoted context omitted.

How easy is it to make quick changes to hand-written parsers. Imagine, for example, that the input being parsed is not under the authors control and it suddenly changes. The parser must be changed.

The input changing shouldn't matter. Maybe you could see a performance issue if you had to parse a really large file/files. The issue would be if the grammar to what you are parsing changes. If that changes, then no parser will work, correctly until it supports the new grammar. As a practical matter the grammar can act like a contract between the two systems. You don't have control of the input, but the input must fo…

Does the separation of the grammar into its own file, like a .y file for yacc, make it easier to modify the grammar at a later point. Similarly, is it easier to modify a parser later on, if the input changes, if the patterns to be searched in the input and the state machine are stored in a separate file, like an .l file for flex.

Re: Why you should not use (f)lex, yacc and bison

#38

Post by db48x »

eqvinox wrote: post #29
dmitrygr wrote: post #27

Earlier quoted context omitted.

flex & bison use globals to report things per-rule. Of course they are not re-entrant. This is documented by virtue that those are globals.

And the fix is to not use globals. cf. my toplevel post about "%option reentrant", "%lex-param" and "%parse-param". Globals are just the 80's setup, and flex/bison are supposed to be POSIX lex/yacc compatible, so I'm gonna count this as "shitty defaults due to compatibility with stone tablets".

Correct. It’s the default but it’s no longer necessary.

Flex has had some work done to push it towards being able to generate Go and Rust code too, though this work isn’t finished. In principle it is now possible to generate any Algol–style language.

Re: Why you should not use (f)lex, yacc and bison

#39

Post by karmakaze »

ksaj wrote: post #18
AshamedCaptain wrote: post #7

Glad to hear the ANTLR vs Yacc holy way (a.k.a. LL vs LR) is still going strong. This one holy war may actually be older than emacs vs vi. Have the ANTLR guys dropped their Java credo or they're still thinking they are going to convince any C developer with a .jar?

This is actually the first time I've even heard of ANTLR, which came about in the early 90's. But yacc/lex/vi/emacs, et al have been around since the 70's. The emacs vs. vi holy war was pretty much instant, but doomed every time one spoke of headless devices such as routers.

Antlr and LL(k) parsers in general are great when they fit the grammar you want to make well. I remember discovering it when it was called JavaCC a nod to yacc.
Post reply on HN