Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Pest – Fast parser generator written in Rust

github.com

Post reply on HN
Print view View original

11–20 of 28 posts

Re: Show HN: Pest – Fast parser generator written in Rust

#11

Post by yellowapple »

jmcomets wrote: post #9
DC-3 wrote: post #8

What type of parser does this produce?

"Parsing expression grammars (PEGs) are an alternative to context free grammars for formally specifying syntax" [1] Basically, you have tools for context free grammars such as GNU Bison or Yacc. This is an alternative, but with PEG. For more in-depth info, there's the original PEG paper. [2] [1]: http://bford.info/packrat [2]: http://bford.info/pub/lang/peg

That's the grammar, not the parser resulting from the grammar.

The README doesn't say, but most PEGs in the wild (AFAICT) compile down to / are implemented by packrat parsers.

Re: Show HN: Pest – Fast parser generator written in Rust

#12

Post by yellowapple »

OtterCoder wrote: post #4
runevault wrote: post #3

Earlier quoted context omitted.

Rust by default uses Unicode for strings. I would be surprised if this had problems with them.

The difficulty wasn't in the engine. JS itself is just as good with accepting Unicode characters as Rust, both in code and strings. The problem was with the PEG implementation. Without some sort of shorthand for Unicode families, defining programming languages that are valid in multiple real-world languages becomes a serious burden.

This is one of the things Perl 6 does really well,).

Re: Show HN: Pest – Fast parser generator written in Rust

#13

Post by dikaiosune »

reificator wrote: post #7
hobofan wrote: post #5

Shouldn't the title better by "Pest - Fast, modern parser _in_ Rust"? The current title made me think that this would be a Rust AST parser, similar to syntex.

Exactly what I came into this thread expecting. Was curious why when it looks like there's already great IDE support for Rust with existing tools.

There's actually a parser for rust called syn, and it's very useful for procedural macros, as the current API exposes a stream of tokens, not a full AST.

Re: Show HN: Pest – Fast parser generator written in Rust

#14

Post by dragostis »

DC-3 wrote: post #8

What type of parser does this produce?

For now, it produces a recursive descent parser. Packrat parsing is an open-question since I'm afraid that adding a memoization layer all throughout the parser will lead to a consistent general slowdown.

Re: Show HN: Pest – Fast parser generator written in Rust

#15

Post by DC-3 »

yellowapple wrote: post #10
DC-3 wrote: post #8

What type of parser does this produce?

With PEGs (like Pest), it's probably going to be a packrat parser (which would also make sense given the project's name).

Thank you for the actual answer. Funny how some people are so excited to give an RTFM answer that they forget to read the question.

E: turns out this is wrong anyway.

Re: Show HN: Pest – Fast parser generator written in Rust

#16

Post by DC-3 »

dragostis wrote: post #14
DC-3 wrote: post #8

What type of parser does this produce?

For now, it produces a recursive descent parser. Packrat parsing is an open-question since I'm afraid that adding a memoization layer all throughout the parser will lead to a consistent general slowdown.

Interesting. I'll try and find some time to read the source - am interested to see how this is implemented.

Re: Show HN: Pest – Fast parser generator written in Rust

#17

Post by weberc2 »

This is a neat project! I might take a look at this if/when I get started on my dream language. :p

One constructive criticism, which may be only an unpopular opinion, but I find creative operator overloading unnecessarily hard to read. And I don't think my opinion is entirely subjective since, by definition, you have to learn new semantics whereas a good function name would make the meaning obvious. Maybe this syntax would be familiar to people who are already familiar with formal grammar notations?

Re: Show HN: Pest – Fast parser generator written in Rust

#18

Post by steveklabnik »

dikaiosune wrote: post #13
reificator wrote: post #7

Earlier quoted context omitted.

Exactly what I came into this thread expecting. Was curious why when it looks like there's already great IDE support for Rust with existing tools.

There's actually a parser for rust called syn, and it's very useful for procedural macros, as the current API exposes a stream of tokens, not a full AST.

To elaborate a bit on that, exposing an AST directly means that you're tied to that version of the AST; this was considered really hard for compatibility over versions. Token streams are much easier to define an interface for, and more flexible: you're not just stuck with whatever AST representation we'd have given you.

For example: https://doc.rust-lang.org/book/first-edition/procedural-macr...

Re: Show HN: Pest – Fast parser generator written in Rust

#19

Post by dragostis »

DC-3 wrote: post #16
dragostis wrote: post #14

Earlier quoted context omitted.

For now, it produces a recursive descent parser. Packrat parsing is an open-question since I'm afraid that adding a memoization layer all throughout the parser will lead to a consistent general slowdown.

Interesting. I'll try and find some time to read the source - am interested to see how this is implemented.

A good place to start would be in the manually written example. [1] My current plan is to try and limit the the use of memoization such that it still guarantees linear parsing, but it doesn't memoize unless necessary.

[1]: https://github.com/pest-parser/pest/blob/master/pest/example...

Re: Show HN: Pest – Fast parser generator written in Rust

#20

Post by dragostis »

hobofan wrote: post #5

Shouldn't the title better by "Pest - Fast, modern parser _in_ Rust"? The current title made me think that this would be a Rust AST parser, similar to syntex.

I agree. I guess I was trying hard to suggest the fact that it can be used _with_ Rust and not the fact that it is written _in_ Rust.
Post reply on HN