Single header Parser Combinators for C • Hacker News Board

Re: Single header Parser Combinators for C

#11

Post by RossBencina » Wed, Jul 01, 2026, 1:22 PM UTC

zombot wrote: post #3

So many parser combinators operate on bytes assuming ASCII input only. I'd be more interested in a parser combinator lib that has UTF-8 decoding already abstracted away, operating on `wchar_t`, or even polymorphic input stream element types.

I'm not familiar with parser combinators. The parser generators that I'm familiar with (YACC, ANTLR3,5) parse a stream of lexemes/tokens, not characters. Is there a reason why combinators don't operate on lexemes?

Re: Single header Parser Combinators for C

#12

Post by eska » Wed, Jul 01, 2026, 2:20 PM UTC

zombot wrote: post #3

So many parser combinators operate on bytes assuming ASCII input only. I'd be more interested in a parser combinator lib that has UTF-8 decoding already abstracted away, operating on `wchar_t`, or even polymorphic input stream element types.

I’d still use a byte slice for that. Some formats may mix encodings, or have a text header and binary payload. For those cases one would need to use memchr for the first byte, then compare the remaining few bytes. So I don’t think it would be a huge performance impact

Re: Single header Parser Combinators for C

#13

Post by Jtsummers » Wed, Jul 01, 2026, 4:24 PM UTC

RossBencina wrote: post #11

zombot wrote: post #3

So many parser combinators operate on bytes assuming ASCII input only. I'd be more interested in a parser combinator lib that has UTF-8 decoding already abstracted away, operating on `wchar_t`, or even polymorphic input stream element types.

I'm not familiar with parser combinators. The parser generators that I'm familiar with (YACC, ANTLR3,5) parse a stream of lexemes/tokens, not characters. Is there a reason why combinators don't operate on lexemes?

They can, it's just that often people seem to use parser combinators to build both the lexer and the parser, not just the parser, which means dealing with the character stream. If you separate the two steps, parser combinators just dealing with tokens works just fine.

Re: Single header Parser Combinators for C

#14

Post by paulddraper » Wed, Jul 01, 2026, 5:08 PM UTC

t-3 wrote: post #6

lokeg wrote: post #5

Earlier quoted context omitted.

Isn't working with the utf8 stream sufficient? Especially if you only have ASCII keywords/operators/brackets, I feel a ASCII parser should work with utf8 out of the box

Yeah, a parser has no need to understand what a string or glyph is, let alone ASCII or UTF-8. The point is to take a stream of arbitrary data and process it into something that can be reasoned about. Unless you know your input stream is regular in some way, processing it at the finest level of granularity (usually bytes) is probably the only thing to do.

Well it depends whether you parsing binary (byte stream) or text (character stream).

In practice, lots of text formats (JSON, XML) embed or hint the character encoding in the format.

Re: Single header Parser Combinators for C

#15

Post by t-3 » Wed, Jul 01, 2026, 5:54 PM UTC

RossBencina wrote: post #11

zombot wrote: post #3

So many parser combinators operate on bytes assuming ASCII input only. I'd be more interested in a parser combinator lib that has UTF-8 decoding already abstracted away, operating on `wchar_t`, or even polymorphic input stream element types.

I'm not familiar with parser combinators. The parser generators that I'm familiar with (YACC, ANTLR3,5) parse a stream of lexemes/tokens, not characters. Is there a reason why combinators don't operate on lexemes?

A parser combinator takes parsers as input and produces a new parser. The basic parsers are very simple, but they are combined together to produce more complex parsers.

Re: Single header Parser Combinators for C

#16

Post by steve-chavez » Wed, Jul 01, 2026, 9:19 PM UTC

sph wrote: post #2

Beautiful work! I'm not even gonna wonder if any of it was AI-generated, because the code is clearly crafted meticulously by an experienced C engineer, very readable, and shorter than I expected.

Many thanks! It took me a long time to come up with the interface. I've been wanting a good parser combinator library for C since 2023 when I contributed to mpc[1]:

[1]: https://github.com/orangeduck/mpc/commits?author=steve-chave...