Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parsers don't have to be complicated

bkaradzic.github.io

Post reply on HN
Print view View original

71–77 of 77 posts

Re: Parsers don't have to be complicated

#71

Post by mamcx »

imoverclocked wrote: post #8

The hardest thing about writing a parser is cognitively accepting what is going to be considered valid input. You can make the best parser that is fast and well specified but invariably someone will (ab)use it in an unexpected way. Famous examples: despite so many initial good intentions, html tags don’t need to be closed, JSON numbers are too often encoded as strings, YAML can look like what most people expect or it…

And harder than that? Report the error, in a way that make some sense.

This is compounded by the fact that you need the semantics involved, the environment (ie: everything on scope), the source (that means you need to keep carrying big strings around).

And what is efficient means to be destructive, but you need instead the opposite for semantics, error messages, optimizations and the like.

Re: Parsers don't have to be complicated

#72

Post by aw1621107 »

srean wrote: post #65
aw1621107 wrote: post #64

Earlier quoted context omitted.

What favorite feature(s) do you miss when working in other languages?

Coroutines, cooperative threading using fibres, type classes, generics, type deduction, easy interface with C++. The functional style, pattern matching. Flow based programming using 'chips and wires' abstraction. It has many other interesting capabilities, for example, the ability to change its own grammar, that is rather too much, not for a pleb like me. It has unique (linear and affine) types too. It is really quit…

Huh, that does sound like quite the grab bag of features. Think I'll have to find time to further investigate. Thanks for taking the time to elaborate!

Re: Parsers don't have to be complicated

#73

Post by meindnoch »

afiori wrote: post #69
meindnoch wrote: post #21

Earlier quoted context omitted.

Is that so? RFC 3986 Appendix B [1] "Parsing a URI Reference with a Regular Expression": The following line is the regular expression for breaking-down a well-formed URI reference into its components. ^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))? scheme = $2 authority = $4 path = $5 query = $7 fragment = $9 Let's test your URI with this regex, shall we? [2] $2 (scheme) = http $4 (authority) = [f021:d981:b…

> well-formed URI A parser that assumes the input to be already valid is usually not enough for most applications according to that regex this is a valid url __..__..%%%zz..

>according to that regex this is a valid url >__..__..%%%zz..

Correctly. It is a valid relative URI, whose path is "__..__..%%%zz..".

Re: Parsers don't have to be complicated

#74

Post by meindnoch »

f311a wrote: post #68
meindnoch wrote: post #21

Earlier quoted context omitted.

Is that so? RFC 3986 Appendix B [1] "Parsing a URI Reference with a Regular Expression": The following line is the regular expression for breaking-down a well-formed URI reference into its components. ^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))? scheme = $2 authority = $4 path = $5 query = $7 fragment = $9 Let's test your URI with this regex, shall we? [2] $2 (scheme) = http $4 (authority) = [f021:d981:b…

Regexes are pretty slow, though.

cc @burntsushi

Re: Parsers don't have to be complicated

#75

Post by srean »

aw1621107 wrote: post #72
srean wrote: post #65

Earlier quoted context omitted.

Coroutines, cooperative threading using fibres, type classes, generics, type deduction, easy interface with C++. The functional style, pattern matching. Flow based programming using 'chips and wires' abstraction. It has many other interesting capabilities, for example, the ability to change its own grammar, that is rather too much, not for a pleb like me. It has unique (linear and affine) types too. It is really quit…

Huh, that does sound like quite the grab bag of features. Think I'll have to find time to further investigate. Thanks for taking the time to elaborate!

https://felix-tutorial.readthedocs.io/en/latest/

This would be a good starting point. More in the manual.

Re: Parsers don't have to be complicated

#76

Post by imoverclocked »

craftkiller wrote: post #66
imoverclocked wrote: post #8

The hardest thing about writing a parser is cognitively accepting what is going to be considered valid input. You can make the best parser that is fast and well specified but invariably someone will (ab)use it in an unexpected way. Famous examples: despite so many initial good intentions, html tags don’t need to be closed, JSON numbers are too often encoded as strings, YAML can look like what most people expect or it…

> JSON numbers are too often encoded as strings There's a good reason for that, since JSON comes from JavaScript, many JSON parsers treat numbers as double-precision floats. By encoding your number as a string, you ensure that the JSON parser has not modified your number. https://blog.json-everything.net/posts/numbers-are-numbers-n...

Is the good reason that parsers are doing this in the first place? To me, it sounds like a practice/implementation that necessitates a second layer of parsing. If so, then maybe JSON is not the optimal medium to be using for the data being distributed.

Re: Parsers don't have to be complicated

#77

Post by craftkiller »

imoverclocked wrote: post #76
craftkiller wrote: post #66

Earlier quoted context omitted.

> JSON numbers are too often encoded as strings There's a good reason for that, since JSON comes from JavaScript, many JSON parsers treat numbers as double-precision floats. By encoding your number as a string, you ensure that the JSON parser has not modified your number. https://blog.json-everything.net/posts/numbers-are-numbers-n...

Is the good reason that parsers are doing this in the first place? To me, it sounds like a practice/implementation that necessitates a second layer of parsing. If so, then maybe JSON is not the optimal medium to be using for the data being distributed.

The parsers doing this in the first place is the problem. The JSON spec doesn't specify how numbers should be stored/interpreted, so theoretically you could have a 2,000 bit integer in JSON. The lack of clarity around that could also be considered a flaw in the JSON spec. But from a practical standpoint, the most common JSON parser is in web browsers running javascript, and they'll parse to double-precision floats. So from a practical standpoint, if you don't want your numbers distorted, you need to be defensive about it and encode your numbers as strings (or use something other than JSON).
Post reply on HN