Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parsers don't have to be complicated

bkaradzic.github.io

Post reply on HN
Print view View original

61–70 of 77 posts

Re: Parsers don't have to be complicated

#61

Post by trashb »

imoverclocked wrote: post #8

The hardest thing about writing a parser is cognitively accepting what is going to be considered valid input. You can make the best parser that is fast and well specified but invariably someone will (ab)use it in an unexpected way. Famous examples: despite so many initial good intentions, html tags don’t need to be closed, JSON numbers are too often encoded as strings, YAML can look like what most people expect or it…

> html tags don’t need to be closed

That's a very explicit and much debated feature, even self closing tags. It is also one of the main factors that makes HTML distinct from xml. And a big reason why xhtml was created.

I agree that it makes for a much more complicated interpretation.

Re: Parsers don't have to be complicated

#62

Post by tester756 »

r3d wrote: post #23
tester756 wrote: post #22

Earlier quoted context omitted.

Why care about lang which doesnt really support strings well?

What do you think the libraries you use to parse these things are doing under the hood? Maybe you don't care? Fair enough.

use 10 meters of wrappers around C and its quirks? :P

Re: Parsers don't have to be complicated

#63

Post by srean »

aw1621107 wrote: post #56
srean wrote: post #54

Earlier quoted context omitted.

Take a look at Felix. https://felix-lang.github.io/felix/ Ignore the 'scripting' language claim.

Oh, that is certainly not what I was expecting at all. I stand corrected! I do have to wonder though - do you know what proportion of the C++ compiler time is spent parsing your generated C++ code vs. optimizing it?

Unfortunately no.

Felix is quite old at this point. It's a very interesting language, with many interesting ideas. It did not quite take off though.

Re: Parsers don't have to be complicated

#64

Post by aw1621107 »

srean wrote: post #63
aw1621107 wrote: post #56

Earlier quoted context omitted.

Oh, that is certainly not what I was expecting at all. I stand corrected! I do have to wonder though - do you know what proportion of the C++ compiler time is spent parsing your generated C++ code vs. optimizing it?

Unfortunately no. Felix is quite old at this point. It's a very interesting language, with many interesting ideas. It did not quite take off though.

What favorite feature(s) do you miss when working in other languages?

Re: Parsers don't have to be complicated

#65

Post by srean »

aw1621107 wrote: post #64
srean wrote: post #63

Earlier quoted context omitted.

Unfortunately no. Felix is quite old at this point. It's a very interesting language, with many interesting ideas. It did not quite take off though.

What favorite feature(s) do you miss when working in other languages?

Coroutines, cooperative threading using fibres, type classes, generics, type deduction, easy interface with C++. The functional style, pattern matching. Flow based programming using 'chips and wires' abstraction.

It has many other interesting capabilities, for example, the ability to change its own grammar, that is rather too much, not for a pleb like me. It has unique (linear and affine) types too. It is really quite a handful.

Go did bring coroutines back into limelight but Felix predates Go by a margin.

Skaller, Felix's author, used Felix as a playground for novel language design ideas, so it was always in a state of flux.

Re: Parsers don't have to be complicated

#66

Post by craftkiller »

imoverclocked wrote: post #8

The hardest thing about writing a parser is cognitively accepting what is going to be considered valid input. You can make the best parser that is fast and well specified but invariably someone will (ab)use it in an unexpected way. Famous examples: despite so many initial good intentions, html tags don’t need to be closed, JSON numbers are too often encoded as strings, YAML can look like what most people expect or it…

> JSON numbers are too often encoded as strings

There's a good reason for that, since JSON comes from JavaScript, many JSON parsers treat numbers as double-precision floats. By encoding your number as a string, you ensure that the JSON parser has not modified your number.

https://blog.json-everything.net/posts/numbers-are-numbers-n...

Re: Parsers don't have to be complicated

#67

Post by HackerThemAll »

inigyou wrote: post #41
HackerThemAll wrote: post #20

Earlier quoted context omitted.

\r\n?|\n handles all EOL sequences without backtracking. Or write a non-regex equivalent of that.

I'm sure every time you split something on newlines you remember to use a regex.

I tend not to, because it's an overkill, but this regex nicely sums what needs to be done.

Re: Parsers don't have to be complicated

#68

Post by f311a »

meindnoch wrote: post #21
f311a wrote: post #4

Unfortunately, simple URL parsing breaks on so many things. There is a reason on why every URL parsing library is at least a few thousand LOCs. One common way to test it is just to pass ipv6 url: http://[f021:d981:b487:e57d:193e:550e::]/

Is that so? RFC 3986 Appendix B [1] "Parsing a URI Reference with a Regular Expression": The following line is the regular expression for breaking-down a well-formed URI reference into its components. ^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))? scheme = $2 authority = $4 path = $5 query = $7 fragment = $9 Let's test your URI with this regex, shall we? [2] $2 (scheme) = http $4 (authority) = [f021:d981:b…

Regexes are pretty slow, though.

Re: Parsers don't have to be complicated

#69

Post by afiori »

meindnoch wrote: post #21
f311a wrote: post #4

Unfortunately, simple URL parsing breaks on so many things. There is a reason on why every URL parsing library is at least a few thousand LOCs. One common way to test it is just to pass ipv6 url: http://[f021:d981:b487:e57d:193e:550e::]/

Is that so? RFC 3986 Appendix B [1] "Parsing a URI Reference with a Regular Expression": The following line is the regular expression for breaking-down a well-formed URI reference into its components. ^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))? scheme = $2 authority = $4 path = $5 query = $7 fragment = $9 Let's test your URI with this regex, shall we? [2] $2 (scheme) = http $4 (authority) = [f021:d981:b…

> well-formed URI

A parser that assumes the input to be already valid is usually not enough for most applications

according to that regex this is a valid url

__..__..%%%zz..

Re: Parsers don't have to be complicated

#70

Post by conartist6 »

roaringrocky wrote: post #60
zabzonk wrote: post #50

FORTH parsers are ultra simple - get the next space-separated token, if it is a number, push it on the stack, otherwise it's a word - look it up in the dictionary and (if it exists there) execute it.

Even simpler is you go the colorforth route, part of the source code is "pre parsed" by the editor, a prefix byte is added to each word (which is shown as different colors), then a simple dispatch loop with that prefix as a sort of opcode.

This feels like a precursor to the project I'm doing, which I us to make an editor whose state and output are expressed as such "pre-parsed" syntax trees. I'm trying to make mine fully language agnostic though
Post reply on HN