Live data from Hacker News

Show HN: Regular expression compilation visualized

compiler.org

11–20 of 40 posts

Re: Show HN: Regular expression compilation visualized

#11
I guess this might be hard, but it would be pretty cool to be able to "stop" the optimizations at some point and see the resulting code.

Say "give me the generated code for the Rabin–Scott powerset construction (NFA to DFA)". That way, one could see the impact of each optimization step.

Nevertheless, this is a pretty cool proeject!

Re: Show HN: Regular expression compilation visualized

#13
post #8

This is really cool, im just really surprised that you can afford that domain and just put one thing on it :D

Heh, the idea was to put more things on it, would be cool to visualize more algorithms. Too bad ReasonML became such a mess, I really liked the coding experience in it. Now I would probably need to look for alternatives before implementing more algorithms. I actually bought compiler.org to get a nice personal email address after selling a company I co-founded, but I didn't realize how difficult it is to spell "compil…

I do not program with ReasonML, but have looked it few times as it was touted as replacement for JS/TS/React. What made it a mess?

Re: Show HN: Regular expression compilation visualized

#14
I wonder what the state of the art is in regex compilation? It feels to me like it shouldn’t be compilation to a program that processes one character at a time as you’ll be doing a lot of branches and so can’t process the data that fast. It feels like there ought to be a way to take advantage of ILP or SIMD operations. Maybe regex just doesn’t matter enough for that to be important.

Re: Show HN: Regular expression compilation visualized

#15

I wonder what the state of the art is in regex compilation? It feels to me like it shouldn’t be compilation to a program that processes one character at a time as you’ll be doing a lot of branches and so can’t process the data that fast. It feels like there ought to be a way to take advantage of ILP or SIMD operations. Maybe regex just doesn’t matter enough for that to be important.

Many high-performance regex libraries use SIMD these days. Hyperscan might be the most prominent example of that, being made by Intel and all.

Re: Show HN: Regular expression compilation visualized

#16
post #13

Earlier quoted context omitted.

Heh, the idea was to put more things on it, would be cool to visualize more algorithms. Too bad ReasonML became such a mess, I really liked the coding experience in it. Now I would probably need to look for alternatives before implementing more algorithms. I actually bought compiler.org to get a nice personal email address after selling a company I co-founded, but I didn't realize how difficult it is to spell "compil…

I do not program with ReasonML, but have looked it few times as it was touted as replacement for JS/TS/React. What made it a mess?

See: https://news.ycombinator.com/item?id=24119838

Re: Show HN: Regular expression compilation visualized

#17

I wonder what the state of the art is in regex compilation? It feels to me like it shouldn’t be compilation to a program that processes one character at a time as you’ll be doing a lot of branches and so can’t process the data that fast. It feels like there ought to be a way to take advantage of ILP or SIMD operations. Maybe regex just doesn’t matter enough for that to be important.

Author here. The generated LLVM IR at the bottom actually make uses of vector instructions, which will make LLVM generate SIMD instructions if supported by the CPU.

For instance, try entering just a long word as the regex, e.g. "helloworld":

  %state0.goto.1.0.cmp_mask = icmp eq  %state0.rhs,  ; helloworld

Re: Show HN: Regular expression compilation visualized

#19

most of the crazy email regexes found here https://stackoverflow.com/questions/201323/how-to-validate-a... fail to validate in the input field

That's probably, because the regexes you linked to are Perl-flavoured regexes, whereas this website seems to use the "Unix Extended" variant of regex.

Re: Show HN: Regular expression compilation visualized

#20

I wonder what the state of the art is in regex compilation? It feels to me like it shouldn’t be compilation to a program that processes one character at a time as you’ll be doing a lot of branches and so can’t process the data that fast. It feels like there ought to be a way to take advantage of ILP or SIMD operations. Maybe regex just doesn’t matter enough for that to be important.

https://www.microsoft.com/en-us/research/wp-content/uploads/... is an interesting read on ILP with FSM's.

Regex is used all over the place, so it's certainly worthwhile to speed things up. And huge FST's are also used in language tech, e.g. to analyze all frillions of the forms of Finnish verbs into all their ambiguous readings with tags and dictionary forms: https://beta.apertium.org/index.eng.html?choice=fin&qA=Taite... The language supported by that FST is basically infinite due to derivations and compounding, but the zipped binary fits on a handful of floppies.

Post reply on HN