Live data from Hacker News

RE#: how we built the fastest regex engine in F#

iev.ee

71–80 of 90 posts

Re: RE#: how we built the fastest regex engine in F#

#71

F# is one of the biggest 'What could have beens'. Great language, that just didn't hit the right time, or reach critical mass of the gestalt of the community.

I convinced one boss to let me spike out a project with it. I was in love with OCaml at the time. OCaml's docs are... I'm just going to say it, they're terrible. F# on the other hand, has fantastic docs. In the end, the only real gripe I had was the significant whitespace. I'm just not a fan.

Re: RE#: how we built the fastest regex engine in F#

#73
post #62

Earlier quoted context omitted.

While i completely understand it, the lack of capitalization is just an indication that a human wrote this, it has to be imperfect i see enough slop and Look At Me on a daily basis. i don't want it to look like an ad or a LinkedIn post in 2026.

No one will mistake your posts for LinkedIn slop. You actually have something to say, with coherent arguments presented in paragraphs containing multiple sentences. If you want sentences without capitalization to be your thing, then go for it. It's just a weird hill to die on, taking away from the readability of your posts for no real reason.

In all honesty it's just never bothered me before and i've havent met many people bothered by it either

It's the same thing with dark mode as default, i chose it because it's my own preference and i'd love it everywhere, but i'm constantly being flashbanged by phone apps because someone decided #FFFFFF is a good background color while the app is loading.

Re: RE#: how we built the fastest regex engine in F#

#74
post #69
post #65

Earlier quoted context omitted.

> Multithreading is generally a non-issue, you just wrap the function that creates the state behind a lock/mutex, this is usually the default. But you also have to lock when reading the state, not just when writing/creating it. Wouldn’t that cause lock contention with sufficiently concurrent use?

No, we do not lock reading the state, we only lock the creation side and the transition table reference stays valid during matching even if it is outdated. Only when a nonexistent state is encountered during matching it enters the locked region.

Ah, I see, so it’s basically the Racy Single-Check Idiom.

Re: RE#: how we built the fastest regex engine in F#

#75

One thing I don't understand is what does _* mean? It seems like the paper refers to .* (which I understand) and _* (which I don't) in sometimes the same context? Normally _* would mean "an underscore zero or more times".

That's noted further down the page: - `_*` = any string

I guess _ is trying to be like, "No, really, anything," while . has some limitations?

Re: RE#: how we built the fastest regex engine in F#

#76
post #32
post #30

Earlier quoted context omitted.

Would SearchValues help there for a fallback to a SIMD optimized simple string literal search rather than the happy path?

Yes, that's exactly what we did to be competitive in the benchmarks. There's a lot of simple cases where you don't really need a regex engine at all. integrating SearchValues as a multi-string prefix search is a bit harder since it doesn't expose which branch matched so we would be taking unnecessary steps. Also .NET implementation of Hyperscan's Teddy algorithm only goes left to right.. if it went right to left it w…

So, there's still room for significant improvement.

Re: RE#: how we built the fastest regex engine in F#

#77
post #76
post #32

Earlier quoted context omitted.

Yes, that's exactly what we did to be competitive in the benchmarks. There's a lot of simple cases where you don't really need a regex engine at all. integrating SearchValues as a multi-string prefix search is a bit harder since it doesn't expose which branch matched so we would be taking unnecessary steps. Also .NET implementation of Hyperscan's Teddy algorithm only goes left to right.. if it went right to left it w…

So, there's still room for significant improvement.

There is plenty still to do.

One part of this is SIMD algorithms to better compete with Hyperscan/Rust, another is the decades of optimizations that backtracking engines have for short anchored matches for validation.

There's analysis to do for specific patterns so we can opt for specialized algorithms, eg. for fixed length patterns we skip the left-to-right pass entirely since we already know the match start + match length.

Lots of opportunistic things like this which we haven't done. Also there are no statistical optimizations in the engine right now. Most engines will immediately start looking for a 'z' if there is one in the pattern since it is rare.

Re: RE#: how we built the fastest regex engine in F#

#78

I love regular expression derivatives. One neat thing about regular expression derivatives is they are continuation-passing style for regular expressions. The derivative is "what to do next" after seeing a character, which is the continuation of the re. It's a nice conceptual connection if you're into programming language theory. Low-key hate the lack of capitalization on the blog, which made me stumble over every se…

It's so uncomfortable to read. Why do people do this? They capitalize names, so clearly their shift key works. Do they do it feel special or like some sort of rebel?

some folks prefer the aesthetic of lowercase and find capitalization unnecessary and ugly

Re: RE#: how we built the fastest regex engine in F#

#79

Earlier quoted context omitted.

That's noted further down the page: - `_*` = any string

I guess _ is trying to be like, "No, really, anything," while . has some limitations?

.* does NOT match newlines, so always will stop a match at the end of a line.

Re: RE#: how we built the fastest regex engine in F#

#80
post #66

Earlier quoted context omitted.

Maybe they drafted it on a phone where capitalization is harder. My guess is the all-lowercase world is mostly people who do most of their text creation on phones and similar, not keyboards.

I don’t really see how capitalization is harder on phones, I do it all the time.

if you turned on the autocorrects and spell checks etc, would automatically capitalize your sentences. Even when you don't want them to!
Post reply on HN