Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The true power of regular expressions (2012)

npopov.com

Post reply on HN
Print view View original

11–20 of 62 posts

Re: The true power of regular expressions (2012)

#11

Post by Gabriel54 »

lbriner wrote: post #4

Something that seems obvious but not always implied by people's comments is that people are rarely trying to match an entire document with a regular expression so it doesn't really matter that "HTML is not a regular language". If I am trying to e.g. count div tags with a regex like " As soon as you also add character classes to ignore various parts of the document that you are not interested in like " ]*>" or whateve…

Regex can also be horribly slow - it depends on the particular regex you are using.

Re: The true power of regular expressions (2012)

#12

Post by AussieWog93 »

It might just be a me problem, but I've always been wary of regexes. They're not too bad to write, but reading them back and understanding what's actually going on can get a bit hairy. Plus, all of the subtle differences between regex libraries seems like a bit of a footgun.

Obviously they have their place, but I know a lot of the older guys seemed to love them way more than the young.

Re: The true power of regular expressions (2012)

#14

Post by rhdunn »

AussieWog93 wrote: post #12

It might just be a me problem, but I've always been wary of regexes. They're not too bad to write, but reading them back and understanding what's actually going on can get a bit hairy. Plus, all of the subtle differences between regex libraries seems like a bit of a footgun. Obviously they have their place, but I know a lot of the older guys seemed to love them way more than the young.

Various libraries (e.g. Python's `re` library) support comments and whitespace as an option allowing you to format the regex on multiple lines with commenting to document what each part does.

I'm not sure if there are any regex libraries that support DSLs and easy composability (e.g. the email RFC regex would be easier to read/maintain if you could specify the individual parts like are defined in the RFCs).

Re: The true power of regular expressions (2012)

#15

Post by jjice »

isqueiros wrote: post #13

Before the AI craze, I'd gotten quite good at writing regexes. Regexr was quite useful for decoding and composing them. I feel like they're going to become a lost art.

Totally agree. Selfishly, I was always the "regular expression" guy because they were a bit hobby space of mine (engine implementation and such), so seeing LLMs rip them is a bid of a bummer.

Half the reason it's a bummer is because I've seen coworkers who don't know when a regular expression is very suboptimal performance wise, but the LLM has no problem spitting it out. Part of really understanding regular expressions is knowing when to not use them.

The one that sticks in my head is when I was debugging some code that I was suspicious was causing our high memory consumption on a simple API service just to find out the regular expression was being used to strip a potential "data" front of a base64 encoded file (apparently someone thought we should do that instead of rejecting the payload). The regular expression scanned an entire base64 string that was up to 50 MB for the raw file, so about 66MB base64 encoded. I'll tell you what, replacing it with a loop over the first handful of characters solved all the problems. It should've never been a regular expression. If you see regular expressions as an archaic language that solve string problems, and now the magic box can make them for you, you're in for hell.

Re: The true power of regular expressions (2012)

#16

Post by bob1029 »

lbriner wrote: post #4

Something that seems obvious but not always implied by people's comments is that people are rarely trying to match an entire document with a regular expression so it doesn't really matter that "HTML is not a regular language". If I am trying to e.g. count div tags with a regex like " As soon as you also add character classes to ignore various parts of the document that you are not interested in like " ]*>" or whateve…

Are we also counting divs inside comments or literals within scripts?

There is a reason this advice is default. The chances an edge case exist are probably a lot higher than anyone is prepared to accept. Even in the "simple" cases.

Re: The true power of regular expressions (2012)

#18

Post by AussieWog93 »

rhdunn wrote: post #14
AussieWog93 wrote: post #12

It might just be a me problem, but I've always been wary of regexes. They're not too bad to write, but reading them back and understanding what's actually going on can get a bit hairy. Plus, all of the subtle differences between regex libraries seems like a bit of a footgun. Obviously they have their place, but I know a lot of the older guys seemed to love them way more than the young.

Various libraries (e.g. Python's `re` library) support comments and whitespace as an option allowing you to format the regex on multiple lines with commenting to document what each part does. I'm not sure if there are any regex libraries that support DSLs and easy composability (e.g. the email RFC regex would be easier to read/maintain if you could specify the individual parts like are defined in the RFCs).

I honestly never knew that, should give it another go.

Re: The true power of regular expressions (2012)

#20

Post by ogogmad »

isqueiros wrote: post #13

Before the AI craze, I'd gotten quite good at writing regexes. Regexr was quite useful for decoding and composing them. I feel like they're going to become a lost art.

Regular expressions will always remain fundamental to computer science: They characterise all of those - and only those - conditions on bytestrings (or bitstrings, or Unicode strings, etc) which are checkable in constant memory.* In other words, they characterise the set of all "regular languages", which is a name for DSPACE(O(1)). Furthermore, regular expressions can be matched in O(n) time and O(1) memory, within a single left-to-right pass, which is the highest level of efficiency mathematically possible. Since they operate on bytestrings, they can be applied to computer memory and computer state itself, which are ultimately just bytestrings, and not just to text.

To be fair, you might know all of that, but I wanted to highlight this. LLMs are a lot less efficient than regular expressions wherever both are applicable, simply because everything is less efficient than regular expressions.

* By constant memory, I mean that the memory usage has a maximum value independent of the size or the contents of the input bytestring.

Post reply on HN