Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adding lookbehinds to rust-lang/regex

systemf.epfl.ch

Post reply on HN
Print view View original

11–20 of 36 posts

Re: Adding lookbehinds to rust-lang/regex

#11

Post by librasteve »

It’s odd to see such a widely adopted language as Rust only just getting some regex basics. Whereas Raku (https://raku.org) has made a strong forward step in regex syntax over PCRE, made by the same language designer with implementation of modern unicode savvy features like Grapheme and Diacritic handling that are essential to building consistent code to handle multilingual needs.

  say "Cool" ~~ /* /; # ｢Cool｣
  say "Cześć" ~~ m:ignoremark/ Czesc /;               # ｢Cześć｣
  say "WEIẞE" ~~ m:ignorecase/ weisse /;              # ｢WEIẞE｣
  say "หนูแฮมสเตอร์" ~~ /+/;                    # ｢หนูแฮมสเตอร์｣

Re: Adding lookbehinds to rust-lang/regex

#12

Post by librasteve »

librasteve wrote: post #11

It’s odd to see such a widely adopted language as Rust only just getting some regex basics. Whereas Raku ( https://raku.org ) has made a strong forward step in regex syntax over PCRE, made by the same language designer with implementation of modern unicode savvy features like Grapheme and Diacritic handling that are essential to building consistent code to handle multilingual needs. say "Cool" ~~ / * /; # ｢Cool｣ say…

huh … guess HN blocks emojis

Re: Adding lookbehinds to rust-lang/regex

#13

Post by burntsushi »

librasteve wrote: post #11

It’s odd to see such a widely adopted language as Rust only just getting some regex basics. Whereas Raku ( https://raku.org ) has made a strong forward step in regex syntax over PCRE, made by the same language designer with implementation of modern unicode savvy features like Grapheme and Diacritic handling that are essential to building consistent code to handle multilingual needs. say "Cool" ~~ / * /; # ｢Cool｣ say…

It's not only just getting some "regex basics." The `fancy-regex` crate has provided look-behind for years. The OP is about adopting look-behind to the linear time guarantee required by the `regex` crate.

My main focus for the `regex` crate has been on performance: https://github.com/BurntSushi/rebar

How does Raku's regex performance compare to Perl?

Re: Adding lookbehinds to rust-lang/regex

#14

Post by shawn_w »

librasteve wrote: post #11

It’s odd to see such a widely adopted language as Rust only just getting some regex basics. Whereas Raku ( https://raku.org ) has made a strong forward step in regex syntax over PCRE, made by the same language designer with implementation of modern unicode savvy features like Grapheme and Diacritic handling that are essential to building consistent code to handle multilingual needs. say "Cool" ~~ / * /; # ｢Cool｣ say…

I don't think Philip Hazel, who wrote PCRE, has anything to do with perl or raku development.

Re: Adding lookbehinds to rust-lang/regex

#15

Post by masklinn »

RadiozRadioz wrote: post #7

From a user perspective, this is extremely valuable. What an amazing improvement; unbounded especially. I do hope this would make it into actual RE2 & go. When I use regex, I expect to be able to lookbehind, so I am routinely hit by RE2's limitations in places where it's used. Sometimes the software uses the entire matched string and you can't use non-capturing groups to work around it. I understand go's reasons, ReD…

The authors’ previous article (linked in this one) was about doing this in re2 (https://systemf.epfl.ch/blog/re2-lookbehinds/), and they have a fork with those changes though I don’t know that they have a PR.

> the "purism" of RE2 does fly in the face of practicality to an irksome degree

It’s not purism tho. There are very practical reasons to want an FA-based engine, and if you compromise that to get additional features then the engine is pointless, you could have just used a backtracking engine in the first place.

Re: Adding lookbehinds to rust-lang/regex

#16

Post by quotemstr »

librasteve wrote: post #11

It’s odd to see such a widely adopted language as Rust only just getting some regex basics. Whereas Raku ( https://raku.org ) has made a strong forward step in regex syntax over PCRE, made by the same language designer with implementation of modern unicode savvy features like Grapheme and Diacritic handling that are essential to building consistent code to handle multilingual needs. say "Cool" ~~ / * /; # ｢Cool｣ say…

This right here is one of the foundational splits in the programming community. This article is all about how cool an _implementation_ is. This comment is about some other engine's cool _syntax_. Deep versus superficial. The two camps can't stand each other.

Re: Adding lookbehinds to rust-lang/regex

#17

Post by kibwen »

burntsushi wrote: post #13
librasteve wrote: post #11

It’s odd to see such a widely adopted language as Rust only just getting some regex basics. Whereas Raku ( https://raku.org ) has made a strong forward step in regex syntax over PCRE, made by the same language designer with implementation of modern unicode savvy features like Grapheme and Diacritic handling that are essential to building consistent code to handle multilingual needs. say "Cool" ~~ / * /; # ｢Cool｣ say…

It's not only just getting some "regex basics." The `fancy-regex` crate has provided look-behind for years. The OP is about adopting look-behind to the linear time guarantee required by the `regex` crate. My main focus for the `regex` crate has been on performance: https://github.com/BurntSushi/rebar How does Raku's regex performance compare to Perl?

> the linear time guarantee required by the `regex` crate

Making sure this line isn't glossed over: the point of the regex crate is that it provides linear-time guarantees for arbitrary regexes, making it safe (within reason) to expose the regex engine to untrusted input without running the risk of trivial DoS. From what I can tell, supporting lookbehinds in such a context is something that researchers have only recently described.

Re: Adding lookbehinds to rust-lang/regex

#18

Post by chubot »

RadiozRadioz wrote: post #7

From a user perspective, this is extremely valuable. What an amazing improvement; unbounded especially. I do hope this would make it into actual RE2 & go. When I use regex, I expect to be able to lookbehind, so I am routinely hit by RE2's limitations in places where it's used. Sometimes the software uses the entire matched string and you can't use non-capturing groups to work around it. I understand go's reasons, ReD…

What are some examples of problems where you’ve used lookbehinds?

Re: Adding lookbehinds to rust-lang/regex

#19

Post by librasteve »

shawn_w wrote: post #14
librasteve wrote: post #11

It’s odd to see such a widely adopted language as Rust only just getting some regex basics. Whereas Raku ( https://raku.org ) has made a strong forward step in regex syntax over PCRE, made by the same language designer with implementation of modern unicode savvy features like Grapheme and Diacritic handling that are essential to building consistent code to handle multilingual needs. say "Cool" ~~ / * /; # ｢Cool｣ say…

I don't think Philip Hazel, who wrote PCRE, has anything to do with perl or raku development.

sorry I didn't know that Philip Hazel wrote PCRE ... and I certainly credit the initiative to release Perl Compatible Regular Expressions from the grip of perl

my main point is that PCRE was based on perl regexes and that these were designed by Larry Wall and so he had some experience when it came to the strengths and weaknesses of of perl RE when it came to designing the Raku RE syntax (ie. the language formerly known as Perl 6)

Re: Adding lookbehinds to rust-lang/regex

#20

Post by progbits »

RadiozRadioz wrote: post #7

From a user perspective, this is extremely valuable. What an amazing improvement; unbounded especially. I do hope this would make it into actual RE2 & go. When I use regex, I expect to be able to lookbehind, so I am routinely hit by RE2's limitations in places where it's used. Sometimes the software uses the entire matched string and you can't use non-capturing groups to work around it. I understand go's reasons, ReD…

While I agree this is a common golang theme, in this case I believe this decision predates the golang implementation and comes from the C++ RE2 days, no?
Post reply on HN