Needle: A DFA Based Regex Library That Compiles to JVM ByteCode
1–10 of 16 posts
Re: Needle: A DFA Based Regex Library That Compiles to JVM ByteCode
#2looks very interesting - coming from the perl (ie the original PCRE) it’s rather a pity that this is focused on the JVM since Java is not the most regex friendly authoring environment (sorry I do not know if other JVM denizens like Kotlin / Clojure have more natural regex syntax)
Larry Walls version 2 of PCRE is doing some interesting innovation which, while continuing the model of a character level syntax where punctuation and so on are the “verbs” as the regex Sub-Language (one of the Slangs of raku), now provides deep support for unicode (eg. a digit test looks at the unicode properties, not just ), and can be gradually unpacked using and methods all the way up to a full blown composable class based Grammar … so you can still write regex one-liners perl style but then evolve that to a small Grammar to reduce line noise and drive up code clarity
Are there any plans to provide your library on the MoarVM?
Re: Needle: A DFA Based Regex Library That Compiles to JVM ByteCode
#3C# does this as well. Its OOB Regex engine can compile expressions to automata at runtime, or there’s an alternate newer back-end to generate the source code at compile time, which is useful in AOT as you can’t JIT the IL there.
Similar to the blog post, this allows it to be one of the fastest: https://github.com/BurntSushi/rebar?tab=readme-ov-file#summa...
Re: Needle: A DFA Based Regex Library That Compiles to JVM ByteCode
#4A related project that might interest some people:
https://github.com/telekons/one-more-re-nightmare
And the pretty hard to find blog post about it: https://applied-langua.ge/posts/omrn-compiler.html
Re: Needle: A DFA Based Regex Library That Compiles to JVM ByteCode
#5A related project that might interest some people: https://github.com/telekons/one-more-re-nightmare And the pretty hard to find blog post about it: https://applied-langua.ge/posts/omrn-compiler.html
Now you had me hoping for an implementation of fitting Needle behind the existing Pattern/Matcher API, and a compiler plugin that substitutes Pattern.compile accordingly wherever the inputs are known at compile time.
Re: Needle: A DFA Based Regex Library That Compiles to JVM ByteCode
#6By looking at the benchmarks, it is interesting to see how much an advantage have non-backtracking engines versus the default Java one. It is also interesting that most of the regular expressions I happen to write are not backtracking, so it’s likely I’m paying a ticket every time for a feature I’m not going to see.
It would be interesting if the default Java class the implements regular expressions would detect whether backtracking is actually used in the regular expression when it is compiled and decide whether the full blown engine is really necessary or a lightweight one would be enough.
Re: Needle: A DFA Based Regex Library That Compiles to JVM ByteCode
#7looks very interesting - coming from the perl (ie the original PCRE) it’s rather a pity that this is focused on the JVM since Java is not the most regex friendly authoring environment (sorry I do not know if other JVM denizens like Kotlin / Clojure have more natural regex syntax) Larry Walls version 2 of PCRE is doing some interesting innovation which, while continuing the model of a character level syntax where punc…
Author here. I don't know much about the MoarVM, so no plans.
I have very idly thought about being able to output native code for use in other systems, but realistically, I have enough other ideas to keep my nights and weekends busy for a long time.
Re: Needle: A DFA Based Regex Library That Compiles to JVM ByteCode
#8A related project that might interest some people: https://github.com/telekons/one-more-re-nightmare And the pretty hard to find blog post about it: https://applied-langua.ge/posts/omrn-compiler.html
Now you had me hoping for an implementation of fitting Needle behind the existing Pattern/Matcher API, and a compiler plugin that substitutes Pattern.compile accordingly wherever the inputs are known at compile time.
Author here: one thing I didn't mention in the post, but which is mentioned in the issues/readme on GitHub is that while I've been working on this for awhile, it's still a pretty new project in many ways. So what you're describing is interesting, but probably premature. The generated class files can still be very large, and the compiler itself has comments like "TODO: O(n^2)" that make throwing arbitrary input at it a dodgy idea.
Beyond those factors, there's the matter of just proving out that my testing is good enough, and I haven't missed edge cases.
Re: Needle: A DFA Based Regex Library That Compiles to JVM ByteCode
#9FWIW, there is a similar project [1] which compiles a regex to bytecode. The benchmarks [2] are impressive.
[1] https://github.com/humio/jitrex [1] https://github.com/humio/jitrex?tab=readme-ov-file#performan...
Re: Needle: A DFA Based Regex Library That Compiles to JVM ByteCode
#10looks very interesting - coming from the perl (ie the original PCRE) it’s rather a pity that this is focused on the JVM since Java is not the most regex friendly authoring environment (sorry I do not know if other JVM denizens like Kotlin / Clojure have more natural regex syntax) Larry Walls version 2 of PCRE is doing some interesting innovation which, while continuing the model of a character level syntax where punc…
> the Slangs of raku
Sounds like the name of some race in a sci-fi story :)