Hyperscan: A Fast Multi-Pattern Regex Matcher for Modern CPUs
1–10 of 37 posts
Re: Hyperscan: A Fast Multi-Pattern Regex Matcher for Modern CPUs
#2Author here, in case anyone wants to question, abuse, argue, etc.
Re: Hyperscan: A Fast Multi-Pattern Regex Matcher for Modern CPUs
#3This seems tailor-made for deep packet inspection.
Re: Hyperscan: A Fast Multi-Pattern Regex Matcher for Modern CPUs
#4This seems tailor-made for deep packet inspection.
There is no "seems" about it. The linked paper mentions DPI over and over.
Re: Hyperscan: A Fast Multi-Pattern Regex Matcher for Modern CPUs
#5Author here, in case anyone wants to question, abuse, argue, etc.
Thank you! I' m very excited to test in my projects ( I found a working Python extension here https://github.com/shenfe/python-hyperscan)
Let me seize the opportunity. I have a problem where I need to match multiple person names (hundreds of thousands) in huge texts. Aho-Corasick works good for exact matches. Could HyperScan works for approximated matches?
Re: Hyperscan: A Fast Multi-Pattern Regex Matcher for Modern CPUs
#6This seems tailor-made for deep packet inspection.
Well, it kinda is. AFAIK, that’s exactly what this was made for, and Intel/McAfee was going to create a DPI device using this tech. That never happened, and now Snort uses/will use hyperscan for its pattern matcher
Re: Hyperscan: A Fast Multi-Pattern Regex Matcher for Modern CPUs
#7This seems tailor-made for deep packet inspection.
Well, it kinda is. AFAIK, that’s exactly what this was made for, and Intel/McAfee was going to create a DPI device using this tech. That never happened, and now Snort uses/will use hyperscan for its pattern matcher
Hello, fellow Sensory employee!
Re: Hyperscan: A Fast Multi-Pattern Regex Matcher for Modern CPUs
#8This appears to be the same algorithm used by ripgrep (https://github.com/BurntSushi/ripgrep) for searching when SIMD/AVX is enabled. Specifically, it uses the algorithm Teddy from the library that’s derived from this paper.
Re: Hyperscan: A Fast Multi-Pattern Regex Matcher for Modern CPUs
#9This appears to be the same algorithm used by ripgrep ( https://github.com/BurntSushi/ripgrep ) for searching when SIMD/AVX is enabled. Specifically, it uses the algorithm Teddy from the library that’s derived from this paper.
Teddy seems to be just one small specialized chunk of hyperscan.
Re: Hyperscan: A Fast Multi-Pattern Regex Matcher for Modern CPUs
#10This appears to be the same algorithm used by ripgrep ( https://github.com/BurntSushi/ripgrep ) for searching when SIMD/AVX is enabled. Specifically, it uses the algorithm Teddy from the library that’s derived from this paper.
ripgrep author here. Yes, the Teddy algorithm was originally extracted from Hyperscan. But this is a teeny tiny piece of Hyperscan. And AFAIK, ripgrep's implementation (which is actually in the underlying regex library) doesn't carry over the full Teddy algorithm. Or so I've been told. :-)