Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Nevod is easier and faster than RegExp

nevod.nezaboodka.com

Post reply on HN
Print view View original

41–50 of 82 posts

Re: Show HN: Nevod is easier and faster than RegExp

#41

Post by mikecapizzi »

bazzargh wrote: post #35
jasode wrote: post #25

Like the other sibling comments mentioned, I too was confused about "100x faster than regex" and what the actual product was about. After digging around their website, I found this blog post which explains it better: https://blog.nezaboodka.com/post/2019/594-using-nevod-for-te... So my summary would be: 1) it works "faster" than regex in a specific scenario of treating text as entities in natural language . (E.g. hig…

My reading of that is that they compared apples and oranges. They didn't use NLP there at all (stemming, parts of speech tagging, etc); they just relaxed handling of whitespace. The Nevod 'equivalent' was a less general expression to make it seem more maintainable. The Nevod example translates to (ruby): pattern = Regexp.new( "(? ejection fraction|LVEF)( by visual inspection)? (? (is|of)( (at least|about|greater than…

If you're looking for a tool that allows you to incorporate legitimate NLP approaches, you should have a look at `odin`. Here's a paper https://doi.org/10.1093/database/bay098 showing its usage in the medical domain.

And the code is open-sourced as part of the `processors` library out of the CLULab at the University of Arizona: https://github.com/clulab/processors

The most detailed (though not completely up-to-date) documentation is probably in the manual here: https://arxiv.org/abs/1509.07513

I'm using it at my current job to build an analysis tool for customer-agent phone calls.

It allows you to build rules that match on different levels of abstraction: tokens, pos-tags, dependency paths. You can even match tokens based on word similarity (as measured by cosine similarity of word vectors).

And these rules can "cascade" (i.e. build off of each other). So you can find an entity or event in rule 1 and then look for how that interacts with another matched entity or event in a later rule.

Re: Show HN: Nevod is easier and faster than RegExp

#42

Post by ggm »

Is this another syntax (of the mechanisms written form) vs semantics (of how it actually operates) confusion moment?

If Nevod builds a different textual model and applies what you "say" to it, differently to the regex underlying model, thats about the speedup. how you say what you want in pattern matching, thats just a pure syntactic moment: I like regex from the UNIX philosophy because of the syntax fluidity for saying things. What actually happens when I say a|b|c|d is it builds a DFA and its not that bad, but if I do individual /a/p /b/p /c/p patterns in SED, same engine, but no DFA builder, its slow.

So is Nevod a new syntax and a DSL tied to language, or is it a new syntax and a generalized text matching model with some real semantic shift from regex?

Re: Show HN: Nevod is easier and faster than RegExp

#44

Post by cestith »

Drup wrote: post #32
tux1968 wrote: post #29

Earlier quoted context omitted.

Regular grammars a strict subset of what Perl 6 will parse, no? If you stick to that subset it will parse in a single pass.

Except Perl 6 doesn't enforce it, so you have to guess yourself if you are really in the regular case. Additionally, PCRE used to be exponential for certain "bad cases", some of which were (truly) regular expressions. My point is simply that Perl doesn't give you any guarantee except "we will try to parse it". Maybe you will hit the right case for the right version of Perl, who knows ? The Web is full of DDOS attack…

PCRE isn't Perl.

Re: Show HN: Nevod is easier and faster than RegExp

#48

Post by ychetyrko »

First of all, thanks everyone for the valuable feedback, critics, suggestions, etc. Truly appreciate that!

And thanks for patience to all people who are playing with the Nevod right now and getting errors. We have an unexpectedly high interest and number of visitors is very high in our playground. Meanwhile, it's a preview of the technology, please keep in mind.

Let me clarify few things. The speed of Nevod is based on two things:

1. Nevod operates on words, not individual characters. From my 30 years of coding experience I would say that roughly 95% of text processing/rules are word-based. So it covers most of tasks.

2. Nevod matches MULTIPLE patterns against document in ONE PASS. Patters/expressions are indexed by state machine and filtered effectively during matching.

So, at a RULE DECK of 1000 patterns, we got 300 times improvement comparing to RegExp. This is a kind of tasks when you let say need to highlight syntax, to massively verify and route incoming requests in cloud, perform massive text validations, track certain phrases and relations in massive human communication, etc.

With growing number of patterns the difference between Nevod and RegExp is higher and higher and go far beyond 300 times. So, I'm a kind of disagree with moderators removed the word "hundreds" from the headline. :)

We will publish benchmark results and we will also release "negrep" (Nevod GREP) command line tool for Windows, Linux, Mac, so everyone will be able to run "negrep", play with it and verify benchmark him/herself. Generally, Nevod technology will be available both in form of a library, in form of command line tool, and in form of service.

Re: Show HN: Nevod is easier and faster than RegExp

#49

Post by al2o3cr »

feanaro wrote: post #36

I see two novel aspects of this language: 1. the ability to easily break patterns into named subpatterns which can be referenced later on 2. the `@` operator which gives you the ability to talk about things inside these subpatterns These seem like worthwhile additions which would make regex more manageable. I don't see any reason why the whole of regex would need to be abandoned for some completely new, potentially p…

The Oniguruma engine has had named, callable subpatterns for a while:

https://github.com/kkos/oniguruma/blob/master/doc/RE#L400

Supported (via Oniguruma) in Ruby 2.0.0+, dunno about other languages.

Re: Show HN: Nevod is easier and faster than RegExp

#50

Post by jehna1 »

Here's a somewhat related project that aims to make regular expressions more verbose:

https://github.com/VerbalExpressions/JSVerbalExpressions

Here's a simple example for matching URLs:

    const tester = VerEx()
      .startOfLine()
      .then('http')
      .maybe('s')
      .then('://')
      .maybe('www.')
      .anythingBut(' ')
      .endOfLine();
The project has been ported to many different languages and it outputs a normal regular expression that you can use to match your text.
Post reply on HN