Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In search of the perfect URL validation regex (2010)

mathiasbynens.be

Post reply on HN
Print view View original

61–67 of 67 posts

Re: In search of the perfect URL validation regex (2010)

#61

Post by Too »

tester756 wrote: post #52
azalemeth wrote: post #48

Honest question: there is a famous and very funny stack exchange answer on the topic of parsing html with a regex [1] that states that the problem is in general impossible and if if you find yourself doing this, something has gone wrong and you should re-evaluate your life choices / pray to Cthulu. So, does this apply to URLs? The fact that these regexes are....so huge...makes me think that something is fundamentally…

>So, does this apply to URLs? The fact that these regexes are....so huge...makes me think that something is fundamentally wrong Yes, if your regex is above {.../50/100/...} characters, then write parser. I struggle to understand why do people write those crazy regexes for emails, urls, html when probably in all popular technologies there are battle-tested parsers for those things.

On top of this the error messages with a regex will be very one-dimensional.

As an example, http://localhost/ is technically valid url, which he wants to block. Should this error say misformatted URL like all others?

Using regex to cover all such cases is really the wrong tool for the job.

Re: In search of the perfect URL validation regex (2010)

#62

Post by prox »

maciejgryka wrote: post #16

Using https://regex.help/ , I got this beauty which passes all the ones, which should pass. Obviously some room for improvement ;) But it works! ^(?:http(?:(?:://(?:(?:(?:code\.google\.com/events/#&product=browser|\-\.~_!\$&'\(\)\*\+,;=:%40:80%2f::::::@ex\.com|foo\.(?:bar/\?q=Test%20URL\-encoded%20stuff|com/(?:\(something\)\?after=parens|unicode_\(\)_in_parens|b_(?:\(wiki\)(?:_blah)?#cite\-1|b(?:_\(wiki\)_\(again\)|/…

Any sufficiently advanced technology is indistinguishable from magic.

This kind of feels like a magic spell :)

Re: In search of the perfect URL validation regex (2010)

#63

Post by prox »

aranchelk wrote: post #50
azalemeth wrote: post #48

Honest question: there is a famous and very funny stack exchange answer on the topic of parsing html with a regex [1] that states that the problem is in general impossible and if if you find yourself doing this, something has gone wrong and you should re-evaluate your life choices / pray to Cthulu. So, does this apply to URLs? The fact that these regexes are....so huge...makes me think that something is fundamentally…

Caveats: I know nothing of Chomsky Grammars, and I have only a passing familiarity with Cthulu, but IMO the real crux of the issue parsing html with regex (beyond all the “it’s hard”, “the spec is more complicated than you think”, “regex is impossible to read” etc.) is html is a recursive data structure, e.g. you can have a div, inside a div, inside a div ad infinitum. Regex, AFAIK, doesn’t allow you to describe recu…

The times I had to use it on HTML , I think I combined xPath with RegEx to close the mismatch.

Re: In search of the perfect URL validation regex (2010)

#64

Post by toomanybeersies »

gibsonf1 wrote: post #51

I use this: u.checkURL = function (string) { if ($.type(string) === "string") { if (/^(https?|ftp):(\/\/(((([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(%[\da-f]{2})|[!\$&'\(\)\*\+,;=]|:)*@)?((\[(|(v[\da-f]{1,}\.(([a-z]|\d|-|\.|_|~)|[!\$&'\(\)\*\+,;=]|:)+))\])|((\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])\.(\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])\.(\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])\.(\d|[1-9]\d|1\d\d|2[0-4]\d|25[…

If you're writing browser JS, just use the URL builtin

  const isValidUrl = urlString => {
    try {
      new URL(urlString);
      return true;
    } catch {
      return false;
    }
  };

Re: In search of the perfect URL validation regex (2010)

#65

Post by MPSimmons »

staticassertion wrote: post #53
jt2190 wrote: post #39

Earlier quoted context omitted.

There's also a question of what we're really trying to validate, IMHO. All of these regex patterns will tell you that a string looks like a URL, but they won't actually tell you if: There's any web server listening at that particular URL; Whether that server has the resource in that location; If that server is reachable from where you want to fetch it; etc.

> All of these regex patterns will tell you that a string looks like a URL, yeah that's it that's what they're trying to validate

It seems like the answer is almost always yes.

Re: In search of the perfect URL validation regex (2010)

#66

Post by elif »

chrismorgan wrote: post #10
elif wrote: post #3

Earlier quoted context omitted.

how would you even "parse" a url with a regex? dynamically defined named subpatterns for each url parameter? I think the best i could do on paper with a regex is say "yup this is a url" or maybe "yup i can count the number of params" Unless it was a specific url with specific params?

Match groups so you can split it up into scheme, username, password, host, port, path, query, fragment. Not difficult to approximate, though for best results with diverse schemes you’d want an engine that allows repeated named groups, and I don’t know if any do (JavaScript and Python don’t).

I mean ya that would match a query string, but it wouldn't parse it?
Post reply on HN