Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nginx design details

aosabook.org

Post reply on HN
Print view View original

21–30 of 30 posts

Re: Nginx design details

#21

Post by leeoniya »

mfjordvald wrote: post #20
leeoniya wrote: post #13

Earlier quoted context omitted.

Perhaps you have not been thrown someone else's list of 15-20 locations and tried to determine why one was not matching, and what was being matched before it based on specificity, whether or not it was a literal or regex, whether or not it halted further matching in the order that specific type was defined. The complexity is entirely unneeded, mod_rewrite could def use some minor convenience tweaks, but is far more i…

I've spent about 3 years doing support in #nginx. Perhaps I've just been thrown someone else's list of 15-20 locations too many times. Locations definitely need reworking, never said they didn't, I just took issue with the having to check the verbose logs as it's perfectly possible without.

You don't have to if you've done it for 3 years and are familiar with nginx's "rules". Otherwise if you want to avoid wasting 3 hours, you sometimes have to.

Those rules are not well defined either. What is a "literal string" / "conventional string"? Ok, "=" is clear, but what is /images/, how about /im[ag]es/, and /images/$? Since these literal strings are not quoted, what black magic is used to classify the location directive as regex or literal? I would actually love to know this, probably the biggest stumbling point for me. Especially since it uses decoded uris, because "$" could have been a "%24" in a literal string path, OR it could be a regex end of line assertion...wtf?

if you can somehow understand this without having to poke the logs or through vast experience of prior trial and error, my hat's off to you, sir.

Re: Nginx design details

#22

Post by densh »

pooriaazimi wrote: post #4

This is great! I didn't know that volume II has been published... Also be sure to read the chapter about LLVM compiler family (written by LLVM creator, who is an Apple employee now): http://www.aosabook.org/en/llvm.html It's great . A learnt a lot from that chapter. This book (The Architecture of Open Source Application) is a treasure trove. Just look at the index here: http://www.aosabook.org/en/index.html and tell…

Thank you for pointing out that other chapters are available for free too. It's indeed an amazing read. (It contains chapters about internal organization of Mercurial, Puppet, Haskell compiler and so much more.)

Re: Nginx design details

#23

Post by harshreality »

leeoniya wrote: post #21
mfjordvald wrote: post #20

Earlier quoted context omitted.

I've spent about 3 years doing support in #nginx. Perhaps I've just been thrown someone else's list of 15-20 locations too many times. Locations definitely need reworking, never said they didn't, I just took issue with the having to check the verbose logs as it's perfectly possible without.

You don't have to if you've done it for 3 years and are familiar with nginx's "rules". Otherwise if you want to avoid wasting 3 hours, you sometimes have to. Those rules are not well defined either. What is a "literal string" / "conventional string"? Ok, "=" is clear, but what is /images/ , how about /im[ag]es/ , and /images/$ ? Since these literal strings are not quoted, what black magic is used to classify the loca…

The type of match is made explicit before the string/pattern.

"location = string" and "location string" and "location ^~ string" are non-regex matches.

"location ~ pattern" and "location ~* pattern" are regex matches.

Matching order is unintuitive and therefore bad, and there are plenty of other quirks with nginx configs that should be made more intuitive, but confusion between regex and non-regex is unusual.

It's all described right here: http://wiki.nginx.org/HttpCoreModule#location

Re: Nginx design details

#24

Post by leeoniya »

harshreality wrote: post #23
leeoniya wrote: post #21

Earlier quoted context omitted.

You don't have to if you've done it for 3 years and are familiar with nginx's "rules". Otherwise if you want to avoid wasting 3 hours, you sometimes have to. Those rules are not well defined either. What is a "literal string" / "conventional string"? Ok, "=" is clear, but what is /images/ , how about /im[ag]es/ , and /images/$ ? Since these literal strings are not quoted, what black magic is used to classify the loca…

The type of match is made explicit before the string/pattern. "location = string" and "location string" and "location ^~ string" are non-regex matches. "location ~ pattern" and "location ~* pattern" are regex matches. Matching order is unintuitive and therefore bad, and there are plenty of other quirks with nginx configs that should be made more intuitive, but confusion between regex and non-regex is unusual. It's al…

ok, read through it again. the regex distinction is clear, that was my bad. the choice of prefix symbols is pretty poor. ^~ looks related to ~ and ~*. in the docs, the regex prefixes are presented first, so i associated ~ as a regex indicator. how wrong i was.

i generally disagree with ^ being the universal "not" indicator out of context since it's a "beginning of line assertion" when not preceded by "[" within a regex. The fact that it indicates what follows is a regex-halting literal string prefix match (exactly what ^ would indicate in a normal regex) is plain confusing.

thanks

Re: Nginx design details

#25

Post by mfjordvald »

leeoniya wrote: post #21
mfjordvald wrote: post #20

Earlier quoted context omitted.

I've spent about 3 years doing support in #nginx. Perhaps I've just been thrown someone else's list of 15-20 locations too many times. Locations definitely need reworking, never said they didn't, I just took issue with the having to check the verbose logs as it's perfectly possible without.

You don't have to if you've done it for 3 years and are familiar with nginx's "rules". Otherwise if you want to avoid wasting 3 hours, you sometimes have to. Those rules are not well defined either. What is a "literal string" / "conventional string"? Ok, "=" is clear, but what is /images/ , how about /im[ag]es/ , and /images/$ ? Since these literal strings are not quoted, what black magic is used to classify the loca…

As has been covered the type of matching being done is easy to tell from the prefix, though, until you get used to them you will probably be checking the documentation for just what each prefix means.

The real WTF is when choosing which location is the most specific when there is a string literal and a regex match.

Ignoring ^~ locations which will always be preferred over regex the only locations that can actually be chosen over regex is exact match "=" or default string literal (no prefix) that has an exact match. This part is what is typically the real cause of "I don't know WTF is going on here."

Re: Nginx design details

#26

Post by tuxychandru »

cluda01 wrote: post #19
tuxychandru wrote: post #17

Earlier quoted context omitted.

I do not know about others but I do it because not mixing up web server responsibilities with application server is a good idea. For example, I leave things like gzip compression and TLS handling to nginx without having to deal with them in my app.

Using HTTP in general as an internal transport protocol has always struck me as odd. But I come from a financial services perspective where latency is the key driver.

I use ZeroMQ or Go's RPC libraries for internal communication and mostly use nginx as reverse proxy and to serve static files on the edge.

Re: Nginx design details

#27

Post by zaphar »

genwin wrote: post #15

I've read several times about people using the Go language's (golang) web server in conjunction with Nginx. Apparently Nginx provides some benefit that Go's web server doesn't, but I haven't seen that thing called out. Can anyone tell me what it is? Thanks. (I'd also love to see a performance comparison between these two.)

I do something like this so that I can serve more than one standalone golang web application behind the same server. It allows me to multiplex several different go binaries and an apache server on the same box with the same address. These are all personal projects that aren't high traffic so one vps is fine and I don't have to remember which app is on which port.

Re: Nginx design details

#28

Post by nl »

lnanek2 wrote: post #2

My one experience with this server was quite bad. The web group we contracted used it, but would constantly run into issues us developers could have helped them with had they used Apache. Things like needing to return certain headers in the request, or return certain status codes, or do certain things handled easily by Apache's large ecosystem of modules became very grueling trials full of meetings with the contracto…

would constantly run into issues us developers could have helped them with had they used Apache

I'm not sure that says what you think it does about Ngnix and the developers you deal with...

Re: Nginx design details

#29

Post by reirob »

I much appreciated the article. And even more when I discovered from other commenters that http://www.aosabook.org is a collection of architectural description of a lot of Open-Source projects. I immediately read about sendmail by Eric Allman (http://www.aosabook.org/en/sendmail.html) and I appreciated it actually more than the article about Nginx. There is a lot of wisdom in it and now I understand the reasons for some architectural decisions that have been taken for sendmail.

Re: Nginx design details

#30

Post by Vidart »

leeoniya wrote: post #5

My only quibble with Nginx so far is the magical order in which it evaluates location directives. http://wiki.nginx.org/HttpCoreModule#location On more than one occasion, you have to unintuitively try to figure out whether a rule at the bottom is getting matched before a rule at the top. The matching algo is quite intricate and easy to to trip up on. It tries to help novices by matching literal strings first for them…

It's not a magic. It's just an optimization. Prefixed strings are organized to the radix tree for fast configuration lookup.

But the only way to deal with regexps is to execute them sequentially.

btw, the official doc is here: http://nginx.org/en/docs/http/ngx_http_core_module.html#loca...

Post reply on HN