Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nginx Unit

nginx.com

Post reply on HN
Print view View original

191–200 of 236 posts

Re: Nginx Unit

#191

Post by ekimekim »

vertex-four wrote: post #174
ekimekim wrote: post #146

Earlier quoted context omitted.

Minor quibble, in the context of serving static files (ie. from disk), go doesn't use async I/O, the file I/O blocks the thread until it's complete. But since go's scheduler is M:N this doesn't lock up the whole program, so your point stands.

Err, no, this is a misconception. All IO in Go is async - there is no sync IO in Go (as sync IO would block an entire OS thread). There is an internal registry mapping blocked file descriptors to goroutines - when a kernel IO function returns EAGAIN, the goroutine throws the file descriptor + goroutine info onto the registry and yields back to the scheduler. The scheduler occasionally checks all descriptors on the re…

I'm referring specifically to disk IO, which on linux using standard read(2) and write(2) is (almost) always blocking. What you describe is true of socket fds and some other things, but on most systems a file read/write which goes to a real disk will never return EAGAIN.

This is why systems like aio[1] exist, though afaik most systems tend to solve this with a thread pool rather than aio, which can be very complicated to use properly.

[1] http://man7.org/linux/man-pages/man7/aio.7.html

Re: Nginx Unit

#192

Post by Matthias247 »

tlb wrote: post #165
agnivade wrote: post #42

Earlier quoted context omitted.

Wait, zeromq lost momentum ? When did that happen ?

I recently switched from zeromq to straight libuv sockets with jsonl (\n-separated json) payloads. Because I'm working inside a Node process, combining zmq's threading model with Node's threading model was a pain. Now, there's a single IO thread which is the same as the Javascript engine thread, and I can use uv_work to run CPU-intensive tasks on multiple cores.

Do you not allow \n inside your JSON or encode your JSON as base64? If not you might have problems with disambiguating frame ends from line breaks inside frames.

A common way for framing is to prepend each frame with it's encoded length. That's easier, faster and less error-prone than searching for ASCII delimeters.

Re: Nginx Unit

#193

Post by shoover »

RantyDave wrote: post #51
FooBarWidget wrote: post #43

Earlier quoted context omitted.

When was the last time you heard something about it?

I'm using it for a project now. It's a bit weird, but it does work. Cool thing: you can slot a file descriptor into the zmq provided poll ... point is that you can poll on both zmq and sockets in the one loop. Pro Tip: Use 'cbor' for serialising.

Thanks for the tip. In which language(s) do you use CBOR? I want to like it, but the various C APIs look a bit cumbersome and lacking docs.

Re: Nginx Unit

#194

Post by sslalready »

bkeroack wrote: post #128

It's worth noting that it's rarely necessary or desirable to put an app server like nginx in front of Go HTTP server applications. The Go standard library http and TLS stack are production quality and rock solid. Putting something in front is mostly cargo culting from people more used to the worlds of PHP/Python/Ruby/etc.

I find it useful for filtering and caching. Things like redirecting traffic to /.well-known/acme-challenge/ to your certificate management host, providing an endpoint for machine status or filtering requests to dot-files. Or telling Nginx to cache responses and allow it to serve stale content when the backend server returns 4xx/5xx status codes during deploys or high load. Handling things like client certificate authentication in Nginx instead of doing it in every backend application is another thing I've found useful.

Re: Nginx Unit

#195

Post by vertex-four »

ekimekim wrote: post #191
vertex-four wrote: post #174

Earlier quoted context omitted.

Err, no, this is a misconception. All IO in Go is async - there is no sync IO in Go (as sync IO would block an entire OS thread). There is an internal registry mapping blocked file descriptors to goroutines - when a kernel IO function returns EAGAIN, the goroutine throws the file descriptor + goroutine info onto the registry and yields back to the scheduler. The scheduler occasionally checks all descriptors on the re…

I'm referring specifically to disk IO, which on linux using standard read(2) and write(2) is (almost) always blocking. What you describe is true of socket fds and some other things, but on most systems a file read/write which goes to a real disk will never return EAGAIN. This is why systems like aio[1] exist, though afaik most systems tend to solve this with a thread pool rather than aio, which can be very complicate…

Ah, absolutely, I forgot that the state of disk IO on Linux is terrible - although this still isn't quite the case, since there's a network socket involved in copying from disk to socket, so if the socket's buffer becomes full the scheduler will run.

It seems that nginx can use thread pools to offload disk IO, although doesn't unless configured to - by default disk IO will block the worker process. And FreeBSD seems to have a slightly better AIO system it can use, too.

Re: Nginx Unit

#196

Post by ivoras »

dec0dedab0de wrote: post #98
agnivade wrote: post #42

Earlier quoted context omitted.

Wait, zeromq lost momentum ? When did that happen ?

I think it happened in 2012 when this came out http://250bpm.com/blog:4 Before that, I used to hear people talking about it all the time.

Cool article! I guess that's why Go does its error handling without exceptions.

Re: Nginx Unit

#197

Post by falsedan »

falsedan wrote: post #92
etatoby wrote: post #72

Earlier quoted context omitted.

So they deployed a bad config file to all nodes and restarted the service, which then failed to start. How is this specific to Nginx? This same mistake is possible with any other software ever written.

nginx is faster at stop/starting?

downvoters: I'm deadly serious. I've seen plenty of deployment systems which were unbearably slow because it gave more time for a human to spot a bad deploy and cancel it, and who were afraid to replace it with something faster because it would lack this safety net.

Re: Nginx Unit

#198

Post by adrianratnapala »

rch wrote: post #122
agnivade wrote: post #42

Earlier quoted context omitted.

Wait, zeromq lost momentum ? When did that happen ?

It has the same momentum, but too much mass and not enough velocity.

In which direction do you want it do move, if you want velocity that is.

I more often find platforms have too much velocity. And if not too little mass, then too little solidity.

Re: Nginx Unit

#199

Post by adrianratnapala »

VT_Drew wrote: post #188
overcast wrote: post #132

Earlier quoted context omitted.

I never found Nginx especially simple to setup, the config files were always messy. Caddy seems to have knocked this out of the park for me, especially considering automated https, and redirection.

>the config files were always messy. How? It so much cleaner and simpler than Apache. I don't get this sentiment.

Simpler than apache doesn't mean simple. As someone who sets up HTTP servers rarely, I had trouble when I tried out NGINIX.

But I suspect for people who do it more seriously, then it's nginix config hits the sweet spot. To me the language seems sophisticated, well documented and fairly well behaved if you pay attention to the rules.

That's can make it too hard for someone casually trying to quick-start some experimental project. But it's exactly what you want if you are maintaining a long-lived set up that is likely to grow and become complicated over time.

Re: Nginx Unit

#200

Post by AgentME »

Matthias247 wrote: post #192
tlb wrote: post #165

Earlier quoted context omitted.

I recently switched from zeromq to straight libuv sockets with jsonl (\n-separated json) payloads. Because I'm working inside a Node process, combining zmq's threading model with Node's threading model was a pain. Now, there's a single IO thread which is the same as the Javascript engine thread, and I can use uv_work to run CPU-intensive tasks on multiple cores.

Do you not allow \n inside your JSON or encode your JSON as base64? If not you might have problems with disambiguating frame ends from line breaks inside frames. A common way for framing is to prepend each frame with it's encoded length. That's easier, faster and less error-prone than searching for ASCII delimeters.

If you do the JSON serialization yourself, there's no reason for newlines to be in the JSON. (Newlines within strings are encoded as \n.)
Post reply on HN