Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nginx 1.7.1 adds syslog support

nginx.org

Post reply on HN
Print view View original

41–45 of 45 posts

Re: Nginx 1.7.1 adds syslog support

#41

Post by noselasd »

IgorPartola wrote: post #38
lobster_johnson wrote: post #36

Earlier quoted context omitted.

In addition to what danudey says, modern sysloggers also support Unix domain sockets, which are reliable. Typically this is /dev/log; on Linux, I believe the GNU C Library's syslog() uses this by default. On Linux, sending UDP to localhost is very reliable and fast, essentially going through kernel buffers with very little overhead. You will only see dropped data if the system is extremely overloaded. I did some test…

While I agree with you on the whole, one minor point about UDP/datagrams: they are not reliable even on localhost under some circumstances. The point of datagrams is that they are allowed to be lost without a trace if the consumer (syslog) is not consuming fast enough. For example, if process A starts spewing 10,000 log records (UPD or datagram UNIX socket packets) a second at syslog, and syslog can only handle 5,000…

While it's certainly true that UDP is unreliable even on localhost, unix datagram sockets arn't, they have flow control.

Re: Nginx 1.7.1 adds syslog support

#42

Post by stonogo »

peterwwillis wrote: post #34
Thaxll wrote: post #30

Finally... it worked well with logstash but a native implementation is always better. Is that TCP only, I don't see any UDP settings?

It's UDP only. They also hard-code the port instead of looking it up with getservbyname, which is a little weird. If you want to use TCP, send syslog messages to localhost, and have a local rsyslog/RELP forwarder send messages remotely.

getservbyname under glibc uses dlopen, effectively working around static compilation. while this is a non-issue on ubuntu, the nginx devs are aware of the number of people building against uclibc or musl to produce static binaries for embedded use. I've seen nginx running on bare metal -- just it and libc, no kernel underneath. kudos to them for making this easier than they have to.

Re: Nginx 1.7.1 adds syslog support

#44

Post by IgorPartola »

noselasd wrote: post #41
IgorPartola wrote: post #38

Earlier quoted context omitted.

While I agree with you on the whole, one minor point about UDP/datagrams: they are not reliable even on localhost under some circumstances. The point of datagrams is that they are allowed to be lost without a trace if the consumer (syslog) is not consuming fast enough. For example, if process A starts spewing 10,000 log records (UPD or datagram UNIX socket packets) a second at syslog, and syslog can only handle 5,000…

While it's certainly true that UDP is unreliable even on localhost, unix datagram sockets arn't, they have flow control.

I have never seen a reference to this. Can you link a source?

Re: Nginx 1.7.1 adds syslog support

#45

Post by lobster_johnson »

IgorPartola wrote: post #44
noselasd wrote: post #41

Earlier quoted context omitted.

While it's certainly true that UDP is unreliable even on localhost, unix datagram sockets arn't, they have flow control.

I have never seen a reference to this. Can you link a source?

AF_UNIX with SOCK_STREAM will behave like TCP, in that a "full" stream will block. See http://stackoverflow.com/questions/1478975/unix-domain-socke..., for example.
Post reply on HN