Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Websocketd

websocketd.com

Post reply on HN
Print view View original

131–140 of 233 posts

Re: Websocketd

#131

Post by arendtio »

masklinn wrote: post #129
arendtio wrote: post #127

Earlier quoted context omitted.

Interesting. My interpretation was more along the lines of having its own process identifier (PID) and to my knowledge, OS processes have that, OS threads don't and I thought Erlangs processes don't get PIDs by the operating system either (although that might be wrong too).

> OS processes have that, OS threads don't At the kernel level, both "processes" and "threads" have PIDs since they're the same thing. In fact, before NTPL was implemented Linux broke posix because it returned the kernel PID directly when queried[0]. That's no more than an interface choice of POSIX though. And POSIX threads still have a thread id, a (pid, tid) is a unique visible identifier. So the distinction seems…

Thanks for clarifying.

Re: Websocketd

#132

Post by jondubois »

inglor wrote: post #125
geocar wrote: post #115

Earlier quoted context omitted.

The myth that fork is expensive is pervasive, and speaking to the ways that it is true†, well: performance is relative. fork() only takes around 8ms on my Linux machine and I can get 100,000 posix_spawn() per second there with 100MB RSS. That's "fast enough" for a large number of applications. †: fork() is a lot slower (over 20x) on Windows

8ms per fork means you can only accept 125 connections per second. (per core) That means it's only viable for connections where the connection is very long lived and messages are very sparse (because context switches).

I was going to mention CPU context switching as well, it's very expensive even when you don't account for the initial fork overhead but I guess it's definitely a better idea than CGI for HTTP given the long lived connection.

In an ideal scenario, you shouldn't have more active processes than you have CPU cores. As soon as that happens, context switching kicks in and performance degrades sharply.

Re: Websocketd

#133

Post by mig_ »

geocar wrote: post #115
jchw wrote: post #26

This is the second or third "it's CGI again" thing I've seen in the past year. While these things are cool and definitely have their place, it's still worth noting that process per connection scales fairly poorly, simply because processes and forking are relatively expensive, and therefore it's probably unwise to deploy something like this in production anymore. It is what it is, I suppose.

The myth that fork is expensive is pervasive, and speaking to the ways that it is true†, well: performance is relative. fork() only takes around 8ms on my Linux machine and I can get 100,000 posix_spawn() per second there with 100MB RSS. That's "fast enough" for a large number of applications. †: fork() is a lot slower (over 20x) on Windows

The initial fork is fast because of the copy on write memory semantic. You'll pay a price later.

Re: Websocketd

#134

Post by kingosticks »

hnarn wrote: post #103
mcpherrinm wrote: post #14

Earlier quoted context omitted.

Is unix defined as having small binaries? I don't think that's true. Yes, it's statically linked and as a result somewhat large. This is a program built around executing programs and communicating over pipes. It's composing programs together. I think that's certainly UNIX-ish.

> Is unix defined as having small binaries? I don't think that's true. Yes, it's statically linked and as a result somewhat large. No, but literally the first point is: > Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new features. It's not hard to see how this would apply to libraries, and how including third party code in your binary would break this…

It then goes on to list a whole load of feature creep so the claim of doing one thing is a bit ridiculous.

Re: Websocketd

#135

Post by sebazzz »

chrisseaton wrote: post #52
symisc_devel wrote: post #37

Earlier quoted context omitted.

Not that much on modern Linux/BSD systems at least. The kernel is smart enough to just copy fewer pages when forking not the entire parent process image.

Then you run a GC and dirty all your pages.

GC usually runs on a separate thread right? Isn't it true you shouldn't run fork() when you have a multithreaded program?

Re: Websocketd

#136

Post by function_seven »

masklinn wrote: post #123
function_seven wrote: post #109

Earlier quoted context omitted.

I don't understand why the daemon needs to client to escape newlines in the message. Can't it handle that before feeding the message to STDIN? For example, my browser sends: This is line one... ...and this is line two as a string ("This is line one...\n...and this is line two") and the websocketd receives that, and passes along This is line one...\\n...andthis is line two\n to the STDIN. It knows that the message is…

> It'd also be nice to use EOF as the indicator to flush STDOUT to the client, so the program can also emit newlines without needing to escape them first. EOF is not an actual control code though (and is implementation dependent). However there are control codes for message framing e.g. ETB, ETX, EOT (often used as terminal EOF), FF (form feed / page break), RS/GS/FS

> EOF is not an actual control code though

You're right. I think I have to relearn that periodically :)

Re: Websocketd

#137

Post by chrisseaton »

sebazzz wrote: post #135
chrisseaton wrote: post #52

Earlier quoted context omitted.

Then you run a GC and dirty all your pages.

GC usually runs on a separate thread right? Isn't it true you shouldn't run fork() when you have a multithreaded program?

Not usually. Only an advanced GC will be concurrent. Something like Perl or Ruby for example runs in the foreground. If it is concurrent then you shut down the GC thread, fork, and then start a new thread in each new process.

Re: Websocketd

#138

Post by mpweiher »

Matthias247 wrote: post #39

Doesn’t this have an impedance mismatch? Stdin/out are stream based. Websocket is message based. There is no guarantee you can transmit the content of a single WebSocket message inside a single os read or write call. Unless you expect that on both sides messages might be fragmented across multiple calls and callbacks. But I don’t see the docs mentioning that.

> Stdin/out are stream based. Websocket is message based.

WebSocket is implemented on top of TCP/IP, which is stream based.

Re: Websocketd

#140

Post by perlgeek »

hnarn wrote: post #100
rileytg wrote: post #28

i’ve been using this in production to stream logs to a web console (~2,000 sets of logs distributed across 4 server with about 350 active connections at a time) and have never had any issues

How do you know that you haven't had any issues?

Usually you notice issues by the server locking up, the clients reporting problems, or messages being missing from the log (which you tend to notice when you search for specific things in the logs).
Post reply on HN