Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Erlang Matters

sameroom.io

Post reply on HN
Print view View original

121–130 of 210 posts

Re: Why Erlang Matters

#121

Post by mistercow »

jeremyjh wrote: post #84
atemerev wrote: post #47

AFAIK, Erlang is still (as of 2016) the only distributed actor model implementation with preemptive scheduler. All other major implementations (including Akka) have cooperative scheduling, i.e. forbidding blocking code in actors. Erlang allows it. This is huge. And actor supervision is the best way to write reliable systems. I have wrote some code in Akka without much effort and testing (streaming market data aggrega…

There is one other. Haskell has green threads, a preemptive scheduler, and it has a pretty decent implementation of Erlang-inspired multi-node concurrency primitives and higher-level framework including supervisors, gen_server equivalent etc in the Cloud Haskell project. It does NOT have Erlangs deployment base and track record but it is still a very promising framework and very appealing if you like Haskell's type s…

I've been confused for some time as to why people get excited about green threads. From what I've read, the main advantage seems to be that you can have threads on hardware that doesn't support threads natively, which is cool if you're on that kind of hardware. There's also some spin-up advantages I guess? But they don't get load-balanced across cores, right?

I feel like I'm missing something important.

Re: Why Erlang Matters

#122

Post by DasIch »

incepted wrote: post #105
atemerev wrote: post #47

AFAIK, Erlang is still (as of 2016) the only distributed actor model implementation with preemptive scheduler. All other major implementations (including Akka) have cooperative scheduling, i.e. forbidding blocking code in actors. Erlang allows it. This is huge. And actor supervision is the best way to write reliable systems. I have wrote some code in Akka without much effort and testing (streaming market data aggrega…

> And actor supervision is the best way to write reliable systems. Need some citation there, because we've been writing extremely reliable systems (multiple nines) for decades in various languages (mostly C, C++ and Java). All these languages (including Erlang) come with pros and cons to write such systems, but so far, Erlang has failed to deliver on most of the promises that its advocates repeatedly make.

Erlang was developed for developing telecommunication systems and is still used heavily in that area today. When was the last time you heard of someone unable to make a phone call because a phone network went down?

Re: Why Erlang Matters

#123

Post by theseoafs »

fantasticsid wrote: post #119
rdtsc wrote: post #103

Earlier quoted context omitted.

> I'll warn you that it's very 80's hackeresque, but in a good way. Incredibly pragmatic. I consider the BEAM VM as one of the marvels of software engineering. You know it is good, when you explain to other programmers that you can have something like an isolated memory process just like an OS process, with preemption and only a few Ks of memory, with a low latency GC, with distribution across machines built in -- an…

I'm interested. How does the preemptive scheduling work with blocking system calls? So if an Erlang process tries to read from standard input using the read syscall (suppose we haven't set non blocking mode on the fd), why does it not block the scheduler that it runs on? Or does Erlang implements its own set of syscall wrappers that uses epoll under the hood?

You've got it: processes are not allowed to run blocking syscalls under the hood. Transparently that work is handled by backing IO threads.

Re: Why Erlang Matters

#124

Post by logicchains »

jeremyjh wrote: post #84
atemerev wrote: post #47

AFAIK, Erlang is still (as of 2016) the only distributed actor model implementation with preemptive scheduler. All other major implementations (including Akka) have cooperative scheduling, i.e. forbidding blocking code in actors. Erlang allows it. This is huge. And actor supervision is the best way to write reliable systems. I have wrote some code in Akka without much effort and testing (streaming market data aggrega…

There is one other. Haskell has green threads, a preemptive scheduler, and it has a pretty decent implementation of Erlang-inspired multi-node concurrency primitives and higher-level framework including supervisors, gen_server equivalent etc in the Cloud Haskell project. It does NOT have Erlangs deployment base and track record but it is still a very promising framework and very appealing if you like Haskell's type s…

Just a nitpick, but Haskell's scheduler (like Go's) is "less preemptive" than Erlang's. If you have e.g.

    i = 0
    for{
        i += 1
    }
or equivalent in Go/Haskell, then the HS thread/goroutine running it cannot be interrupted, as interruption can only take place at certain points (e.g. allocation or function calls).

Erlang on the other hand assigns a certain amount of time units to each process, and each pure-Erlang operation consumes these, so an Erlang process can always be interrupted as it will eventually use up this time allocation and yield (unless it's actually calling a C function etc).

Re: Why Erlang Matters

#125

Post by rdtsc »

bitmadness wrote: post #53

Erlang is glacially slow. Even on a 20 core machine, a multithreaded Erlang implementation will usually be trounced by a good singlethreaded C++/Go/Java implementation. All this stuff about multicore scaling is baloney - who cares it is scales and is still slow?

> multithreaded Erlang implementation will usually be trounced by a good singlethreaded C++/Go/Java implementation

And Go/Java implementation can be trounced by hand written assembly and ASIC accelerators probably.

> All this stuff about multicore scaling is baloney

You say baloney I say money in the pocket. I've seen it scale, I've seen it work reliably in large clusters, I've been able to inspect, debug and hotpatch running systems while they are still running. I have seen systems which had non-critical components crash and auto-restart for days without impacting customers and needed teams of "devops" to babysit it.

Moreover I've see single and multi-threaded C++ and Java applications with threads and and data races which take weeks or months to find. Or they are screaming fast until they take a nosedive and segfault (also in some minor stupid new feature which nobody uses). You know what the transaction processing rate of a segfaulted process is? - 0 tps.

That's why teams like Whatsapp could get by with only 10 or so back-end engineers handling billions of messages / day from various devices new and old, while other companies need 10x or even 20x more than that.

It is not just being able to run fast. Assembly runs very fast. It is also about being able to have the right tools and abstraction to define a problem. Erlang has those and they come built-in (the OTP library, the distribution protocol etc), C++ doesn't, so have to start from STL and boost and so on, then get serialization, monitoring, supervision, etc bootstrapped.

Re: Why Erlang Matters

#126

Post by logicchains »

mistercow wrote: post #121
jeremyjh wrote: post #84

Earlier quoted context omitted.

There is one other. Haskell has green threads, a preemptive scheduler, and it has a pretty decent implementation of Erlang-inspired multi-node concurrency primitives and higher-level framework including supervisors, gen_server equivalent etc in the Cloud Haskell project. It does NOT have Erlangs deployment base and track record but it is still a very promising framework and very appealing if you like Haskell's type s…

I've been confused for some time as to why people get excited about green threads. From what I've read, the main advantage seems to be that you can have threads on hardware that doesn't support threads natively, which is cool if you're on that kind of hardware. There's also some spin-up advantages I guess? But they don't get load-balanced across cores, right? I feel like I'm missing something important.

The advantage is you can have many more green threads than you could have OS threads (hundreds of thousands vs thousands), due to green threads being more lightweight. This allows a programming model based on message passing between green threads, which many people consider nicer to reason about. E.g. for a web server you could have a goroutine/Erlang process for each client with 50k clients and still have excellent performance, whereas if you had 50k OS threads you'd likely suffer performance issues and use a heap more ram.

Re: Why Erlang Matters

#127

Post by losvedir »

jerf wrote: post #91
adrusi wrote: post #81

Earlier quoted context omitted.

You can't really achieve Erlang's goals with a statically typed language, at least it would be very hard to make it easy to intuit whether a live reload will be sound. A language that allows mutable state aside from at process scope is also a no-go. In Erlang you're not supposed to think about what code will be running on which machines as you write the business logic, so you have to assume that every process is runn…

Live reload is a funny thing with Erlang. When I claim it's a feature, but describe the pain it is to use, I get told nearly nobody uses it. When I claim that nobody uses it, I get told that lots of people use it. I'm not sure it's something that an Erlang competitor would have to get right. And it would be valid to use a different mechanism for live reloads, perhaps something that explicitly migrates state between O…

> yet (almost) none of the dynamic languages go faster than "an order of magnitude slower than C with a huge memory penalty" even today, after a ton of effort has been poured into them

Interesting. What are the exceptions you have in mind to warrant the (almost) hypothetical? You mention LuaJIT. I've also heard that Q/KDB are quite fast. Anything else?

Re: Why Erlang Matters

#128

Post by incepted »

DasIch wrote: post #122
incepted wrote: post #105

Earlier quoted context omitted.

> And actor supervision is the best way to write reliable systems. Need some citation there, because we've been writing extremely reliable systems (multiple nines) for decades in various languages (mostly C, C++ and Java). All these languages (including Erlang) come with pros and cons to write such systems, but so far, Erlang has failed to deliver on most of the promises that its advocates repeatedly make.

Erlang was developed for developing telecommunication systems and is still used heavily in that area today. When was the last time you heard of someone unable to make a phone call because a phone network went down?

That would be a powerful argument if you could prove the code path enabling this uses Erlang.

According to Wikipedia, shortly after Armstrong was let go of Ericsson, the company quickly ripped out Erlang from all its products and replaced it with C and C++.

Re: Why Erlang Matters

#129

Post by incepted »

daveguy wrote: post #115
incepted wrote: post #105

Earlier quoted context omitted.

> And actor supervision is the best way to write reliable systems. Need some citation there, because we've been writing extremely reliable systems (multiple nines) for decades in various languages (mostly C, C++ and Java). All these languages (including Erlang) come with pros and cons to write such systems, but so far, Erlang has failed to deliver on most of the promises that its advocates repeatedly make.

> but so far, Erlang has failed to deliver on most of the promises that its advocates repeatedly make. Speaking of citations...

Well, Erlang is a niche programming language, isn't that evidence enough that all these claims that Erlang will save us from the multithreaded hell are vastly overblown?

We're doing multithreaded programming just fine in a lot of very varied languages.

Re: Why Erlang Matters

#130

Post by signa11 »

jlouis wrote: post #25
jandrese wrote: post #2

I'm a very green Erlang noob, but given what I have seen from it I find articles like this kind of strange. Sure concurrent programming is difficult and we need to think hard about how to make programs run quickly in a multiprocessor environment, but the fundamental architecture of Erlang seems to be in conflict with big data and high speed computing. It seems like a language that can scale much better, but has such…

The interesting aspect of scaling up is that it doesn't matter how fast you are at individual single-core computation. Fast single core computation, or even SIMD GPU processing, is largely an "easy" problem: get a stream of data going, or get a chunk of data into the system, and work away on it. What makes scaling up hard is moving data around. Once you have more than a single computer, there is no way you can easily…

> Before Erlang, Tandem systems built hardware/software with many of the same ideas in them.

Indeed, Jim-gray's (from tandem) paper 'why computers stop and what we can do about it' is an quite good. It contains a detailed report of machine failure including s/w and h/w and details techniques for reducing the mtbf by these.

Erlang's language and runtime seems to have picked seminal ideas from here...

Post reply on HN