Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Erlang Matters

sameroom.io

Post reply on HN
Print view View original

71–80 of 210 posts

Re: Why Erlang Matters

#71

Post by devbug »

atemerev wrote: post #47

AFAIK, Erlang is still (as of 2016) the only distributed actor model implementation with preemptive scheduler. All other major implementations (including Akka) have cooperative scheduling, i.e. forbidding blocking code in actors. Erlang allows it. This is huge. And actor supervision is the best way to write reliable systems. I have wrote some code in Akka without much effort and testing (streaming market data aggrega…

This unfortunately breaks down with (bad acting) NIFs. Thankfully you can mark 'em as the dirty evil little things that they are (with negligible overhead): ERL_NIF_DIRTY_JOB_CPU_BOUND. [1]

I implore anyone interested in Erlang or its surrounding languages, to read its source code. [2] More specifically, the BEAM. I'll warn you that it's very 80's hackeresque, but in a good way. Incredibly pragmatic. The way they achieve their world-class scheduling? Tip: not via a perfect theoretical design cooked up in some comp-sci lab. [3] If you've been following game engine design within the last 9 years or so, you'll probably have a good idea of how they do it. [4][5]

[1]: https://medium.com/@jlouis666/erlang-dirty-scheduler-overhea...

[2]: https://github.com/erlang/otp/tree/maint/erts/emulator/beam

[3]: https://hamidreza-s.github.io/erlang/scheduling/real-time/pr...

[4]: http://blog.molecular-matters.com/2012/04/05/building-a-load...

[5]: There's a hell of a lot of depth to this problem domain. Charles Bloom's guiding principles for Oodle are spot, and well worth a look.

Re: Why Erlang Matters

#72

Post by jlouis »

andy_ppp wrote: post #60
jlouis wrote: post #29

Earlier quoted context omitted.

I was wondering this when Go came out. But time has shown they solve problems very differently, so they are not really direct competitors. Erlangs primary difference over Go is that it can gracefully handle the case where a programmer has made an error in the program by accident. A typical Go program cannot gracefully recover from an error which were unforseen and never considered by the programmer. The Erlang equiva…

https://github.com/thejerf/suture Supervisors for golang. It looks excellent. I personally have fallen for Elixir and think the syntax, immutability, community and class functionality like Phoenix.Presence make me bet that it'll be bigger than Python for jobs in 5 years.

Supervisors is only half the game here. Suppose A sends a message to B and decides to wait around for the answer. B now divides by zero.

In Go, your whole program is in consistency trouble and you have to write code to handle the case.

In Erlang, your monitor on B means you get told it is dead via an async exception delivered into your mailbox.

I know what kind of system I want to work with here :)

Re: Why Erlang Matters

#73

Post by davidw »

devbug wrote: post #71
atemerev wrote: post #47

AFAIK, Erlang is still (as of 2016) the only distributed actor model implementation with preemptive scheduler. All other major implementations (including Akka) have cooperative scheduling, i.e. forbidding blocking code in actors. Erlang allows it. This is huge. And actor supervision is the best way to write reliable systems. I have wrote some code in Akka without much effort and testing (streaming market data aggrega…

This unfortunately breaks down with (bad acting) NIFs. Thankfully you can mark 'em as the dirty evil little things that they are (with negligible overhead): ERL_NIF_DIRTY_JOB_CPU_BOUND. [1] I implore anyone interested in Erlang or its surrounding languages, to read its source code. [2] More specifically, the BEAM. I'll warn you that it's very 80's hackeresque, but in a good way. Incredibly pragmatic. The way they ach…

> This unfortunately breaks down with (bad acting) NIFs.

Where possible, I think the Erlangy thing to do would be to just have a separate executable and communicate with that. Also ensures that things keep running if there's a segfault or something.

Re: Why Erlang Matters

#74

Post by ssmoot »

vvanders wrote: post #40
ssmoot wrote: post #35

This seems like it could've been called "Why Actor Systems Matter". If you're on the JVM, I'm not sure what Erlang buys you in practice. It's slower and more obscure. It has a much smaller ecosystem. While process-safety is frequently touted, in the real world this is a non-issue among non-issues. It's just not an actual thing. It's not like the JVM goes around Segfaulting all the time. I'm totally sold on Actor Syst…

My Stop-The-World GCs would like to have a word with you.

And maybe in a project with truly massive memory requirements that's an issue. I've never encountered it (as an issue that is) But I only got into the JVM on 1.6 with Jruby, made the switch to Scala around 1.7, and have been on 1.8 pretty much since release.

A 3GB heap is pretty typical, but that's mostly just room for cache.

If you're writing web-apps, there's a ridiculous amount of opportunity for using Actors to good effect.

On the other hand I try to keep an eye on resources. I try not to throw millions of messages at an Actor. If I'm batch processing, I'm probably work-pulling, mapping and prefetching my Sources to attempt to minimize latency. Even if I could tune mailbox sizes to avoid all that I prefer not to code in upper bounds on input.

I would be surprised to hear this is a widespread problem in people's work. You don't often hear the same complaints from Rubyists for example and they're much worse off in that regard. So I feel like this case is massively overstated.

But if you have that problem I can certainly see how that might figure into your equation.

Re: Why Erlang Matters

#75

Post by ssmoot »

Xixi wrote: post #50
ssmoot wrote: post #35

This seems like it could've been called "Why Actor Systems Matter". If you're on the JVM, I'm not sure what Erlang buys you in practice. It's slower and more obscure. It has a much smaller ecosystem. While process-safety is frequently touted, in the real world this is a non-issue among non-issues. It's just not an actual thing. It's not like the JVM goes around Segfaulting all the time. I'm totally sold on Actor Syst…

It's not at all about segfaults. Assuming the Erlang VM is as likely/unlikely to crash as the JVM, then it's a complete wash: Erlang processes are green-threads, not OS processes, so you don't gain anything there. It's about the share-nothing architecture of Erlang: it means your processes are isolated and can be stopped/restarted independently, crash/hang without writing into the memory of another process, have thei…

Akka checks most of those boxes AFAIK. And there's really not much you can do in the way of cheating unless I misunderstand you. Or at least idiomatically you wouldn't cheat in Scala anyways.

Speaking of which, I just realized the AtomicLong I'm using in my IdGenerationActor (performs an atomic increment of a processId counter in the database, then uses that with the AtomicLong as the input to Hashids; fast, in-process, cluster-safe short-Id generation that will leave popular Redis based solutions in the dust) is completely unnecessary. Copied and pasted from non-Actor code without consideration.

I guess Actors still can't keep you from doing dumb things yet. ;-)

Re: Why Erlang Matters

#76

Post by eddd »

To me the power of Erlang comes from a combination of powerful VM and OTP. In the era of hype for SOA, OTP

You can run multiple apps on the same VM and they will run concurrently and communicate with each other using protocols that are core of the language.

I agree, erlang seems weird at first, simply because there is not anything like it. It also solved todays problems with software a decade ago.

Re: Why Erlang Matters

#77

Post by arh68 »

wcummings wrote: post #37
jerf wrote: post #6

Earlier quoted context omitted.

Erlang may be useful for coordinating computation tasks, but, yes, even with HIPE it is not a good numerical language on its own. It would be interest to re-engineer a language today that tries to fit into Erlang's niche but has a stronger performance focus. Rust, Cloud Haskell, and Go all sort of cluster in the area I'm thinking about, but none are quite what I'm thinking of. Cloud Haskell is probably closest but wr…

You could write NIFs in Rust (well not sure if you can now, but I don't see any reason it couldn't be supported) for the high perf bits and use Erlang to coordinate, I figure. At least Rust code is less likely to explode and bring down the whole VM than C.

Any idea if a network-level FFI has been started? I'm thinking along the lines of the Haskell erlang-ffi [0].

    Speaks the Erlang network protocol and impersonates 
    an Erlang node on the network. Fully capable of 
    bi-directional communication with Erlang.
NIFs still limit you to [0] https://hackage.haskell.org/package/erlang

Re: Why Erlang Matters

#78

Post by vvanders »

ssmoot wrote: post #74
vvanders wrote: post #40

Earlier quoted context omitted.

My Stop-The-World GCs would like to have a word with you.

And maybe in a project with truly massive memory requirements that's an issue. I've never encountered it (as an issue that is) But I only got into the JVM on 1.6 with Jruby, made the switch to Scala around 1.7, and have been on 1.8 pretty much since release. A 3GB heap is pretty typical, but that's mostly just room for cache. If you're writing web-apps, there's a ridiculous amount of opportunity for using Actors to g…

Not everything is web-apps either. If you've got soft realtime requirements(Erlang's GC model is just incredibly elegant. Almost like pooled allocators that we used to use in GameDev but in a completely transparent and intuitive way.

Re: Why Erlang Matters

#79

Post by devbug »

davidw wrote: post #73
devbug wrote: post #71

Earlier quoted context omitted.

This unfortunately breaks down with (bad acting) NIFs. Thankfully you can mark 'em as the dirty evil little things that they are (with negligible overhead): ERL_NIF_DIRTY_JOB_CPU_BOUND. [1] I implore anyone interested in Erlang or its surrounding languages, to read its source code. [2] More specifically, the BEAM. I'll warn you that it's very 80's hackeresque, but in a good way. Incredibly pragmatic. The way they ach…

> This unfortunately breaks down with (bad acting) NIFs. Where possible, I think the Erlangy thing to do would be to just have a separate executable and communicate with that. Also ensures that things keep running if there's a segfault or something.

Exactly! Well, kinda.

You can always throw the NIFs onto other nodes. You can also write ports that are glorified external processes.

Other times, you just need low-latency and high throughput. In that case, you expect failure and design accordingly.

Re: Why Erlang Matters

#80

Post by jallmann »

adrusi wrote: post #63
jallmann wrote: post #58

Earlier quoted context omitted.

> the fundamental architecture of Erlang seems to be in conflict with big data and high speed computing Because they are different problems. Concurrency, meet parallelism. Concurrency: many smaller tasks that can be multiplexed over one core. The core doesn't need to be particularly fast; it just needs to be able to handle multiple tasks in-flight at once. Web serving, etc. Parallelism: one big, honking task that can…

I know you weren't aiming for maximum accuracy in your definitions, but I think this is worth bringing up. Parallelism and Concurrency are not mutually exclusive terms. All parallelism is concurrent. Concurrency is whenever more than one thing is happening at the same time conceptually . Everything from iterators to threads. Parallelism is whenever more than one thing is happening at the same time physically . From a…

Yes. The intuition is a good one of parallelism being a deliberate way of designing a system to run its parts simultaneously (physically), with concurrency being a property of a system that may or may not have its parts running simultaneously (conceptually, 'overlapping', whether physically or logically).

There are many, many ways to think about this difference, and it's fun (and beneficial!) to do so every once in a while.

> From a user-space perspective, this means threads

Interestingly enough, before SMP and multicore, thread-level "parallelism" was actually disguised by a time-sharing concurrent implementation. That remains true for most threading libraries in languages with a GIL, and any time you have threads exceeding the number of physical cores. In fact, the primary purpose of threads was (and still mostly is) to get a semblance of concurrency... Even the Erlang implementation was single-threaded 2008 or so.

Post reply on HN