Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Erlang Matters

sameroom.io

Post reply on HN
Print view View original

21–30 of 210 posts

Re: Why Erlang Matters

#21

Post by pcwalton »

jerf wrote: post #6
jandrese wrote: post #2

I'm a very green Erlang noob, but given what I have seen from it I find articles like this kind of strange. Sure concurrent programming is difficult and we need to think hard about how to make programs run quickly in a multiprocessor environment, but the fundamental architecture of Erlang seems to be in conflict with big data and high speed computing. It seems like a language that can scale much better, but has such…

Erlang may be useful for coordinating computation tasks, but, yes, even with HIPE it is not a good numerical language on its own. It would be interest to re-engineer a language today that tries to fit into Erlang's niche but has a stronger performance focus. Rust, Cloud Haskell, and Go all sort of cluster in the area I'm thinking about, but none are quite what I'm thinking of. Cloud Haskell is probably closest but wr…

Agreed. It'd be really interesting to see such a language.

One language that I think is woefully underappreciated for how ubiquitous it is is GLSL. With OpenGL 4 Compute Shader you get surprisingly close to general-purpose use for the type of tasks that benefit from massive parallelism. And GLSL is really quite a nice language; driver bugs are the main things holding it back.

Re: Why Erlang Matters

#23

Post by vvanders »

pcwalton wrote: post #21
jerf wrote: post #6

Earlier quoted context omitted.

Erlang may be useful for coordinating computation tasks, but, yes, even with HIPE it is not a good numerical language on its own. It would be interest to re-engineer a language today that tries to fit into Erlang's niche but has a stronger performance focus. Rust, Cloud Haskell, and Go all sort of cluster in the area I'm thinking about, but none are quite what I'm thinking of. Cloud Haskell is probably closest but wr…

Agreed. It'd be really interesting to see such a language. One language that I think is woefully underappreciated for how ubiquitous it is is GLSL. With OpenGL 4 Compute Shader you get surprisingly close to general-purpose use for the type of tasks that benefit from massive parallelism. And GLSL is really quite a nice language; driver bugs are the main things holding it back.

Eeeeeeh. I don't know.

Maybe for highly SIMD stuff but it seems like pull in a bunch of baggage around GPUs, etc.

On the other hand it would force you to do your data partitioning right up-front(much like the SPUs on the PS3).

Re: Why Erlang Matters

#24

Post by aaron-lebo »

rb808 wrote: post #8

I'm really tempted to use Erlang/Elixir for a project at work but unsure of its traction. Is it leading edge or trailing edge? I don't even know, but don't want to saddle the firm with a white elephant - even one that is impeccably fault-tolerant. Is Erlang too esoteric?

I think it depends on your hiring process.

Do you hire people who know how to code in language X and are really good at coding in language X but nothing else?

Or, do you hire people who are go-getters, want to use the best tool for the job, and want to learn?

Because if the latter, Erlang/Elixir might be esoteric, but Elixir specifically is a simpler language than currently more popular languages like Python or Ruby. It's also more elegant. I like to think of it as Scheme + Ruby syntax + pattern matching + concurrency. If that excites you, carry on. If that scares you, go for something more traditional (and no hard feelings) :).

Re: Why Erlang Matters

#25

Post by jlouis »

jandrese wrote: post #2

I'm a very green Erlang noob, but given what I have seen from it I find articles like this kind of strange. Sure concurrent programming is difficult and we need to think hard about how to make programs run quickly in a multiprocessor environment, but the fundamental architecture of Erlang seems to be in conflict with big data and high speed computing. It seems like a language that can scale much better, but has such…

The interesting aspect of scaling up is that it doesn't matter how fast you are at individual single-core computation. Fast single core computation, or even SIMD GPU processing, is largely an "easy" problem: get a stream of data going, or get a chunk of data into the system, and work away on it.

What makes scaling up hard is moving data around. Once you have more than a single computer, there is no way you can easily share memory between them, so you have to impose some kind of copying for the system to work. If you want to demux a stream for multiple workers, you have to distribute work to the workers. If you have massive amounts of data in a cluster, you have to move the computation to the nodes in the cluster on which the data resides.

Moving data around requires you to have good orchestration of "mostly stateless" computations, with a couple of pinches of persistence strewn in as well. You can do this well in any language, but what makes Erlang well suited for it is that it provides some decent primitives for you with a lot of time sunk into the architecture. Beating this architecture in any other system requires you to spend some time doing that. And chances are it isn't as general, so when the world around you change, the framework you used is left behind.

Before Erlang, Tandem systems built hardware/software with many of the same ideas in them. They built these systems primarily for fault tolerance and robustness, but they found, somewhat to their surprise, the same architecture is good at scaling. The reason I believe, after 10 years of Erlang programming, is mostly that the computation model of isolated services forces you to think distribution into the system from day one. Your solution naturally gravitates toward the distributed model, and this in turn means it is easier to scale out later. The model also makes it hard to accidentally build a part of the system which can slow down everything. I think this should be given more credit than it is normally given.

And once you have your problem distributed, you call into that CUDA GPU code on the node to obtain the high computation speed. Or you call into your FPGA or DSP ASIC. Any problem on the CPU is slow because of its general purpose behavior (the exception: You are Fabrice Bellard)

Re: Why Erlang Matters

#26

Post by elihu »

jandrese wrote: post #2

I'm a very green Erlang noob, but given what I have seen from it I find articles like this kind of strange. Sure concurrent programming is difficult and we need to think hard about how to make programs run quickly in a multiprocessor environment, but the fundamental architecture of Erlang seems to be in conflict with big data and high speed computing. It seems like a language that can scale much better, but has such…

For years, I didn't see what the point of Erlang was. It might scale well, but if it takes an Erlang process running on a hundred processors to equal the performance of a single-threaded C application, what's the point?

When I learned a bit more about Erlang, I realized that the point is that there are a lot of applications that are IO bound rather than computation bound, and for those apps Erlang performs very well and is easy to program in. Crunching numbers the fastest isn't always the most important thing.

Re: Why Erlang Matters

#27

Post by zem »

jerf wrote: post #6
jandrese wrote: post #2

I'm a very green Erlang noob, but given what I have seen from it I find articles like this kind of strange. Sure concurrent programming is difficult and we need to think hard about how to make programs run quickly in a multiprocessor environment, but the fundamental architecture of Erlang seems to be in conflict with big data and high speed computing. It seems like a language that can scale much better, but has such…

Erlang may be useful for coordinating computation tasks, but, yes, even with HIPE it is not a good numerical language on its own. It would be interest to re-engineer a language today that tries to fit into Erlang's niche but has a stronger performance focus. Rust, Cloud Haskell, and Go all sort of cluster in the area I'm thinking about, but none are quite what I'm thinking of. Cloud Haskell is probably closest but wr…

pony [http://www.ponylang.org/] follows the actors-everywhere model, and has a strong performance focus. not sure how well it fits into erlang's niche, but at least the built-in actor model means a lot of erlang idioms and design patterns should port over easily.

Re: Why Erlang Matters

#29

Post by jlouis »

dboreham wrote: post #5

Doesn't the existence of Golang remove most of the reasons to use Erlang these days?

I was wondering this when Go came out. But time has shown they solve problems very differently, so they are not really direct competitors.

Erlangs primary difference over Go is that it can gracefully handle the case where a programmer has made an error in the program by accident. A typical Go program cannot gracefully recover from an error which were unforseen and never considered by the programmer. The Erlang equivalent Erlang program, however, has mechanisms to safely clean up resources for the faulty part and then get on solving work. This is the reason Erlang has a nice robustness story.

Re: Why Erlang Matters

#30

Post by ccozan »

SpacemanSpiff wrote: post #17
rb808 wrote: post #8

I'm really tempted to use Erlang/Elixir for a project at work but unsure of its traction. Is it leading edge or trailing edge? I don't even know, but don't want to saddle the firm with a white elephant - even one that is impeccably fault-tolerant. Is Erlang too esoteric?

Embedded systems programmer here. I'm learning Elixir and am very excited about the nerves project: http://nerves-project.org/ https://www.youtube.com/watch?v=kpzQrFC55q4 Edit: There also seems to be a good Elixir/Erlang community here in Berlin Germany where I live at the moment.

Do you have some link or contacts in/to this community? I am in process to develop a quite interesting application on Erlang and I am struggling to find a community (in Germany) where I can either ask questions or have a pool of possible people to hire.
Post reply on HN