Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fearless Concurrency: Clojure, Rust, Pony, Erlang and Dart

sites.google.com

Post reply on HN
Print view View original

121–130 of 145 posts

Re: Fearless Concurrency: Clojure, Rust, Pony, Erlang and Dart

#121

Post by lugg »

kccqzy wrote: post #115

I'd also like to mention cooperative concurrency. I know the author kind of skipped over this from the first sentence of the article where he defined a concurrent program as having more than one thread of execution. But with coroutines, we can have a single threaded-application that handles multiple tasks concurrently too. In fact I believe this is much easier to reason about for the programmer: the kernel won't rand…

Did you just say single threaded concurrently reasoned applications are more performant than multithreaded applications?

Trying to reason about how this could be possible. I'm still amazed by the performance you can get out of async non blocking style application code so don't take this the wrong way, truly trying to understand here.

I'm also a little confused about threads being rescheduled by the OS being treated like an inconvenience, to me it's a feature that prevents one application being too hoggish.

I can see your point that it's nice to have these labelled with await but I'm struggling to pinpoint a time I've asked myself what thread scheduling might mean for my code. Other than just assuming that another thread could be anywhere doing anything which I think is the correct approach no?

Locks btw, totally non issue with the right API/Lang support.

Re: Fearless Concurrency: Clojure, Rust, Pony, Erlang and Dart

#122

Post by lugg »

dmix wrote: post #71
macintux wrote: post #26

Earlier quoted context omitted.

That’s a big reason I’m a fan of Erlang: individually, its features are interesting, but collectively they form an amazing system. Greater than the sum of its parts.

Agreed, it's one thing to have good primitives like Go channels but Erlang/Elixir provide an entire system from which to build a concurrent/async application. Things like error handling, messaging, storage, and structuring your application well for concurrency are already basically built-in into the standard approach you take building Erlang/Elixir apps.

Whats a tldr for how Erlang handles error handling?

This is something that is usually glossed over / afterthought when it is kind of a big deal in real life programming.

Re: Fearless Concurrency: Clojure, Rust, Pony, Erlang and Dart

#123

Post by choochootrain »

lugg wrote: post #121
kccqzy wrote: post #115

I'd also like to mention cooperative concurrency. I know the author kind of skipped over this from the first sentence of the article where he defined a concurrent program as having more than one thread of execution. But with coroutines, we can have a single threaded-application that handles multiple tasks concurrently too. In fact I believe this is much easier to reason about for the programmer: the kernel won't rand…

Did you just say single threaded concurrently reasoned applications are more performant than multithreaded applications? Trying to reason about how this could be possible. I'm still amazed by the performance you can get out of async non blocking style application code so don't take this the wrong way, truly trying to understand here. I'm also a little confused about threads being rescheduled by the OS being treated l…

its a two-edged sword. if you can trust the application code to effectively schedule itself then it's good but you can also get bugs and perf issues where a synchronous chunk of code blocks progress on everything (see js blocking the main thread and causing the ui to freeze)

the pro is that your async code explicitly defines points where context switching is okay since you're blocking on something anyways. this could be good for perf if context switching in the middle of a synchronous operation is expensive.

the con is that your async code might not cede control often enough to allow other coroutines to make progress.

so yes, you can have something hogging the runtime but in the context of an application that you control as a whole this is something you can avoid/fix if necessary.

at the OS level this might not make sense because you have to assume that applications are adversarial and will try to hog cpu time...

Re: Fearless Concurrency: Clojure, Rust, Pony, Erlang and Dart

#124

Post by fogetti »

leshow wrote: post #53
fogetti wrote: post #43

I am not sure but I think Go channels were inspired by CSP (communicating sequential processes) which is not inherently unsafe though.

from the article: > Notice that this article does not include Go, a language that admittedly has an elegant concurrency solution as well (Go channels) because that solution is not actually thread-safe - it's not very hard to have race conditions in Go or corrupt state because Go does not enforce a separation of shareable and not-shareable mutable state.

So you didn't understand what I said? CSP is not unsafe. What is your point of quoting if you don't even understand what I said?

Re: Fearless Concurrency: Clojure, Rust, Pony, Erlang and Dart

#125

Post by macintux »

lugg wrote: post #122
dmix wrote: post #71

Earlier quoted context omitted.

Agreed, it's one thing to have good primitives like Go channels but Erlang/Elixir provide an entire system from which to build a concurrent/async application. Things like error handling, messaging, storage, and structuring your application well for concurrency are already basically built-in into the standard approach you take building Erlang/Elixir apps.

Whats a tldr for how Erlang handles error handling? This is something that is usually glossed over / afterthought when it is kind of a big deal in real life programming.

The language is built upon a few core concepts:

* Processes are extremely lightweight, orders of magnitude smaller than operating system processes or JVM threads.

* Exceptions should generally not be handled; instead, those lightweight processes are allowed to fail, and an external supervisor process will re-launch if appropriate.

* Assertions about the state of the data are effectively enabled on every line of code, so the processes crash early, before corrupting other parts of the system.

* Data is immutable, which plays a role in making those assertions happen.

You could probably strip one or two of those bullet points and/or add a couple of more, but I think that captures the highlights.

I'll toot my own horn. If you find that interesting, this is my favorite talk I've given about the above: https://youtu.be/E18shi1qIHU

Re: Fearless Concurrency: Clojure, Rust, Pony, Erlang and Dart

#126

Post by int_19h »

Animats wrote: post #92
felixgallo wrote: post #77

Earlier quoted context omitted.

Exactly. Because the recorded history shows those constructs are nearly impossible to use correctly.

Yes. Having spent too much time debugging code by people who thought they were clever enough not to need language safety, I want language-level protection. (Did the Python crowd ever fix the race condition in CPickle?[1] They were in denial about this about eight years ago when I reported it. Doing multiple CPickle operations in separate threads can crash CPython. If you search for "CPickle thread crash" you find man…

I don't see a denial in that bug report - they're just saying that they don't have an actionable repro, and don't know how to get one.

Re: Fearless Concurrency: Clojure, Rust, Pony, Erlang and Dart

#127

Post by yogthos »

jerf wrote: post #103
yogthos wrote: post #68

Earlier quoted context omitted.

Technically, STM requires that you're not updating your state via side effects. That's a different proposition from requiring your functions to be pure. For example, you could have a function with a print statement inside your STM transaction just fine.

If you want to go way down that rabbit hole, it's an active topic of conversation in the Haskell community right now how to break up IO into more granular pieces than what the IO type natively provides, where a function is either "pure" or it's a dirty rotten effects producer, and no middle ground in between. Although, even a stray print inside an STM transaction could actually do you a lot of damage, since "print" i…

That's the difference between Clojure and Haskell mindsets in a nutshell. Clojure approach is to have sane defaults and guide the programmer towards doing the right thing, but ultimately letting them do what they need to. Whether it makes sense to do something or not is context dependent in practice. Ultimately, the person writing the code understands their situation best, and the language shouldn't get in the way of them doing what they need to.

You could of course argue that by preventing the user from doing certain things you avoid some classes of errors. However, I will in turn argue that by forcing the user to write code for the benefit of the type checker often results in convoluted solutions that are hard to understand and maintain. So, you just end up trading one set of problems for another.

Re: Fearless Concurrency: Clojure, Rust, Pony, Erlang and Dart

#128

Post by kccqzy »

lugg wrote: post #121
kccqzy wrote: post #115

I'd also like to mention cooperative concurrency. I know the author kind of skipped over this from the first sentence of the article where he defined a concurrent program as having more than one thread of execution. But with coroutines, we can have a single threaded-application that handles multiple tasks concurrently too. In fact I believe this is much easier to reason about for the programmer: the kernel won't rand…

Did you just say single threaded concurrently reasoned applications are more performant than multithreaded applications? Trying to reason about how this could be possible. I'm still amazed by the performance you can get out of async non blocking style application code so don't take this the wrong way, truly trying to understand here. I'm also a little confused about threads being rescheduled by the OS being treated l…

No I didn't say "single threaded concurrently reasoned applications are more performant than multithreaded applications" because I was highlighting this style of concurrency from the programmer's perspective. But yes, for certain types of applications it can be more performant.

Naturally, if your application is computationally intensive, a single thread can't compete with a multithreaded application. But for applications that use a lot of slow, blocking I/O, converting them to a single-threaded application that uses non-blocking I/O is a significant reduction in overhead. Compare a traditional web server that uses one thread per request and blocking I/O, versus one that uses an event loop, non-blocking I/O. You will see why the latter is more efficiently on system resources. Again this isn't a panacea, and for some applications you do have to use threads. I'm just pointing out an omission of the article.

Re: Fearless Concurrency: Clojure, Rust, Pony, Erlang and Dart

#129

Post by lastofus »

tombert wrote: post #116
lastofus wrote: post #95

Earlier quoted context omitted.

I believe all I/O is blocking unless you use something that wraps NIO. This is definitely true for core.async threads, but I'm not sure why it would any different for agents. This is also a bit of a departure for people coming from Go or Erlang/Elixir.

Agents are run in a separate pool, and no one agent is processing at a time, so I suppose they're not "parallel", but concurrent still. I think the agents are preemptive, but I'm not 100% sure on that.

I found this comment enlightening: https://clojuredocs.org/clojure.core/send-off#example-593472...

It seems like there are a lot of parallels between agents and core.async.

Agents, like go blocks, run in a thread pool. Any blocking IO should be pushed off to a separate thread outside of the thread pool. In the case of agents, there is (send-off). In the case of core.async, there is (thread)

Re: Fearless Concurrency: Clojure, Rust, Pony, Erlang and Dart

#130

Post by daotoad »

It's too bad Perl6's interesting approaches to concurrency aren't mentioned here. Supporting concurrency and parallelism were key points in the design of Perl6, and in the multi-paradigmatic way of Perl, it provides a variety of tools.

Jonathan Worthington can write and speak about this topic far better than I can so I refer you to his talk and slides:

1. http://www.jnthn.net/papers/2018-conc-par-8-ways.pdf 2.C https://www.youtube.com/watch?v=l2fSbOPeSQs

Post reply on HN