Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Clojure Concurrency Tutorial

purelyfunctional.tv

Post reply on HN
Print view View original

31–40 of 43 posts

Re: Clojure Concurrency Tutorial

#31

Post by setr »

quadcore wrote: post #10
tombert wrote: post #7

Earlier quoted context omitted.

To a reasonable extent, yes, they're the same. The biggest issue is that goroutines are (more or less) fully preemptive (for all intents and purpose) with their thread pooling and go blocks in core.async are not. Basically, in Go, you can have as many blocking IO thing taking millions of seconds to run, but it won't block the thread pool, and your other goroutines will run fine. Since core.async go blocks aren't full…

I'm a bit surprised if I understand you correctly. I though a goroutine would run and hold the thread until any kind of sleep. edit: Oh but there might be no "core.async sleep" in most of clojure libraries like IO/net stuff, which would explain why async go blocks hold the thread more often than they should, whereas Go has "goroutines sleeps" everywhere.

As I recall it, goroutines are in fact cooperative with implicit yield points. The implicit behavior is what I believe go means by “for all intents and purposes”

Particularly, I believe all function calls are implicitly yield, which means you can usually pretend that its preemptive through normal coding practices; but you can still never yield through a while(1){} (i think there was a proposal to add implicit yield on loops? Not sure if it went through)

I would assume core.async does not have many, if any, implicit yields, leading to the different behavior. But technically, both are cooperative, go is just more convenient about it

Re: Clojure Concurrency Tutorial

#32

Post by kitd »

chongli wrote: post #28
tombert wrote: post #7

Earlier quoted context omitted.

To a reasonable extent, yes, they're the same. The biggest issue is that goroutines are (more or less) fully preemptive (for all intents and purpose) with their thread pooling and go blocks in core.async are not. Basically, in Go, you can have as many blocking IO thing taking millions of seconds to run, but it won't block the thread pool, and your other goroutines will run fine. Since core.async go blocks aren't full…

What's the advantage of downloading multiple massive files in parallel, assuming you can saturate the link with just one? Isn't it better to finish one file before moving on to the next? If you can't saturate the link with one file, then you should consider something like bit torrent.

It's an example. Think of something else, eg an HTTP client call that sits waiting for a response, instead. The call parallelism is the issue being discussed.

Re: Clojure Concurrency Tutorial

#33

Post by Blackthorn »

jwr wrote: post #30
Blackthorn wrote: post #12

Earlier quoted context omitted.

core.async is a macro. Call an already compiled function under a library that blocks and the whole system will come to a screeching halt. Goroutines don't have that limitation.

Please see my other comment about using the same channels and calling I/O-heavy (blocking) functions from a `(thread)` instead of from a go block. You indeed shouldn't call blocking functions from go blocks, but core.async gives you the option of freely mixing separate threads and go blocks.

Having to use full threads in some cases and core.async in others pretty much defeats the purpose. Core.async is just nowhere near as good as goroutines. It'll continue to be like that until Project Loom finally has a release.

Re: Clojure Concurrency Tutorial

#34

Post by Blackthorn »

setr wrote: post #31
quadcore wrote: post #10

Earlier quoted context omitted.

I'm a bit surprised if I understand you correctly. I though a goroutine would run and hold the thread until any kind of sleep. edit: Oh but there might be no "core.async sleep" in most of clojure libraries like IO/net stuff, which would explain why async go blocks hold the thread more often than they should, whereas Go has "goroutines sleeps" everywhere.

As I recall it, goroutines are in fact cooperative with implicit yield points. The implicit behavior is what I believe go means by “for all intents and purposes” Particularly, I believe all function calls are implicitly yield, which means you can usually pretend that its preemptive through normal coding practices; but you can still never yield through a while(1){} (i think there was a proposal to add implicit yield o…

The real magic is where you can call blocking functions from goroutines and have it Just Work!

Re: Clojure Concurrency Tutorial

#35

Post by [deleted] »

tombert wrote: post #13
jakeinspace wrote: post #11

Earlier quoted context omitted.

Most Clojure libraries I've worked with, even the "larger" ones, are fairly small and concise by modern standards. I wonder if this has something to do with the lack of activity (in addition to being a small community). It feels a bit like the Unix philosophy - lots of small/medium libraries handling discrete problems, rather than many competing mega-frameworks. To me, this feels like a silver lining that comes from…

This is why I love functional programming as a whole. I feel like my code is really terse, without little/no sacrifices in expressivity, and usually no substantial sacrifice in performance.

[deleted]

Re: Clojure Concurrency Tutorial

#36

Post by tombert »

chongli wrote: post #28
tombert wrote: post #7

Earlier quoted context omitted.

To a reasonable extent, yes, they're the same. The biggest issue is that goroutines are (more or less) fully preemptive (for all intents and purpose) with their thread pooling and go blocks in core.async are not. Basically, in Go, you can have as many blocking IO thing taking millions of seconds to run, but it won't block the thread pool, and your other goroutines will run fine. Since core.async go blocks aren't full…

What's the advantage of downloading multiple massive files in parallel, assuming you can saturate the link with just one? Isn't it better to finish one file before moving on to the next? If you can't saturate the link with one file, then you should consider something like bit torrent.

I'm just saying that as a dumb example of something that isn't terribly CPU intensive, but would block up the threads, but even then, let's assume that the downloads are really slow, so that it's not anywhere near saturating my network speed. The go-blocks in core.async would still block the other go blocks. This wouldn't be an issue with Go.

Re: Clojure Concurrency Tutorial

#37

Post by tombert »

Blackthorn wrote: post #33
jwr wrote: post #30

Earlier quoted context omitted.

Please see my other comment about using the same channels and calling I/O-heavy (blocking) functions from a `(thread)` instead of from a go block. You indeed shouldn't call blocking functions from go blocks, but core.async gives you the option of freely mixing separate threads and go blocks.

Having to use full threads in some cases and core.async in others pretty much defeats the purpose. Core.async is just nowhere near as good as goroutines. It'll continue to be like that until Project Loom finally has a release.

Sorry, what is Project Loom? I'm not overly familiar with it and a quick search for `clojure project loom` seems to indicate that it's a graphing library.

Re: Clojure Concurrency Tutorial

#38

Post by quadcore »

setr wrote: post #31
quadcore wrote: post #10

Earlier quoted context omitted.

I'm a bit surprised if I understand you correctly. I though a goroutine would run and hold the thread until any kind of sleep. edit: Oh but there might be no "core.async sleep" in most of clojure libraries like IO/net stuff, which would explain why async go blocks hold the thread more often than they should, whereas Go has "goroutines sleeps" everywhere.

As I recall it, goroutines are in fact cooperative with implicit yield points. The implicit behavior is what I believe go means by “for all intents and purposes” Particularly, I believe all function calls are implicitly yield, which means you can usually pretend that its preemptive through normal coding practices; but you can still never yield through a while(1){} (i think there was a proposal to add implicit yield o…

The implicit yield points you're talking about is sleep or wait (noop in other terms). It's also not exactly a yield, it calls the scheduler to tell it to schedule (meaning the scheduler could decide to run again the exact same goroutine that noop-ed). Finally, goroutines are not just cooperative, they also run in parallel in a pool of threads.

Re: Clojure Concurrency Tutorial

#39

Post by quadcore »

chongli wrote: post #28
tombert wrote: post #7

Earlier quoted context omitted.

To a reasonable extent, yes, they're the same. The biggest issue is that goroutines are (more or less) fully preemptive (for all intents and purpose) with their thread pooling and go blocks in core.async are not. Basically, in Go, you can have as many blocking IO thing taking millions of seconds to run, but it won't block the thread pool, and your other goroutines will run fine. Since core.async go blocks aren't full…

What's the advantage of downloading multiple massive files in parallel, assuming you can saturate the link with just one? Isn't it better to finish one file before moving on to the next? If you can't saturate the link with one file, then you should consider something like bit torrent.

I/O is exactly the kind of stuff that should happen smoothly in goroutines or go blocks. It's kind of the quintessential case because I/O has a lot of sleeps and waits (noop or stalls in other terms).

When you're downloading a big file for example, I'm not an expert of that but I expect that there's gonna be a lot of cpu stalls in there. Same for the network adaptor and disk.

Re: Clojure Concurrency Tutorial

#40

Post by lackbeard »

tombert wrote: post #37
Blackthorn wrote: post #33

Earlier quoted context omitted.

Having to use full threads in some cases and core.async in others pretty much defeats the purpose. Core.async is just nowhere near as good as goroutines. It'll continue to be like that until Project Loom finally has a release.

Sorry, what is Project Loom? I'm not overly familiar with it and a quick search for `clojure project loom` seems to indicate that it's a graphing library.

https://wiki.openjdk.java.net/display/loom/Main
Post reply on HN