Using core.async and transducers to upload files from the browser to S3

#1

Post by mklappstuhl » Thu, Sep 04, 2014, 2:12 PM UTC

Using core.async and transducers to upload files from the browser to S3
martinklepsch.org

Re: Using core.async and transducers to upload files from the browser to S3

#2

Post by danneu » Thu, Sep 04, 2014, 5:03 PM UTC

    > I still need to look into some things like
    > how to properly shut down the `go` blocks.

I have limited core.async experience, but one simple solution would be to replace `(go (while true ... (For more control, you can pass a poison channel in from the outside. In the go-loop, `alts!` from `[report-chan, poison-chan, (timeout 10000)]`. Like the timeout channel, don't recur if the value comes from poison-chan. And that's a chan that your exterior code will have access to.

Re: Using core.async and transducers to upload files from the browser to S3

#3

Post by freshhawk » Thu, Sep 04, 2014, 10:24 PM UTC

I'm pretty sure that the alts! block is absolutely necessary otherwise you will deadlock after 10 files (your channel buffer size) because nothing else is taking from upload-chan or signing-chan.

Wouldn't a pipeline be a better solution to this? Taking and throwing away the values from a channel just to trigger a transducer that puts the value in another channel as a side effect is a strange way to do things.

Also a simple println test will show that the transducer function is triggered when you put on the channel rather than when you take.