Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Color Is Your Function? (2015)

journal.stuffwithstuff.com

Post reply on HN
Print view View original

81–90 of 90 posts

Re: What Color Is Your Function? (2015)

#81

Post by philwelch »

tsimionescu wrote: post #77
philwelch wrote: post #68

Earlier quoted context omitted.

> There are exactly the same reasons in Go to want asynchronous calls Which isn’t the comparison I’m replying to here. Calling an asynchronous function with “await” forces it to behave synchronously. You would only do such a thing if you were operating in a framework or language with colored functions. > in C# or Java or even C++ you can chose between using that OR asynchronous code with futures. Java, C#, and C++ ha…

> Calling an asynchronous function with “await” forces it to behave synchronously. You would only do such a thing if you were operating in a framework or language with colored functions. I don't really think you are right. When you await an async function, you let the async function run asynchronously, but suspend your own execution until you can receive the result from that function (an actual return value, an excep…

> Calling a function directly forces it to run on the same execution thread as you. Calling it with await allows it to run in any thread. This is the actual advantage, and Go doesn't have any equivalent construct that is as convenient for this use case.

OK so Thread A calls “await” on a coroutine that executes in Thread B (where B may or may not be A). Thread A is now blocked on that coroutine. What have I gained by running that coroutine in Thread B?

One potential answer is that while Thread A is blocked by “await”, it can context-switch to a different coroutine. You can effectively do similar things in Go if you want to. But doing so abandons the guarantee that Thread A will pick up where it left off as soon as Thread B is finished.

> Also, note that a function that expects to return data through channels can't be called in a sync manner in Go or it will deadlock. So in essence there is function coloring in Go as well.

Is this a popular or idiomatic interface for Go library code to the same degree it is for “async” libraries in other languages?

In isolation I find it more understandable to do channel writes as an explicit side effect than to manage futures but maybe that’s just my brain.

> Not out of the box, but they are easy to replicate if desired

I’m pretty sure you could implement futures and async/await using Go channels too if you wanted to.

Re: What Color Is Your Function? (2015)

#82

Post by tsimionescu »

philwelch wrote: post #81
tsimionescu wrote: post #77

Earlier quoted context omitted.

> Calling an asynchronous function with “await” forces it to behave synchronously. You would only do such a thing if you were operating in a framework or language with colored functions. I don't really think you are right. When you await an async function, you let the async function run asynchronously, but suspend your own execution until you can receive the result from that function (an actual return value, an excep…

> Calling a function directly forces it to run on the same execution thread as you. Calling it with await allows it to run in any thread. This is the actual advantage, and Go doesn't have any equivalent construct that is as convenient for this use case. OK so Thread A calls “await” on a coroutine that executes in Thread B (where B may or may not be A). Thread A is now blocked on that coroutine. What have I gained by…

> OK so Thread A calls “await” on a coroutine that executes in Thread B (where B may or may not be A). Thread A is now blocked on that coroutine. What have I gained by running that coroutine in Thread B?

If the task that bar() will return is created when you first call it, then you're right, we didn't gain much. However, the task may have already been running for a long time behind the scenes, we may have done things in parallel with that run, and now that we need the result, we can block.

For example:

    myHttpClient.StartReq1()
    myHttpClient.StartReq2()
    auto Res1 = await myHttpClient.WaitReq1()
    auto res2 = await myHttpClient.WaitReq2()
> I’m pretty sure you could implement futures and async/await using Go channels too if you wanted to.

Given the lack of generics, you would get a much worse interface. Btw, here is what a non-buffering channel would look like in Java:

    class Channel {
      T value;

      void publish(T value) {
        synchronized(this) {
           this.value = value;
           try {
             this.wait();
           } catch (InterruptedException e) {} 
        }
      }

      T consume() {
        synchronized(this) {
          this.notify();
          return this.value;
        }
      }
     }

Re: What Color Is Your Function? (2015)

#83

Post by simiones »

philwelch wrote: post #81
tsimionescu wrote: post #77

Earlier quoted context omitted.

> Calling an asynchronous function with “await” forces it to behave synchronously. You would only do such a thing if you were operating in a framework or language with colored functions. I don't really think you are right. When you await an async function, you let the async function run asynchronously, but suspend your own execution until you can receive the result from that function (an actual return value, an excep…

> Calling a function directly forces it to run on the same execution thread as you. Calling it with await allows it to run in any thread. This is the actual advantage, and Go doesn't have any equivalent construct that is as convenient for this use case. OK so Thread A calls “await” on a coroutine that executes in Thread B (where B may or may not be A). Thread A is now blocked on that coroutine. What have I gained by…

> I’m pretty sure you could implement futures and async/await using Go channels too if you wanted to.

Futures, maybe (though without generics you'll be either very dynamic or write a new future for each struct, of course).

But async/await is a syntactic feature and can't be implemented in a language without macros and/or continuations. Basically `await` is a keyword which returns a future that will execute the rest of the function as written. Something like this is relatively easy to implement:

    async Task Foo() { 
       int i := await Bar();
       return i + 1;
    }
You could re-write it to something like this in Go:

    func Foo() func()(int){
         var reply chan int
         go func() {
             reply
Or something similar. Maybe you could even reduce the boilerplate, though it's already much worse than the C# verison. But this is much more difficult to re-write in Go:

    async Task Foo() {
        auto sum = 0;
        foreach (auto x in someEnumerable) {
               switch(x) {
                     case 1:
                          sum += await Bar(x);
                          goto case 2;
                     case 2:
                          sum /= await Bar(x);
                          break;
                   }
        }
        return sum;
    }
Assuming you want to keep the asynchronicity, this gets much uglier to implement in terms of channels (not that this is very common code).

Re: What Color Is Your Function? (2015)

#84

Post by philwelch »

tsimionescu wrote: post #82
philwelch wrote: post #81

Earlier quoted context omitted.

> Calling a function directly forces it to run on the same execution thread as you. Calling it with await allows it to run in any thread. This is the actual advantage, and Go doesn't have any equivalent construct that is as convenient for this use case. OK so Thread A calls “await” on a coroutine that executes in Thread B (where B may or may not be A). Thread A is now blocked on that coroutine. What have I gained by…

> OK so Thread A calls “await” on a coroutine that executes in Thread B (where B may or may not be A). Thread A is now blocked on that coroutine. What have I gained by running that coroutine in Thread B? If the task that bar() will return is created when you first call it, then you're right, we didn't gain much. However, the task may have already been running for a long time behind the scenes, we may have done things…

> If the task that bar() will return is created when you first call it, then you're right, we didn't gain much. However, the task may have already been running for a long time behind the scenes, we may have done things in parallel with that run, and now that we need the result, we can block.

So in Go that’s just a blocking channel receive.

Re: What Color Is Your Function? (2015)

#85

Post by philwelch »

simiones wrote: post #83
philwelch wrote: post #81

Earlier quoted context omitted.

> Calling a function directly forces it to run on the same execution thread as you. Calling it with await allows it to run in any thread. This is the actual advantage, and Go doesn't have any equivalent construct that is as convenient for this use case. OK so Thread A calls “await” on a coroutine that executes in Thread B (where B may or may not be A). Thread A is now blocked on that coroutine. What have I gained by…

> I’m pretty sure you could implement futures and async/await using Go channels too if you wanted to. Futures, maybe (though without generics you'll be either very dynamic or write a new future for each struct, of course). But async/await is a syntactic feature and can't be implemented in a language without macros and/or continuations. Basically `await` is a keyword which returns a future that will execute the rest o…

I don’t really want to implement async/await in terms of channels. I would rather just use channels directly. Notice that even in your Go code it’s explicit which part executes as a separate coroutine and where the blocking communication between coroutines takes place. Async/await is more “magical”, or maybe I’m just dumb.

Re: What Color Is Your Function? (2015)

#86

Post by jayd16 »

philwelch wrote: post #84
tsimionescu wrote: post #82

Earlier quoted context omitted.

> OK so Thread A calls “await” on a coroutine that executes in Thread B (where B may or may not be A). Thread A is now blocked on that coroutine. What have I gained by running that coroutine in Thread B? If the task that bar() will return is created when you first call it, then you're right, we didn't gain much. However, the task may have already been running for a long time behind the scenes, we may have done things…

> If the task that bar() will return is created when you first call it, then you're right, we didn't gain much. However, the task may have already been running for a long time behind the scenes, we may have done things in parallel with that run, and now that we need the result, we can block. So in Go that’s just a blocking channel receive.

Not exactly. Async allows for N tasks of linear code execution on the same thread at the cost of managing cooperative yields.

Go and channels allow you to block many threads with little overhead and little syntax but at the cost of not being able to easily target a single thread without manual orchestration through channels.

Consider a scatter/gather algorithm. N async tasks, N coroutines. Start N from the main thread and block until all complete. Simple right?

Its not!

Imagine there are GPU commands that must be grouped or even logs where you want some of the subroutine's logs to be grouped.

With async/await you have the verbosity to synchronize tasks to thread contexts such that you know chunks will not be run in parallel. Its easy to control when execution leaves the current thread. It can ensure that it both does or does not yield execution. You can easily switch between synchronized main thread execution and parallel chunks without any top down design.

With goroutines you most likely would write the ordered results to a channel that was passed around. I'm not sure if channels support N inserts in an atomic fashion out of the box but if not it must be a channel of arrays or maybe some kind of control channel as well. Hopefully you have access to every piece of code you need to synchronize. This all assumes you can get away with a single main goroutine. If you need a single special OS thread for interop I'm sure its more complex. Its not just a blocking channel.

Essentially the two paradigms can do everything but they both seem to excel in certain cases.

Re: What Color Is Your Function? (2015)

#87

Post by simiones »

philwelch wrote: post #85
simiones wrote: post #83

Earlier quoted context omitted.

> I’m pretty sure you could implement futures and async/await using Go channels too if you wanted to. Futures, maybe (though without generics you'll be either very dynamic or write a new future for each struct, of course). But async/await is a syntactic feature and can't be implemented in a language without macros and/or continuations. Basically `await` is a keyword which returns a future that will execute the rest o…

I don’t really want to implement async/await in terms of channels. I would rather just use channels directly. Notice that even in your Go code it’s explicit which part executes as a separate coroutine and where the blocking communication between coroutines takes place. Async/await is more “magical”, or maybe I’m just dumb.

Hmm, I think it only looks magical because I chose a relatively unfair comparison, I realize now.

To show a more realistic comparison, let's asume Bar() is not "async ready" in either Go or C#. Then, the C# code would look something like this:

    async Task Foo() { 
       var task = Task.Run(() => Bar());
       int i = await task;
       return i + 1;
    }
This compares more clearly to the Go version with the same assumption:

    func Foo(ret chan int) {
        task := make(chan int, 1)
        go func() {
             task
So the readability / cleanliness is pretty similar for simple examples. However, once you start doing more complex things, like I showed in my second example, the difference become much more pronounced. Overall, a `Task` offers much more information to callers than a `chan T`, so it can usually be composed in more interesting ways, but it can also sometimes be more cumbersome to use.

And again, in both languages, there are clear demarcations between async functions and blocking functions. In C#, async functions return `Task` and are usually marked `async`, while in Go async functions have one or more `chan T` arguments and don't normally return values. If you want to call an async function from a non-async function or vice-versa, you need to use special constructs (e.g. Task.Run, Task.Wait or Task.Result for C#, and `go func () {...}` and blocking channel reads/writes in Go).

Re: What Color Is Your Function? (2015)

#88

Post by simiones »

philwelch wrote: post #84
tsimionescu wrote: post #82

Earlier quoted context omitted.

> OK so Thread A calls “await” on a coroutine that executes in Thread B (where B may or may not be A). Thread A is now blocked on that coroutine. What have I gained by running that coroutine in Thread B? If the task that bar() will return is created when you first call it, then you're right, we didn't gain much. However, the task may have already been running for a long time behind the scenes, we may have done things…

> If the task that bar() will return is created when you first call it, then you're right, we didn't gain much. However, the task may have already been running for a long time behind the scenes, we may have done things in parallel with that run, and now that we need the result, we can block. So in Go that’s just a blocking channel receive.

Sure, it is, but now you're moving the goal posts. The initial assertion was that `var x = await Bar()` is equivalent to `x := Bar()` in Go, and I was explaining that it is not.

Yes, `await Bar()` is equivalent to ` <- ch`, but only if there is some goroutine that actually writes something to the channel. So there is no getting around the fact that there are colored functions to the same extent in Go as in C#/Java, in regards to asynchronicity.

Re: What Color Is Your Function? (2015)

#89

Post by philwelch »

simiones wrote: post #88
philwelch wrote: post #84

Earlier quoted context omitted.

> If the task that bar() will return is created when you first call it, then you're right, we didn't gain much. However, the task may have already been running for a long time behind the scenes, we may have done things in parallel with that run, and now that we need the result, we can block. So in Go that’s just a blocking channel receive.

Sure, it is, but now you're moving the goal posts. The initial assertion was that `var x = await Bar()` is equivalent to `x := Bar()` in Go, and I was explaining that it is not. Yes, `await Bar()` is equivalent to ` <- ch`, but only if there is some goroutine that actually writes something to the channel. So there is no getting around the fact that there are colored functions to the same extent in Go as in C#/Java, i…

What’s unclear and magical to me is the notion that Bar() is returning the result of a computation that was potentially started a long time ago. In reality, “await Bar()” could be a clumsy attempt to force a needlessly async function to behave synchronously, or it could be simply receiving a result from some other task that’s been running behind the scenes. So you’re shifting the goalposts a little bit, too—awaiting Bar() is only equivalent to a channel receive if there is some other task that actually started the work.

And I maintain that Go does not have colored functions. Colored functions are when you hijack function call semantics to do asynchronous programming. Go doesn’t do this; the asynchronous behavior has to be done explicitly. Even though it’s possible, it isn’t idiomatic for a function to return a channel that you have to try and receive later, the way async functions have to be awaited before you actually get the return value.

Re: What Color Is Your Function? (2015)

#90

Post by simiones »

philwelch wrote: post #89
simiones wrote: post #88

Earlier quoted context omitted.

Sure, it is, but now you're moving the goal posts. The initial assertion was that `var x = await Bar()` is equivalent to `x := Bar()` in Go, and I was explaining that it is not. Yes, `await Bar()` is equivalent to ` <- ch`, but only if there is some goroutine that actually writes something to the channel. So there is no getting around the fact that there are colored functions to the same extent in Go as in C#/Java, i…

What’s unclear and magical to me is the notion that Bar() is returning the result of a computation that was potentially started a long time ago. In reality, “await Bar()” could be a clumsy attempt to force a needlessly async function to behave synchronously, or it could be simply receiving a result from some other task that’s been running behind the scenes. So you’re shifting the goalposts a little bit, too—awaiting…

> So you’re shifting the goalposts a little bit, too—awaiting Bar() is only equivalent to a channel receive if there is some other task that actually started the work.

The most correct way of looking at this is that async fucntions return a channel that you can read data from, and await is exactly like a channel read. The magic in async/await comes in the function calling 'await', which is suspended and scheduled to continue later. So, just like in Go, you could create a function that needlessly uses channels but is otherwise synchronous, or you can have a function that uses channels because it actually is doing something in the background.

> Colored functions are when you hijack function call semantics to do asynchronous programming. Go doesn’t do this; the asynchronous behavior has to be done explicitly.

Colored functions are a very general concept. They refer to families of functions which perform the same computation but that need to be written differently for syntactic reasons. Famous examples that have nothing to do with async include C++ const-ness, where you have to write a const and a non-const version of a function to to be able to offer the same functionality for this and for const this.

Go has this with functions which return data versus functions that push data on a channel. If you want your function to be callable in a sync manner, it must return the data directly. If you want it to be callable in an async manner, it must take a channel and push its result on that channel. You can't launch a sync function as a new goroutine and get the result back. You can't call an async function as a regular function and read the result back. So, they are two colors of functions in Go.

More concretely, say you are writing an HTTP client lib. If you want to allow your users to start multiple requests in parallel in a nice fashion, you must write your request function to take a channel and push its response there. If you want to make it easy for your users to just call your function in a sync manner, you must write a function that returns the response as its return value.

Of course, you can implement one in terms of the other, as demonstrated earlier. But you still need to add that wrapping, so your functions have a color.

Post reply on HN