Live data from Hacker News

Notes on structured concurrency, or: Go statement considered harmful

vorpus.org

191–200 of 234 posts

Re: Notes on structured concurrency, or: Go statement considered harmful

#191

Earlier quoted context omitted.

>Well, what if you don't? This is kind of a silly question. Its always possible to subvert a safe construct system if you try hard enough. You can write unsafe blocks in rust. You can pass in a callback to a nursery, as is described in the article. >The proposed construct does not require that Well, kind of. The article actually explicitly states that >Here's a simpler primitive that would also satisfy our flow contr…

Oh come on. I don't agree that the proposed idea is fundamentally new and amazing; but I think it is novel, and there may be some value in being able to semantically bind tasks to execution points, specifically when the tasks are spawned in naive (or uncontrolled, eg. library) code, and have not explicitly opted in to the scheme. Does this particular implementation do that perfectly? No, probably not. ...but I think…

>and have not explicitly opted in to the scheme.

I think you're overestimating nurseries here. They still don't handle the case you described of a deeply nested setTimeout:

    import asyncio
    import trio
    import threading

    async def my_tricky_function():
        threading.Timer(5, lambda: print('first')).start()
        await trio.sleep(.5)

    async def a_function():
        async with trio.open_nursery() as n:
            n.start_soon(my_tricky_function)
        print("finished")

    trio.run(a_function)
(requires python 3.6, I'm using threading.timer in place of setTimeout, but they're the same construct). There's a great talk by david beazly about how threads and asyncio don't play nicely, except when they do.

Note that you could imagine that this timer is created in an awaited function or a supposedly synchronous child function.

If the nursery did what you think it did, which is to wait until everything async within the block completes, you'd see this print first, then print finished. It doesn't though, finished is printed first. The reason is that you need to opt in to the trio constructs by using promises/futures/coroutines. Using threaded callback based things give you the same problems with trio as they do in other async/parallel constructs.

If you're a good citizen and opt in to the safety guarantees trio provides, it can keep things clean for you, yes. But you do need to explicitly opt into the scheme by only using code that you know is promise/future/coroutine based. Threads are a doozy.

But the same thing is pretty much true with async/await promises/coroutines in js or python. If you require that you only use promises (or you tightly wrap all of your threads/callbacks in promises and then use that), you can get the safety of nurseries in most of the situations (I noted there were some exceptions before!), but with just the async/await syntax, no need for the extra async context manager.

I agree that trio is cool. I've used it before. But you're ascribing to it and this construct magical powers that they cannot and do not have.

Re: Notes on structured concurrency, or: Go statement considered harmful

#192

Earlier quoted context omitted.

It actually does not solve the problem of lifetimes. It only solves a very isolated instance of it. For example, if we take the problem of an os.File, some library might store the reference, and use it unexpectedly in a later function call after you called Close. This presents the exact same issue as a goroutine holding it. (I honestly have no other examples of this issue than files/connections being closed premature…

I've never encountered a situation in Go where I'm passing file handles around like this. In practice you'd probably have one goroutine manage the resource and its lifespan then other code would communicate with that goroutine using channels.

You might pass an os.File or a network connection as an io.Reader or io.Writer to something that might keep it. The GC actually has an extension to sorta have destructors to help with leaking fd's from forgotten os.File's (runtime.SetFinalizer).

However, having coded Go for quite a few years by now, I haven't found it to be an issue at all.

Re: Notes on structured concurrency, or: Go statement considered harmful

#193

What. This article proposes a "nursery", which is just a wrapped sync.WaitGroup/pthread_join/futures::future::join_all/a reactor that waits for all tasks to terminate/etc. It then uses an exception-like model for error propagation to "solve" error handling (which is fairly easy to handle with a channel). The construct is a decently usable, already applied tool to handle a set of problems, but the article takes the is…

Can I just mention that anyone who begins their comment with What. ...and: [whatever] is just a [whatever] ...comes across as a bit of an asshole? Why not just make your point instead of investing a lot in making sure everyone knows that you're smart and the other person is dumb? Maybe Knuth was just as rude to Dijkstra, who knows?

Of course you can.

I felt that such "aggressive" countering was justified considering the equivalently grand claims of the post ("EXTREMELY_POPULAR_CONSTRUCT considered harmful").

Re: Notes on structured concurrency, or: Go statement considered harmful

#194

What. This article proposes a "nursery", which is just a wrapped sync.WaitGroup/pthread_join/futures::future::join_all/a reactor that waits for all tasks to terminate/etc. It then uses an exception-like model for error propagation to "solve" error handling (which is fairly easy to handle with a channel). The construct is a decently usable, already applied tool to handle a set of problems, but the article takes the is…

> It then uses an exception-like model for error propagation to "solve" error handling (which is fairly easy to handle with a channel). If none of your code ever actually waits on the error channel when you spawn a goroutine, what happens? Is there any circumstance where that behavior is preferable to guaranteeing something must wait on the error channel?

Go's error paradigm is through return values that can be ignored, so an error channel would mimic "non-async" error handling in this case: Checking an error is is always up to the developer.

The discussion about error paradigms (exceptions vs. globals vs. error values, forced checking vs. free choice, etc.) is quite a big one, and arguably an entirely different topic.

Re: Notes on structured concurrency, or: Go statement considered harmful

#195

Earlier quoted context omitted.

As someone whose work includes hardware driver programming, I'd rather not give up goto. It would make a lot of error handling extremely cumbersome, and much less readable. :( However, I find "go" and "goto" to not intersect at all. I've written this many times in other comments on this thread, so I'd rather not type it out again, but the TL;DR: is that "goto" can make understanding a function when read difficult, wh…

> while “go” is clear when read… I believe our industry’s several decades of demonstrated inability to write concurrent code correctly disagrees with you. Languages like go (and Rust, to a greater degree) have improved the situation. But there are still a ton of sharp edges that existing approaches still have, and—often—they’re ones that aren’t readily apparent until after a project starts growing and begins unearthi…

> I believe our industry’s several decades of demonstrated inability to write concurrent code correctly disagrees with you.

Not at all. Concurrent/parallel code isn't particularly difficult to write—there's just a unique class of problems related to it that might occur, but that does not mean it is hard.

Rather, the industry has through several decades demonstrated an inability to write bug free code in general. I wouldn't blame concurrency/parallelism for this.

Re: Notes on structured concurrency, or: Go statement considered harmful

#196
post #68

Earlier quoted context omitted.

Whats the difference between taking a nursery and returning a promise? Both put the context in the signature, but the later is actually composable.

When I call a function, I have no way of knowing if there are any unhandled promises within it. The difference is the restriction: with nurseries, background tasks can only outlive function calls if the calling function allows them to.

Can you force explicit nursery passing though? One might make a global nursery object and then just rely on that.

Re: Notes on structured concurrency, or: Go statement considered harmful

#197

Earlier quoted context omitted.

Oh come on. I don't agree that the proposed idea is fundamentally new and amazing; but I think it is novel, and there may be some value in being able to semantically bind tasks to execution points, specifically when the tasks are spawned in naive (or uncontrolled, eg. library) code, and have not explicitly opted in to the scheme. Does this particular implementation do that perfectly? No, probably not. ...but I think…

>and have not explicitly opted in to the scheme. I think you're overestimating nurseries here. They still don't handle the case you described of a deeply nested setTimeout: import asyncio import trio import threading async def my_tricky_function(): threading.Timer(5, lambda: print('first')).start() await trio.sleep(.5) async def a_function(): async with trio.open_nursery() as n: n.start_soon(my_tricky_function) print…

> they cannot and do not have.

sigh~

In this implementation, or perhaps, in python at all...

Does that make it completely valueless?

I would venture to suggest that maybe there's a big wide world of languages that actually support controlling how threads are spawned, where it might not be.

Maybe its worth considering.

Re: Notes on structured concurrency, or: Go statement considered harmful

#198

This addresses the wrong problem. The real issue is control over data shared between threads, not control flow. C/POSIX type threads have no language support for indicating what data is shared and which locks protect which data. That's a common cause of trouble. The big question in shared memory concurrency is "who locks what". Most of the bugs in concurrent programs come from ambiguities over that question. Early at…

I think threads is a big enough of a problem that it can be divided into multiple sub problems where each individual problem deserves a solution. Yes, the resource sharing problem is harder than the part that this solves, but does that really matter?

Re: Notes on structured concurrency, or: Go statement considered harmful

#199
post #196
post #68

Earlier quoted context omitted.

When I call a function, I have no way of knowing if there are any unhandled promises within it. The difference is the restriction: with nurseries, background tasks can only outlive function calls if the calling function allows them to.

Can you force explicit nursery passing though? One might make a global nursery object and then just rely on that.

In some existing languages, yes. But it’s easy to imagine future languages (including future versions of current languages) that disallow creating nurseries outside of a function scope.

Re: Notes on structured concurrency, or: Go statement considered harmful

#200

Earlier quoted context omitted.

I don’t think there’s any reason a nursery has to have a static lifetime.

The description of the nursery is you can pass it around wherever you like. More generally, this concept seems to have been designed in a language without lifetimes. You obviously can have a scoped threadpool without a 'static lifetime, but if you want to pass it around to arbitrary locations then you do.

I think the whole point of the nursery's design is that you can create and destroy them during the life of your program, and that destroying them forms a barrier where the owner of the nursery waits for its children to finish. Having a nursery with a static lifetime in Rust would therefore be pointless, as it would live until the end of the program.
Post reply on HN