Live data from Hacker News

The Safety Boat: Kubernetes and Rust

msrc-blog.microsoft.com

91–100 of 102 posts

Re: The Safety Boat: Kubernetes and Rust

#91
post #70
post #24

Earlier quoted context omitted.

Even accounting for what you say, it's a short learning curve compared to most languages. It's not like there's any programming language in the world where you just read the manual through once and, boom, instantly you know exactly how to architect a multi-person-century project right out of the gate or something.

Hmm, that’s a bit of a straw man you’re making here. No one is arguing better languages will magic complex architecture efforts, we’re talking about basic state management here... garden variety implementation details

State in the context of concurrent programming is not a "garden variety implementation detail." It's the Great White Whale of our industry. No language does it especially well. I have my favorites in this arena, but it's still hunting a large sea mammal with a harpoon. What you're suggesting is essentially avoiding hunting it altogether. We're not there yet.

Re: The Safety Boat: Kubernetes and Rust

#92
post #87

Earlier quoted context omitted.

I can’t take credit here, while I am around to answer questions, getting folks going is not my job. It is true that we have a chat room with a bunch of folks, of which I’m part.

I wouldn't discount what gp is saying though - having an (or a few) experts on hand from the start, can help training the first new convert "the right way" and they can then mentor the next one and so on. Even just by being availabletto answer questions or help with code review. Doing some pair programming sessions would probably be useful too.

Oh yeah, it’s helpful for sure. I just don’t want to take too much credit!

Re: The Safety Boat: Kubernetes and Rust

#93
post #40

> we caught a significant race condition It is a data race, not a race condition. > and which passed the race checker for Go No, it is not. https://github.com/helm/helm/pull/7820#issuecomment-60436062... There is a comment by issue author which is literally a go data race detector warning. Like "WARNING: DATA RACE".

It is a data race. I'm guessing the race detector (go test -race) didn't detect it because they are layering multiple synchronization primitives (mutexes, channel i/o, and a WaitGroup) and their tests hit the "good" code path but production workloads didn't.

Here's what happens. Delete takes a ResourceList. It delegates to "perform" and then "batchPerform". perform calls batchPerform in a separate goroutine, which calls a helper function in another goroutine for every resource in the ResourceList. The helper function is defined in Delete and updates a data structure defined in Delete. This is a classic case where some synchronization is necessary. The function runs multiple times in multiple goroutines, and updates a single shared structure. (Perhaps not obvious because it delegates to two helper functions, and the list that the function is executed on is a "ResourceList" not a []Resource, so it isn't clear that there is a "for { go func() }" loop anywhere; the programmers did their best to make it non-obvious that a loop is occurring.)

The confounding factor here is that batchPerform tries to synchronize with a WaitGroup, but it's faulty and not enough to protect the data integrity. batchPerform creates a WaitGroup, but only calls Wait() on the WaitGroup when the "kind" of an individual resource is not equal to the "kind" passed to batchPerform. I am guessing that it's very natural to craft some test data where this condition is met, and the for loop in batchPerform only runs the function once at a time (perhaps a ResourceList of length 1). In that case, there is no race condition for the race detector to detect.

All in all, if I were reviewing this code, it would not be checked in its current form. Splitting perform and batchPerform doesn't make sense to me, and they both implement faulty synchronization logic in a slightly different way. (batchPerform uses "for { wg.Add(); go f() }; wg.Wait", perform does "for range x { go func() { ch The root cause is that the caller of Delete isn't really sure about the semantics of "perform". Does it protect the body of the callback function? There is no documentation, and the author thought "yes". But the answer was "no". In general, the convention in go is to consider something thread-unsafe unless it's marked as thread safe. When you see something like "var foo Foo; f(list, func(bar){ foo = bar })" your spidey sense should be concerned about synchronization. But in this case, the code went out of its way to hide the existence of a loop and the existence of parallel processing, and so the programmer made a mistake. A bug or at least VERY confusing use of WaitGroup in batchPerform allowed the tests to pass. Should the compiler detect this? It would be nice. But a code reviewer should have been super concerned about this implementation.

Re: The Safety Boat: Kubernetes and Rust

#94
post #77

Earlier quoted context omitted.

Go is the golden standard for extracting the most value out of unexperienced computer science grads but it is not the measuring stick, not by a long shot.

Java did it first, catching up with 1996 here.

Interestingly enough, Java caught hell from people for "pandering" to "average" programmers, where Go seems to be getting kudos for the same thing. Strange times.

Re: The Safety Boat: Kubernetes and Rust

#95
post #47
post #40

> we caught a significant race condition It is a data race, not a race condition. > and which passed the race checker for Go No, it is not. https://github.com/helm/helm/pull/7820#issuecomment-60436062... There is a comment by issue author which is literally a go data race detector warning. Like "WARNING: DATA RACE".

Data races are a kind of race condition, no?

I can convince myself that data races need not be a race condition. Consider this simple program:

    var i int
    doneCh := make(chan struct{})
    go func() { i = 1; doneCh 
At the end of the program, i is always equal to 1 no matter which order a or b wrote to i. But it's a race because you are assigning to a shared variable without synchronization. A small modification to the program creates a race condition:

    var i int
    doneCh := make(chan struct{})
    go func() { i = 1; doneCh 
Is i 1 or 2? It depends.

It is correct for the race checker to complain about the first program, because after a bit of hacking the first program can very easily change into the second program.

(And I tried it, and it does complain.)

Re: The Safety Boat: Kubernetes and Rust

#96
post #73

Earlier quoted context omitted.

> That is the case, but it's super awkward to use. That's really the case for any language where an eventloop is not part of a builtin runtime (like it e.g. is with Javascript or Dart). E.g. in C++ we also have boost asio, libuv, libevent, wagle, seastar,GUI framework eventloops in GTK, QT, etc. The thing is once you are in async land, nothing is interoperable anymore in most environments. Whether that's ideal or not…

IMO it’s more that the Rust community has fostered a culture of doing things carefully and doing them well whenever possible (I mean, it’s the language that will argue with you for hours over reference lifetimes, after all).

There are certainly high expectations in the Rust community about doing things perfectly. But I don't think those "async ecosystem" discussions are a good example of productive discussions. I think e.g. in C++ there had been far more expert talk on standarization, within expert groups - like for the standarization of executors or the networking TS. And yet after 5 years or so nothing had been standardized yet.

In Rust the amount of people that actually work on the low level details and try to make things better is likely < 5. But there are a lot of expectations from everyone else about having perfect interoperability.

Re: The Safety Boat: Kubernetes and Rust

#97
post #20

Earlier quoted context omitted.

Go has a shorter time, and that’s the measuring stick in this area.

Go appears to have a shorter time, because you don't realize how much higher-level stuff you're just expected to do The Right Way, with no support from the language or libraries. So you're free to think you've finished learning Go, but then the actual learning begins.

Learning should pay off. Bragging that one can learn your language quickly is like bragging that your toolbox is nearly empty.

Re: The Safety Boat: Kubernetes and Rust

#98
post #77

Earlier quoted context omitted.

Go is the golden standard for extracting the most value out of unexperienced computer science grads but it is not the measuring stick, not by a long shot.

Java did it first, catching up with 1996 here.

Java was derided as being for people who thought C++ was too hard. But those people were right. C++ is too hard, in that "undefined behavior" demands an inhuman degree of perfection.

Re: The Safety Boat: Kubernetes and Rust

#100
post #77

Earlier quoted context omitted.

Java did it first, catching up with 1996 here.

Java was derided as being for people who thought C++ was too hard. But those people were right. C++ is too hard, in that "undefined behavior" demands an inhuman degree of perfection.

And now Go is being praised for being for people that think Java and Python are too hard.

It is a better option than keeping using C, and it would have been great in 1996, but that is about it.

Post reply on HN