Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft seeks Rust developers to rewrite core C# code

theregister.com

Post reply on HN
Print view View original

251–256 of 256 posts

Re: Microsoft seeks Rust developers to rewrite core C# code

#251

Post by pdimitar »

neonsunset wrote: post #249
pdimitar wrote: post #248

Earlier quoted context omitted.

To answer your question: To see how far behind everything is in terms of parallelism and concurrency. It's not even funny how primitive 99.9% of everything out there is in this area.

Maybe compared to C++ or even Go (yes, Go is very rudimentary with examples expecting you to synchronize goroutines by hand), but unlikely compared to C#. While both parallelism and concurrency are not as central to it as to Erlang, it is a much more approachable language and achieving either or both is trivial: // Concurrency using var http = new HttpClient(); var req1 = http.GetStringAsync("https://example.org/");…

OK, but syntax by itself is not saying much. Ruby's ractors can look similar but the underlying implementation is subpar.

Does C# have actual green threads or actors? Proper work-stealing schedulers, low-latency guarantees etc.?

If so, that would be cool, Erlang's BEAM VM desperately needs competition.

But I doubt it. From what I am seeing regularly on HN, people prefer to degrade or belittle / minimize the value of the BEAM VM than to admit that the runtime of their favorite language is still not good enough. Cognitive dissonance is getting in the way it seems. Shame.

Re: Microsoft seeks Rust developers to rewrite core C# code

#252

Post by neonsunset »

pdimitar wrote: post #251
neonsunset wrote: post #249

Earlier quoted context omitted.

Maybe compared to C++ or even Go (yes, Go is very rudimentary with examples expecting you to synchronize goroutines by hand), but unlikely compared to C#. While both parallelism and concurrency are not as central to it as to Erlang, it is a much more approachable language and achieving either or both is trivial: // Concurrency using var http = new HttpClient(); var req1 = http.GetStringAsync("https://example.org/");…

OK, but syntax by itself is not saying much. Ruby's ractors can look similar but the underlying implementation is subpar. Does C# have actual green threads or actors? Proper work-stealing schedulers, low-latency guarantees etc.? If so, that would be cool, Erlang's BEAM VM desperately needs competition. But I doubt it. From what I am seeing regularly on HN, people prefer to degrade or belittle / minimize the value of…

"Does this have Green threads?" is a common fallacy (along with mentioning function coloring) and arguably much worse API than Task/Future-based concurrency primitives because it does not give idiomatic control over yielding/dispatching and hides an important characteristic of the executed code, specifically, calls being a promise to produce a result in the future (e.g. the example above with http requests is much clunkier without them).

Some languages intentionally opt for a much more limited way of doing it like Go with goroutines and channels, some have to retrofit an async-alternative to benefit the existing decades of code like Java, and some languages eventually got it right like F#, C#, in a way, Python and JS/TS, and, to dismay of many (and pain I sympathize with), Rust.

Also, in C#, the way to go about it is to just pick the abstraction you think fits the problem best, be it manual threading, async/await Tasks (or anything custom that integrates with it), channels or, as you mentioned, actors for which there are Akka.NET and Orleans. I believe F# is even more flexible in this area.

As for the implementation details of BEAM VM - just look at e.g. the times in 1BRC challenge for BEAM-based submissions - the rift between compiled languages and the former is immense and likely unclose-able, and this trend persists in any (micro)benchmark for BEAM I look at - the overhead of most trivial operations is just too damn high! It was a groundbreaking technology at its inception but today - likely not anymore.

Not sure what* makes you quick to assume that the people working on .NET, JVM and other platforms which care about performance in (massively) parallel domains don't know what they are doing but work-stealing schedulers are yesteryesterday news and "everyone" does them (.NET, Tokio, Go from what I know), both in the form of threadpool implementations and in the form of bespoke parallel abstractions (both examples in my previous comment have these). Same applies to latency-minimization techniques, priority scheduling (worst case you can always have a dedicated worker thread that runs at higher prio, or a scheduler with a pool of those) and more.

(* If you have been burned by Ruby - I did hear about it being subpar in all kinds of ways and observed being very slow but that should rather be an exception than the rule among other languages)

Re: Microsoft seeks Rust developers to rewrite core C# code

#253

Post by pdimitar »

neonsunset wrote: post #252
pdimitar wrote: post #251

Earlier quoted context omitted.

OK, but syntax by itself is not saying much. Ruby's ractors can look similar but the underlying implementation is subpar. Does C# have actual green threads or actors? Proper work-stealing schedulers, low-latency guarantees etc.? If so, that would be cool, Erlang's BEAM VM desperately needs competition. But I doubt it. From what I am seeing regularly on HN, people prefer to degrade or belittle / minimize the value of…

"Does this have Green threads?" is a common fallacy (along with mentioning function coloring) and arguably much worse API than Task/Future-based concurrency primitives because it does not give idiomatic control over yielding/dispatching and hides an important characteristic of the executed code, specifically, calls being a promise to produce a result in the future (e.g. the example above with http requests is much cl…

> "Does this have Green threads?" is a common fallacy (along with mentioning function coloring) and arguably much worse API than Task/Future-based concurrency primitives...

IMO both are really just handy abstractions. The salient question and point is: can we have 10,000+ perceptibly parallel tasks? The BEAM VM does it. And yes we're talking such that are CPU-intensive as well.

> and some languages eventually got it right like F#, C#, in a way, Python and JS/TS

If you say so. I haven't seen any proof of it for my 20+ years of programming. C# is quite the nice language but you are stretching the compliments towards it by ascribing to it that "it got parallelism right". It absolutely did not, as didn't 99% of all languages and runtimes out there. To this day. Having an API for good parallelism doesn't mean your runtime is prepared for it. That's why Orleans and Akka still cannot do what the BEAM VM can do, and likely never will.

And Python and JS? You are just inserting a joke hoping I won't notice here, right? Right? I literally made dozens of thousands of bucks rewriting Python programs where people thought they were oh-so-clever with asyncio et. al., to Golang and to Rust. Easily 200x the throughput, and 99.9% of all parallel bugs disappeared overnight (well, after we launched, I mean). There were a grand total of 7 other bugs remaining we uncovered in the first month in production. I still keep contact with those old colleagues. The app ran unhindered for ~11 months before they finally hired a new team to keep developing it after they let go 10 out of 11 contractors after the project was mostly done (and I was in that group).

> Also, in C#, the way to go about it is to just pick the abstraction you think fits the problem best

Sure, do that, but in the end, and again, what matters most are the building blocks below -- do they enable the true lag-resistant parallelism that doesn't require a lot of fiddling?

> this trend persists in any (micro)benchmark for BEAM I look at - the overhead of most trivial operations is just too damn high! It was a groundbreaking technology at its inception but today - likely not anymore.

What's ground-breaking in terms of academic research matters not to industry, at least 90% of the time.

I work with Elixir (and Golang, and Rust) professionally every day. Even today Phoenix is the most stable web stack I've ever encountered (and one of the very fastest throughout all dynamic languages and no, don't quote TechEmpower; they scarcely know what they're doing, though happily they became more open to PRs gradually which helped their later iterations a lot).

Bombard the BEAM VM, DDoS it, all responses get slightly slower and slower as the load mounts up, but it has to be at its breaking point until you start seeing actually failing request-response pairs. Not to mention parts of your app's supervision tree can fail and they just get restarted without bringing the entire app down (like the database pool).

Try doing that with Ruby. Or Java. Exception after exception, and the server OS process has to be restarted if somebody missed a catch clause (lol).

In truth, Golang and Rust fare quite well too, but that's partly by the virtue of them being able to absorb much more hammering (they are between 20x to 1000x faster than Elixir depending on framework) and not strictly because of their runtimes. They do have good runtimes though. Not as fault-tolerant and lag-minimizing like the BEAM VM, but they are not far.

> Not sure what makes you quick to assume that the people working on .NET, JVM and other platforms which care about performance in (massively) parallel domains don't know what they are doing

1. Sunk cost fallacy;

2. Stockholm syndrome;

3. Risk aversion;

4. Sinclair's law: "It Is Difficult to Get a Man to Understand Something When His Salary Depends Upon His Not Understanding It". You likely command a good salary with C# and have made a good career with it. Of course you will not want to think it's suboptimal.

Shall I go on?

And I am not saying that "they don't know what they are doing". I am saying there is too much inertia that nobody is ever going to make a revolutionary change -- everyone is too afraid and are just swallowing the problems and are becoming experts at avoiding them for as long as possible. Also financial stakeholders will never allow such revolutionary changes, but that's a very different topic.

So yeah, not the same thing.

> work-stealing schedulers are yesteryesterday news

1. I didn't suggest they are revolutionary. I suggested they are a good pattern that's proven, and you also noticed that.

2. What is "yesteryesterday's news" matters not. The ideas of the BEAM VM are quite old and they serve perfectly many businesses today. Are you suggesting trendiness > merit? I hope not.

---

Overall, I will vehemently disagree that you just have to pick a language and it will all work out if you try hard enough. Absolutely not. This forced and imagined equality between languages and runtimes does NOT exist. Some of them are objectively better than others for jobs X and Y and I am tired of people pretending otherwise.

Re: Microsoft seeks Rust developers to rewrite core C# code

#254

Post by neonsunset »

pdimitar wrote: post #253
neonsunset wrote: post #252

Earlier quoted context omitted.

"Does this have Green threads?" is a common fallacy (along with mentioning function coloring) and arguably much worse API than Task/Future-based concurrency primitives because it does not give idiomatic control over yielding/dispatching and hides an important characteristic of the executed code, specifically, calls being a promise to produce a result in the future (e.g. the example above with http requests is much cl…

> "Does this have Green threads?" is a common fallacy (along with mentioning function coloring) and arguably much worse API than Task/Future-based concurrency primitives... IMO both are really just handy abstractions. The salient question and point is: can we have 10,000+ perceptibly parallel tasks? The BEAM VM does it. And yes we're talking such that are CPU-intensive as well. > and some languages eventually got it…

I appreciate the long reply (no sarcasm) and the compliment regarding the salary haha (it makes for a nice aspirational goal, to have good total comp). I probably can't respond to the whole post but feel like we can come to an understanding.

You mentioned 10_000 perceptibly parallel tasks so I threw together a small example - what if we had 10_000_000 concurrently executed tasks instead? This takes 1.2-1.5GB of RAM to run but it can give you a good showcase that .NET ThreadPool can take a lot of punishment and this is far from the worst you could see in one enterprise codebase or another:

    var tenMillionTasks = Enumerable
        .Range(0, 10_000_000)
        .Select(async i =>
        {
            // Force the yield - this way the methods will continue the execution
            // in the form of scheduled continuations (work items), so that we pay
            // for context switching too, similar to a more realistic scenario.
            await Task.Yield();
            for (var j = 10_000; j >= 0; j--)
            {
                // This is an okay proxy for doing some work since the
                // JIT is fairly conservative and won't auto-vectorize this
                // because its computation budget is prioritized on more impactful
                // optimizations like inlining, CSE, devirtualization, etc.
                // (this is completrly compensated by portable SIMD API)
                i++;
            }

            return i;
        });

    // Scales linearly with the number of CPU cores
    var results = await Task.WhenAll(tenMillionTasks);
    var average = results.Average();

    Console.WriteLine($"Done! The avg is {average}");
You can try it yourself if you're interested. To do so you can get an SDK from https://dot.net/download, execute 'dotnet new console', paste the code to Program.cs and then execute 'dotnet run -c Release'. This is more stressing to the runtime and is likely more fair than another comparison posts a year ago or so here on HN where that evaluated various runtimes by having the task simply wait for a period of time. Nonetheless, you can replicate that too by adding 'var delay = Task.Delay(TimeSpan.FromSeconds(5))' before tasks variable and replacing the lambda passed to .Select with 'async _ => await delay' to see how small the task overhead is.

Re: Microsoft seeks Rust developers to rewrite core C# code

#255

Post by pdimitar »

neonsunset wrote: post #254
pdimitar wrote: post #253

Earlier quoted context omitted.

> "Does this have Green threads?" is a common fallacy (along with mentioning function coloring) and arguably much worse API than Task/Future-based concurrency primitives... IMO both are really just handy abstractions. The salient question and point is: can we have 10,000+ perceptibly parallel tasks? The BEAM VM does it. And yes we're talking such that are CPU-intensive as well. > and some languages eventually got it…

I appreciate the long reply (no sarcasm) and the compliment regarding the salary haha (it makes for a nice aspirational goal, to have good total comp). I probably can't respond to the whole post but feel like we can come to an understanding. You mentioned 10_000 perceptibly parallel tasks so I threw together a small example - what if we had 10_000_000 concurrently executed tasks instead? This takes 1.2-1.5GB of RAM t…

I appreciate you putting this together. :)

If I am wrong, I'd be happy. It would be about time the big software stacks took proper parallelism to heart.

Re: Microsoft seeks Rust developers to rewrite core C# code

#256

Post by Capricorn2481 »

wredue wrote: post #250
theon144 wrote: post #145

Earlier quoted context omitted.

>It would be a red flag if you were interviewing for react and decided to bring up vue or svelte or angular or whatever else as well. ...why? Seriously, why on earth? I don't follow this train of thought at all; if they demonstrate proficiency within the scope of the position, why does it matter if they also happen to know other technologies? "Oh, Alice? Yeah, she was a great candidate, unfortunately she also had exp…

Nice strawman bro. Nobody is saying to not expand your knowledge. You’re assuming it of this because it’s literally your only argument, but it’s an unfortunately shitty one, as most logical fallacies tend to be. Nobody said “don’t have wide experience” but you. What I did say was “I’d probably avoid being an ardent fanboy toward an irrelevant to the interview tech stack”. And that “it’s most often best to leave irrel…

You said if someone brought up something like Svelte when interviewing for a React job it would be a red flag. That just sounds silly to people that know how naturally someone could connect Svelte to a discussion about React.

Could I imagine a scenario where it's a non-sequitur? Sure. Not really "don't hire this person" worthy, though.

Post reply on HN