Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Maybe Rust isn’t a good tool for massively concurrent, userspace software

bitbashing.io

Post reply on HN
Print view View original

601–610 of 624 posts

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#601

Post by KingMob »

zdragnar wrote: post #471
throw10920 wrote: post #420

Earlier quoted context omitted.

> The difference is that if you get it wrong in a managed language, you get leaks or stale objects or other logic bugs. Can you provide concrete examples of this? I've literally never had a bug due to the nature of a memory-managed language.

Once upon a time (at least through IE7) Internet Explorer had separate memory managers for javascript and the DOM. If there was a cycle between a JS object and a DOM object (a DOM node is assigned as a property of an object, and another property was assigned as an event handler to the DOM node) then IE couldn't reclaim the memory. Developers of anything resembling complex scripts (for the time) had to manually break…

> separate memory managers

Notwithstanding the rest of your comment, this doesn't seem like a good example of the problem, since most GCs have a complete view of their memory.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#602

Post by lelanthran »

jayd16 wrote: post #587
lelanthran wrote: post #578

Earlier quoted context omitted.

That's an implementation problem, not a problem with the concept of asynchronous execution, and it's specifically a problem in only one popular implementation: Javascript in the browser without web-workers. That's specifically why I called it a Leaky Abstraction in my first post on this: too many people are confusing a particular implementation of asynchronous function calls with the concept of asynchronous function…

I don't see how it can be an implementation detail when fundamentally you must yield execution when the programmer has asked to retain execution. Besides Javascript, its also a common problem in C# when you force synchronous execution of an async Task. I'm fairly sure its a problem in any language that would allow an async call to wait for a thread that could be waiting for it. I really can't imagine how your propose…

> I don't see how it can be an implementation detail when fundamentally you must yield execution when the programmer has asked to retain execution.

It's an implementation issue, because "running on only a single thread" is an artificial constraint imposed by the implementation. There is nothing in the concept of async functions, coroutines, etc that has the constraint "must run on the same thread as the sync waiting call".

An "abstraction" isn't really one when it requires knowledge of a particular implementation. Async in JS, Rust, C#, etc all require that the programmer knows how many threads are running at a given time (namely, you need to know that there is only one thread).

> But I look forward to your implementation.

Thank you :-)[1]. I actually am working (when I get the time, here and there) on a language for grug-brained developers like myself.

One implementation of "async without colored functions" I am considering is simply executing all async calls for a particular host thread on a separate dedicated thread that only ever schedules async functions for that host thread. This sidesteps your issue and makes colored functions pointless.

This is one possible way to sidestep the specific example deadlock you brought up. There's probably more.

[1] I'm working on a charitable interpretation of your words, i.e. you really would look forward to an implementation that sidesteps the issues I am whining about.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#603

Post by kunley »

SubjectToChange wrote: post #597
kunley wrote: post #579

Earlier quoted context omitted.

Java has billions spent on marketing and lobbying. Since the advent of Java in mid-90s I hear about superiority of its VM, yet my observations from the ops PoV claim otherwise. So I suspect a huge hoax... Hey btw, you're saying "Java is _memory intensive_", like it would magically explain everything. Let's get to that more deeply. Why is it so, dear Watson? Have you compared the memory consumption of the same algo an…

Java has billions spent on marketing and lobbying. Marketing is not a silver bullet for success and the tech industry is full of examples of exactly that. The truth is that Sun was able to promote Java so heavily because it was found to be useful. Since the advent of Java in mid-90s I hear about superiority of its VM, yet my observations from the ops PoV claim otherwise. The landscape of the 90s certainly made a VM l…

I would like to answer your points more deeply, not having much time for it now.

But I think you are avoiding a direct answer to the question why Java needs so much memory in the first place. You say about "developer's choice for a big heap", first I don't think it is their choice, but the consequence of the fact that such a big heap is needed at all, for a typical code. Why?

Let's code a basic https endpoint using typical popular framework returning some simple json data. Usually stuff. Why it will be consuming 5x - 10x more memory for Java? And, if one says it's just unrealistic microbenchmark, things go worse when coding more real stuff.

Btw,having more knobs for a gc is not necessarily a good thing, if it means that there are no fire-and-forget good defaults. If an engineer needs continously to get his head around these knobs to have a non-crashing app, then we have problem. Or rather - ops have a problem, and some programmers are, unfortunately, disconnected from the ops realm. Have you been working together with ops guys? On prod, ofc?

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#604

Post by seabrookmx »

ilyt wrote: post #585
seabrookmx wrote: post #392

Earlier quoted context omitted.

I honestly can't tell if you're trolling. - the C# implementation predates even Promises in JS, so it is not "the same implementation" and your implication that C# was inspired by JS as opposed to the other way around is false. More background: [0] - Typescript works fine with the JS implementation so any differences aren't for type safety reasons, but largely because C# has a multithreaded event loop unlike JS Also…

You're implying people went from writing C# to JS code willingly and not just wrote it but went on "improving" the ecosystem and I just don't believe there are people insane enough to do it willingly so "it was invented separately in both instances" is far more likely.

I can't tell if you're talking about the language designers or users.

For language designers I point out in my other comment that Anders authored both C# and TS. TS' influence on ES6 is documented publicly.. heck TC39 has an open proposal to add type annotations to JS now!

As for users, any dev that touches frontend has to write JS, unless you're purely a mobile or desktop shop (even then, there's electron). So yes, I think tons of folks willingly write C# and JS/TS. I'm certainly one (though write more Python than both these days). Was I an early adopter of async in Python because of my familiarity of it in C#/JS? You bet I was. Maybe I'm "insane."

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#605

Post by lmm »

throw10920 wrote: post #476
lmm wrote: post #462

Earlier quoted context omitted.

If you're saying you want GC/Arc then that's more than just "turning off the borrow checker".

Pedantry. Later on in my comment I literally say "manage memory for you" - it should be pretty clear that my intent was to talk about a hypothetical language that allowed you to change between use of a borrow checker and managed memory, even if I didn't use the correct wording ("turn off the borrow checker") in that particular very small section of it.

Bit much to complain about pedantry with how prickly your tone has been in this whole thread. If you only want this functionality for rapid iteration/prototyping, which was what you originally said, then leaking memory in those circumstances is not such a problem.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#606

Post by throw10920 »

lmm wrote: post #605
throw10920 wrote: post #476

Earlier quoted context omitted.

Pedantry. Later on in my comment I literally say "manage memory for you" - it should be pretty clear that my intent was to talk about a hypothetical language that allowed you to change between use of a borrow checker and managed memory, even if I didn't use the correct wording ("turn off the borrow checker") in that particular very small section of it.

Bit much to complain about pedantry with how prickly your tone has been in this whole thread. If you only want this functionality for rapid iteration/prototyping, which was what you originally said, then leaking memory in those circumstances is not such a problem.

You're right, I have been overly aggressive. I apologize.

> If you only want this functionality for rapid iteration/prototyping, which was what you originally said, then leaking memory in those circumstances is not such a problem.

There's use-cases for wanting your language to be productive outside of prototyping, such as scripting (which I explicitly mentioned earlier in this thread[1] - omission here was not intentional), and quickly setting up tools (such as long-running web services) that don't need to be fast, but should not leak memory.

"Use Rust, but turn the borrow checker off" is inadequate.

[1] https://news.ycombinator.com/item?id=37441120

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#607

Post by lmm »

throw10920 wrote: post #606
lmm wrote: post #605

Earlier quoted context omitted.

Bit much to complain about pedantry with how prickly your tone has been in this whole thread. If you only want this functionality for rapid iteration/prototyping, which was what you originally said, then leaking memory in those circumstances is not such a problem.

You're right, I have been overly aggressive. I apologize. > If you only want this functionality for rapid iteration/prototyping, which was what you originally said, then leaking memory in those circumstances is not such a problem. There's use-cases for wanting your language to be productive outside of prototyping, such as scripting (which I explicitly mentioned earlier in this thread[1] - omission here was not intent…

Yeah, I do think the space where manual memory management is actually desirable is pretty narrow - and so I'm kind of baffled that Rust caught on where the likes of OCaml didn't. But seemingly there's demand for it. (Either that, or programming is a dumb pop culture that elevates performance microbenchmarks beyond all reason)

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#608

Post by dgroshev »

endorphine wrote: post #538

Can someone explain where the stackless/stackful naming comes from and what does it mean?

Rust futures don't have stacks, all local variables get translated into struct fields, so suspending execution on an awake point is trivial, you just start executing another future. It also means there is no need for a runtime for a future to make sense (it's just a struct with a trait implemented) and the futures are completely static and amenable to compiler optimisations like inlining. On the other hand, it means that the structure is by necessity self referential (one "local" variable might refer to another, so the struct has addresses in it), which means it can't be safely moved (because will point at the old address).

"Stackful" coroutines, on the other hand, do have runtime stacks (holding local variables) that get swapped out by the runtime on await points. It makes the code behave exactly like non-async code, but requires a runtime to manage those stacks. Rust didn't go this way, preferring the benefits of the stackless approach.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#609

Post by pdimitar »

hgomersall wrote: post #477
vilunov wrote: post #288

Earlier quoted context omitted.

You seem to raise valid points and I don't disagree with you, however I don't see how it's relevant to the original concern regarding colored functions.

I suppose I'm struggling to understand what "colour" means in the context of Rust. It's surely just another word for signature. For some reason it's trotted out every time there's a discussion about async. I can only assume it's to do with the original use of the term for JavaScript async (which I know almost nothing about and have no opinion on), but I just cannot see its point in Rust async.

It has to do with the fact that most of the code in the project is not async but having to call async functions often propagates all the way to your main function. It's infectious and many people don't like it, myself included, that's why I'm working with Elixir and Golang where async is transparent and 99% automatic, or explicit but non-infectious, respectively.

I do love Rust and found a number of very valid uses for it but its async story leaves a lot to be desired. I don't enjoy writing it though I do enjoy the results.

Re: Maybe Rust isn’t a good tool for massively concurrent, userspace software

#610

Post by pdimitar »

dekhn wrote: post #172

I wish people would stop saying concurrency and parallelism are different. Concurrency is a subtype of parallelism. All concurrency is parallelism, but leaving some aspects of parallelism off the table. I've worked in both worlds: I've built codes that manage thousands of connections through the ancient select() call on single processes (classic concurrency- IO multiplexing where most channels are not active simultan…

If that's what you need: it's Erlang / Elixir then.
Post reply on HN