Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rewriting Rust

josephg.com

Post reply on HN
Print view View original

271–280 of 410 posts

Re: Rewriting Rust

#271

Post by ptsneves »

olalonde wrote: post #142
dist1ll wrote: post #15

I think the dependency situation is pretty rough, and very few folks want to admit it. An example I recently stumbled upon: the cargo-watch[0] crate. At its core its a pretty simple app. I watches for file changes, and re-runs the compiler. The implementation is less than 1000 lines of code. But what happens if I vendor the dependencies? It turns out, the deps add up to almost 4 million lines of Rust code, spread acr…

Why do you care how many lines of code the dependencies are? Compile time? Lack of disk space?

Think of the problem as a bill of materials. Knowing the origin and that all the components of a part are fit for purpose is important for some applications.

If I am making a small greenhouse i can buy steel profiles and not care about what steel are they from. If I am building a house I actually want a specific standardized profile because my structure's calculations rely on that. My house will collapse if they dont. If I am building a jet engine part I want a specific alloy and all the component metals and foundry details, and will reject if the provenance is not known or suitable[1].

If i am doing my own small script for personal purposes I dont care much about packaging and libraries, just that it accomplishes my immediate task on my environment. If I have a small tetris application I also dont care much about libraries, or their reliability. If I have a business selling my application and I am liable for its performance and security I damn sure want to know all about my potential liabilities and mitigate them.

[1] https://www.usatoday.com/story/travel/airline-news/2024/06/1...

Re: Rewriting Rust

#272

Post by dartos »

preommr wrote: post #262
cogman10 wrote: post #224

Earlier quoted context omitted.

This is a natural and not really scary thing. All code is built on mountains of dependencies that by their nature will do more than what you are using them for. For example, part of cargo watch is to bring in a win32 API wrapper library (which is just autogenerated bindings for win32 calls). Of course that thing is going to be massive while watch is using only a sliver of it in the case it's built for windows. The st…

> The alternative is the npm hellscape where you have a package for "isOdd" and a package for "is even" that can break the entire ecosystem if the owner is disgruntled because everything depends on them. This used to be true 5-10 years ago. The js ecosystem moves fast and much has been done to fix the dependency sprawl.

I… don’t think that’s true.

Just look at how many downloads some of those packages have today.

Look at the dependency tree for a next or nuxt app.

What the js world did is make their build systems somewhat sane, whatwith not needing babel in every project anymore.

Re: Rewriting Rust

#273

Post by dgroshev »

josephg wrote: post #151
JoshTriplett wrote: post #26

> Now, there are issue threads like this, in which 25 smart, well meaning people spent 2 years and over 200 comments trying to figure out how to improve Mutex. And as far as I can tell, in the end they more or less gave up. The author of the linked comment did extensive analysis on the synchronization primitives in various languages, then rewrote Rust's synchronization primitives like Mutex and RwLock on every major…

Author here. Thanks for the in depth response. I appreciate hearing an insider's perspective. > I always find it amusing to see, simultaneously, people complaining that the language isn't moving fast enough and other people complaining that the language is moving too fast. I think people complain that rust is a big language, and they don't want it to be bigger. But keeping the current half-baked async implementation…

> But keeping the current half-baked async implementation doesn't make the language smaller or simpler. It just makes the language worse.

I can't disagree more.

In fact, I think that the current state of async Rust is the best implementation of async in any language.

To get Pin stuff out of the way: it is indeed more complicated than it could be (because reverse compatibility etc), but when was the last time you needed to write a poll implementation manually? Between runtime (tokio/embassy) and utility crates, there is very little need to write raw futures. Combinators, task, and channels are more than enough for the overwhelming majority of problems, and even in their current state they give us more power than Python or JS ecosystems.

But then there's everything else.

Async Rust is correct and well-defined. The way cancellation, concurrent awaiting, and exceptions work in languages like JS and Python is incredibly messy (eg [1]) and there are very few people who even think about that. Rust in its typical fashion frontloads this complexity, which leads to more people thinking and talking about it, but that's a good thing.

Async Rust is clearly separated from sync Rust (probably an extension of the previous point). This is good because it lets us reason about IO and write code that won't be preempted in an observable way, unlike with Go or Erlang. For example, having a sync function we can stuff things into thread locals and be sure that they won't leak into another future.

Async Rust has already enabled incredibly performant systems. Cloudflare's Pingora runs on Tokio, processing a large fraction of internet traffic while being much safer and better defined than nginx-style async. Same abstractions work in Datadog's glommio, a completely different runtime architecture.

Async Rust made Embassy possible, a genuine breakthrough in embedded programming. Zero overhead, safe, predictable async on microcontrollers is something that was almost impossible before and was solved with much heavier and more complex RTOSes.

"Async Rust bad" feels like a meme at this point, a meme with not much behind it. Async Rust is already incredibly powerful and well-designed.

[1]: https://neopythonic.blogspot.com/2022/10/reasoning-about-asy...

Re: Rewriting Rust

#274

Post by dartos »

simon_o wrote: post #221
lifthrasiir wrote: post #154

Earlier quoted context omitted.

> In my experience, [patch] doesn't cut it or compare. AFAIK what Maven does is an exclusion of dependency edges , which is technically an unsafe thing to do. Cargo [patch] is a replacement of dependency vertices without affecting any edges. (Maven surely has a plugin to do that, but it's not built-in.) They are different things to start with. Also I believe that the edge exclusion as done by Maven is (not just "tech…

> Namespace is not a solution for name squatting: namespace is just yet another identifier that can be squatted. If you are worried about squatting, the only effective solution is sandboxing, everything else is just moving the goal post. The problems crates.io struggles with have never been an issue with Maven, regardless of how creatively you try to redefine words. That's a fact. Deal with it.

Only sith speak in absolutes.

Re: Rewriting Rust

#275

Post by pas »

lifthrasiir wrote: post #211
pas wrote: post #206

Earlier quoted context omitted.

namespaces are a solution to having "authenticated groups of crates", it helps structuring and more importantly restructuring crates.

That's an intention, not the outcome. You might assume that having both `@chrono/chrono` and `@chrono/tz` shows a clear connection between them, but such connection is nothing to do with namespace (the actual crate names are `chrono` and `chrono-tz`), and any authority provided by `@chrono/` prefix is offseted by the availability of similar names like `@chrno/tz` or `chrono-tz`. The only thing namespace can prevent i…

"... any authority provided by [a] prefix is offseted by the availability of [similar prefixes]"

I'm not buying this, sorry. Yes, typos and other deceptive things are possible, but having this authority data would allow tools to then use this signal. Not having it seems strictly worse.

Re: Rewriting Rust

#276

Post by raverbashing »

lifthrasiir wrote: post #121
alexvitkov wrote: post #109

Earlier quoted context omitted.

That's what inevitably happens when you make transitive dependencies easy and you have a culture of "if there's a library for it you must use it!" C/C++ are the only widely used languages without a popular npm-style package manager, and as a result most libraries are self-contained or have minimal, and often optional dependencies. efsw [1] is a 7000 lines (wc -l on the src directory) C++ FS watcher without dependenci…

> The single-header libraries that are popular in the game programming space (stb_* [2], cgltf [3], etc) as well as of course Dear ImGui have been some of the most pleasant ones I've ever worked with. The mainstream game programming doesn't use C at all. (Source: I had been a gamedev for almost a decade, and I mostly dealt with C# and sometimes C++ for low-level stuffs.) Even C++ is now out of fashion for at least a…

> The mainstream game programming doesn't use C at all. (Source: I had been a gamedev for almost a decade

Game programming changed a lot, parent is talking about stuff older than 10 yrs

There was a lot of PC gaming in C/C++, and "Engine" were developed together with games for the most part. Think all the Doom and Quake saga

That's what he's talking about

Re: Rewriting Rust

#277

Post by oneshtein »

senorrib wrote: post #222
withoutboats3 wrote: post #190

Earlier quoted context omitted.

These features are slow to be accepted for good reasons, not just out of some sort of pique. For example, the design space around combining `if let` pattern matching with boolean expressions has a lot of fraught issues around the scoping of the bindings declared in the pattern. This becomes especially complex when you consider the `||` operator. The obvious examples you want to use work fine, but the feature needs to…

I think you just proved his point on how hard it is to understand and correctly use Pin.

Pinned is life time, similar to 'static, but written as Pin!() for an unknown reason.

Re: Rewriting Rust

#278

Post by FrustratedMonky »

"the coroutines RFC has lasted longer than World War 1 or 2"

This sounds bad, but I wonder how many features have taken this long to include in other languages. Is this really as out of step as it sounds?

There is the move-fast-break-things mentality, but is that how you want to design a language?

Seems like we are missing some middle ground step, where there are good features, maybe even done, and stable, but they aren't getting worked into the main language.

Maybe a decision making problem.

Re: Rewriting Rust

#279

Post by tinco »

josephg wrote: post #132
bjackman wrote: post #92

Rust isn't an Exciting New Language any more. It's in the "work towards widespread adoption" phase. Slower feature development is natural and healthy, the stakes are high, mistaken design choices are much more harmful than low velocity at this point. I'm not excited about Rust because of cool features, I'm excited because it's a whole new CLASS of language (memory safe, no GC, production ready). Actually getting it i…

Maybe. But javascript is arguably in that phase of its life as well, and JS has had a oodles of wonderful new features added in the last decade. Features like the spread operator, generator functions, async, arrow functions, leftpad, a new Date, and so on. The list of significant new features is endless. All that, despite JS being much older than rust, and much more widely used. Javascript also has several production…

They got their act together because there was a language built on top of Javascript that fixed all its problems, and it was quickly gaining wide adoption. If they hadn't done anything, we'd probably still be transpiling CoffeeScript.

History repeated itself, and now Typescript has even more popularity than CoffeeScript ever did, so if the ecma committee is still on their act, they're probably working on figuring out how to adopt types into Javascript as well.

More relevant to this argument, is the question if a similar endeavor would work for Rust. Are the features you're describing so life changing that people would work in a transpiled language that had them? For CoffeeScript, from my perspective at least, it was just the arrow functions. All the sugar on top just sealed the deal.

Re: Rewriting Rust

#280

Post by josephg »

withoutboats3 wrote: post #264
josephg wrote: post #239

Earlier quoted context omitted.

> The obvious examples you want to use work fine, but the feature needs to be designed in such a way that the language remains internally consistent and works in all edge cases. True. How long should that process take? A month? A year? Two years? I ask because this feature has been talked about since I started using rust - which (I just checked) was at the start of 2017. Thats nearly 8 years ago now. 6 years ago this…

I don't deny that Pin is complicated to use as it stands (in fact that is the entire thrust of my blog posts!), just that there is some magical easier solution involving Move and changes to the borrow checker. You wrote something on the back of a napkin and you imagine its better, whereas I actually had to ship a feature that works. The state of async Rust is not better because no one hired me to finish it past the M…

Thankyou for all your hard work on this. I'm sorry my post is, in many ways, dismissive of the huge amount of work that you and others have poured into rust, async, Pin, explaining Pin in detail over and over again, and all of the other things I take for granted in the compiler constantly.

But appreciation does little to temper my frustration. Watching the rust project spin its wheels has dulled any enthusiasm I might have once had for its open, consensus based processes. I could get involved - but I worry I'd be yet another commenter making long issue threads even longer. I don't think Rust has a "not enough cooks in the kitchen" shaped problem.

I love that quote. I agree with it - at some point, like with Pin and the 'foo.await' vs 'await foo' discussion - you just have to pick an answer, any answer, and move forward. But the siren song of that "simple and elegent" solution still calls. Alan Kay once made a similar observation. He pointed out that it took humanity thousands of years (and two geniuses) to invent calculus. And now we teach it to 8th grade children. How remarkable. Clearly, the right point of view is worth 50 IQ points.

I look forward to reading your blog posts on the topic. I suspect there's lots of workable solutions out there in the infinite solution space. Research is always harder and slower than I think it should be. And this is very much a research question.

You seem very convinced that replacing Pin with Move would be a mistake. Maybe! I wouldn't be surprised if the Move vs Pin question is a red herring. I suspect there's an entirely different approach which would work much better - something like, as I said in my post, attacking the problem by changing the borrow checker. Something like that. Maybe that wouldn't be viable for rust. Thats fine. There will be more languages following in its footsteps. I want them to be as good as possible.

And I swear, there's a better answer here somewhere.

I can feel it.

Post reply on HN