Live data from Hacker News

I sped up serde_json strings by 20%

purplesyringa.moe

111–120 of 124 posts

Re: I sped up serde_json strings by 20%

#112
post #29

Earlier quoted context omitted.

> The dependency trees for any meaty Rust project quickly become pretty horrifying. s/Rust// This is really no different from any other language. At least Rust, with Cargo, makes it easy to scan your dependencies. And many notable Rust projects attempt to keep third party dependencies to a minimum. C++ gives you absolutely nothing to work with. Other languages with package managers don't keep dependency trees shallow…

> This is really no different from any other language. There are languages with big standard libraries and first party frameworks. I can build a complex web app in C# using only packages published by Microsoft in ASP.NET and EF. Python ships with a lot of "batteries included". Not saying I expect that from rust considering the funding/team size discrepancy and language targets - but I disagree that every language is…

Moment you add something to std it's effectively died. Strong backwards guarantees will choke development of library.

Python std is where libraries go to die.

Re: I sped up serde_json strings by 20%

#113
post #59

Earlier quoted context omitted.

Because Rust has Cargo which makes it trivial to include dependencies. C developers constantly reinvent because C dependency management is such a joke that no one bothers.

This is so on-point. Number of dependencies is correlated with ease of package management. If you have a well-working package manager you're more likely to use a dependency than just rewrite that little part of your code. For little programs written in C or C++ this usually means that people write their own buggy CLI parsing that doesn't produce a proper help message and segfaults with the wrong combination of argume…

> Number of dependencies is correlated with ease of package management.

I disagree with this, Python makes it trivial to add dependencies. [1] And yet Python developers tend to have the mentality of an older generation of software developers, pre-node/npm, and believe in keeping most things light. That starts with the language maintainers having a vision that says this should be possible, by including the batteries most people need for mid-size projects.

This has a compounding effect. People writing the dependencies that your dependencies will use might not even need dependencies outside of the Python standard library. In Rust, those dependencies have dependencies which have dependencies. You get a large tree even when everyone involved is being relatively sane. When one or two projects are not careful, and then everyone uses those projects ...

When I'm writing Rust, I eternally have the infuriating dilemma of picking which dependency to use for a relatively common operation. Having one blessed way to do something is an enormous productivity advantage, because it's never on you to vet the code. The disadvantages of a large stdlib are pretty overstated; even when an outside library outperforms the internal one (like Python's "requests"), the tendency will be for everyone to form a consensus about which outside library to use (like Python's "requests"), because that's how the ecosystem works.

Using technology designed 25 years ago has big advantages for Linux developers as well. Even when I need to add a dependency in Python, chances are I don't have to install one in my development environment because I can experiment with the shared dependencies already installed on my operating system. And I can look to the maintainers of the software I use - and my system maintainers - for implicit advice about which libraries are reliable and trustworthy, and use those. Rust, Go, etc., throw all of this out in the name of convenience for developers, but it's precisely as a developer that I find this mentality frustrating.

[1] People complain about the UX of e.g. pip in certain situations, but (regardless of whether it's a reasonable complaint) it's not the same issue as maintaining a stack of dependencies in the language itself being difficult.

Re: I sped up serde_json strings by 20%

#114

Earlier quoted context omitted.

Built in JSON encoding/decoding is one of the things I’ve enjoyed about Swift. It’s nice when it’s not necessary to shop around for libraries for common needs like that.

Almost nobody is shopping around Rust JSON libraries unless they need some specific feature not provided by serde and serde_json. They are the default everyone reaches for.

The fact that a JSON parser is the sort of ecological niche where you naturally get a single best library which does basically what everyone wants it to do with minimal dependencies is exactly the argument for putting it in the stdlib, though.

Re: I sped up serde_json strings by 20%

#115
post #42

Earlier quoted context omitted.

> And many notable Rust projects attempt to keep third party dependencies to a minimum. I don't think this is true. The only two major Rust crates that manage to keep their dependencies light are tokio and serde, and these are highly atypical projects. For a more typical example, look at something like axum (running `cargo tree` for project with a single dependency on axum returns 121 lines). > This is really no diff…

tokio and serde are certainly not the only ones. You can put almost all of my crates into that category too. The problem with your framing is that you look at this as a "dependency addiction." But that doesn't fully explain everything. The `regex` crate is a good case study. If it were a C library, it would almost certainly have zero dependencies. But it isn't a C library. It exists in a context where I can encapsula…

Super interesting take, thanks for sharing your thoughts.

I like your trade-off framing, but I do think that your argument fails to take account of something important here, which I'll try to illustrate with an example.

Let's say I want to run a simple modern web stack. In C#, I might pull in ASP.NET and EF Core. In Java, I might pull in Spring. In Rust, I might pull in axum, axum-server, rustls and sqlx. From those four deps alone, and before I have written a single line of code, I now have a three hundred and fifty-two line long `cargo tree`. There are three things to note on this.

First, and most obviously, this is far too many dependencies of dependencies, and this is before I add in a lot of other crates I'll need to work with this stack, or for my business logic. I can't seriously rely on - or even keep up with - a dep tree like this for a single and relatively simple project.

Second, a lot of these dependencies are made by anonymous users on github, are documented solely on Discord, change frequently, and do not appear to warrant a high degree of confidence. If Rust continues to gain traction, we will absolutely see security issues à la Pollyfill.JS or xz. For my livelihood, I'm necessarily going to place more trust in Microsoft's ASP.NET or VMWare's Spring than in @animewaifu420's three dozen half-abandoned packages and their 'dank' Discord.

Third, despite pulling in twenty-five times more dependencies than Alpine requires to run an OS, this is actually an extremely barebones web stack. No code-first database migrations, no authentication or authorisation, no internationalisation, no named routes, just to take some random basic features - all of this requires further crates (and, in turn, many dozens of further dependencies of dependencies.)

I'm using web as an illustrative example here, but this is an issue that permeates the ecosystem. I've faced similar issues for other use cases as well.

I like your trade-off framing, but I respectfully think it comes out strongly against Rust from an end user's perspective, at least at this point in Rust's history. I also think it elides over something important - the fact that Rust need not actually be like this. There's no reason Rust couldn't, in the future, have fully-featured crates, with dramatically curtailed dep trees and wide industry buy-in. But it's just not there at this stage. And so I return to my initial post - I quite like Rust, I've previously used it for years, but the ecosystem is years away from a sufficient level of maturity for me to feel inclined to return to it for any significant work at this stage.

Re: I sped up serde_json strings by 20%

#116

Earlier quoted context omitted.

Let's be real about dependencies https://wiki.alopex.li/LetsBeRealAboutDependencies

> A medium-sized Rust project can easily tip the scales at 2-300 crates, which is still rather more dependencies than anything I’ve looked at here, but that’s explained by the simple fact that using libraries in C is such a monumental pain in the ass that it’s not worth trying for anything unless it’s bigger than… well, a base64 parser or a hash function. It's odd that this article spends so much time arguing that Ru…

Odd indeed.

It's also worth noting that the article has a real apples != oranges problem. Things like 'libsystemd' are not in any way comparable to your typical Rust crate.

Re: I sped up serde_json strings by 20%

#117
post #115

Earlier quoted context omitted.

tokio and serde are certainly not the only ones. You can put almost all of my crates into that category too. The problem with your framing is that you look at this as a "dependency addiction." But that doesn't fully explain everything. The `regex` crate is a good case study. If it were a C library, it would almost certainly have zero dependencies. But it isn't a C library. It exists in a context where I can encapsula…

Super interesting take, thanks for sharing your thoughts. I like your trade-off framing, but I do think that your argument fails to take account of something important here, which I'll try to illustrate with an example. Let's say I want to run a simple modern web stack. In C#, I might pull in ASP.NET and EF Core. In Java, I might pull in Spring. In Rust, I might pull in axum, axum-server, rustls and sqlx. From those…

> I can't seriously rely on - or even keep up with - a dep tree like this for a single and relatively simple project.

Our dependency trees in `uv` and `ruff` at Astral are bigger than this. We rely on it and keep up with it. We do not spend most of our time dealing with the dependency tree. (Although I do think our compile times are longer than would be nice.)

> For my livelihood, I'm necessarily going to place more trust in Microsoft's ASP.NET or VMWare's Spring than in @animewaifu420's three dozen half-abandoned packages and their 'dank' Discord.

If this ends up being true, then that means there is financial incentive to provide something more holistic, right?

I don't know what "dank" Discords have to do with anything here.

From my perspective, I see a lot of derision and vague feelings in your commentary, but little hard data. Tons of production web applications are built in NodeJS on orders-of-magnitude larger dependency trees. Tons of peoples' livelihoods depend on it.

To be honest, I have a lot of the same feelings as you. I don't like large dependency trees either. But that tends to be more connected to tangible problems like compile times.

> There's no reason Rust couldn't, in the future, have fully-featured crates, with dramatically curtailed dep trees and wide industry buy-in.

There's no reason C couldn't have better dependency management either, right? But it doesn't. And the problem isn't just technical.

> but the ecosystem is years away from a sufficient level of maturity for me to feel inclined to return to it for any significant work at this stage.

I suspect the ecosystem is more mature than you think it is, particularly if your understanding of maturity boils down to just a single number of dependencies. But like, if you had written this comment originally, I probably wouldn't have replied. I don't think what you're saying is totally unreasonable, but I also don't see this changing either. Like, if Microsoft wanted to come in and build a nice web framework for Rust, I think it's very likely it would be built on top of the existing crate ecosystem. I don't think they would, for example, roll their own HTTP implementation unless there was some technical reason that `hyper` wouldn't work for them. I do not think they would do it just to get the dependency count down.

What you're seeing is a manifestation of a million tiny little decisions. Should I write my own URL parser? Or should I use a crate for it that I know is also used in production by a number of large companies? Why would you choose to roll your own? (Again, excepting technical reasons.) You wouldn't. You would use the one that already exists. And this decision procedure repeats over-and-over. Big companies aren't immune to it either.

In any case, I did say "trade off" and not "Rust's way is always better." So my reply makes room for viewpoints like yours, where as I think your original comment didn't make room for a more balanced take. That's why I responded.

Re: I sped up serde_json strings by 20%

#118
post #115

Earlier quoted context omitted.

Super interesting take, thanks for sharing your thoughts. I like your trade-off framing, but I do think that your argument fails to take account of something important here, which I'll try to illustrate with an example. Let's say I want to run a simple modern web stack. In C#, I might pull in ASP.NET and EF Core. In Java, I might pull in Spring. In Rust, I might pull in axum, axum-server, rustls and sqlx. From those…

> I can't seriously rely on - or even keep up with - a dep tree like this for a single and relatively simple project. Our dependency trees in `uv` and `ruff` at Astral are bigger than this. We rely on it and keep up with it. We do not spend most of our time dealing with the dependency tree. (Although I do think our compile times are longer than would be nice.) > For my livelihood, I'm necessarily going to place more…

Responding to the Discord point, charitably, I think the person you're replying to is saying that Discord is a poor documentation platform that has some fatal flaws, and I do think it has unfortunately replaced web forums in a lot of contexts, which means a couple things:

1. It's unsearchable and opaque: you can't really do anything but the most basic of searches on Discord and getting support or finding previous/similar questions can be incredibly difficult. This also has the strong disadvantage of being unarchivable with makes the next issue more important.

2. It's controlled by a single entity and the viability of Discord, from both a service provider perspective and a community perspective is uncertain. For all we know, Discord, the company, could die without any warning, or there could be policy decisions which make hosting communities untenable. If that happens, it'd be incredibly difficult to get stuff off of there. With more decentralized communities, or even centralized, but internet-visible ones, there are potentially archives and backups that can help restore information.

This, imo, makes it not a great choice for a software project's forum or whatever. Maybe general IRC-style chat where conversations tend to be relatively ephemeral anyway? Sure. But it's not a great forum or wiki replacement imo.

Re: I sped up serde_json strings by 20%

#119

Earlier quoted context omitted.

Almost nobody is shopping around Rust JSON libraries unless they need some specific feature not provided by serde and serde_json. They are the default everyone reaches for.

The fact that a JSON parser is the sort of ecological niche where you naturally get a single best library which does basically what everyone wants it to do with minimal dependencies is exactly the argument for putting it in the stdlib, though.

That would require moving the entirety of serde and its traits (or equivalent) into the standard library. Considering how long it takes to stabilize basic things in the stdlib, I think that’s a terrible idea.

IMO Rust has the best tradeoff between stdlib and ecosystem libraries I’ve ever seen in a (compiled) language, and that includes serde/serde_json.

Re: I sped up serde_json strings by 20%

#120
post #112

Earlier quoted context omitted.

> This is really no different from any other language. There are languages with big standard libraries and first party frameworks. I can build a complex web app in C# using only packages published by Microsoft in ASP.NET and EF. Python ships with a lot of "batteries included". Not saying I expect that from rust considering the funding/team size discrepancy and language targets - but I disagree that every language is…

Moment you add something to std it's effectively died. Strong backwards guarantees will choke development of library. Python std is where libraries go to die.

Who says that's a bad thing ? Sometimes things are good enough and just need to be vetted and maintained.
Post reply on HN