Live data from Hacker News

I sped up serde_json strings by 20%

purplesyringa.moe

61–70 of 124 posts

Re: I sped up serde_json strings by 20%

#61
post #58
post #37

Earlier quoted context omitted.

That still seems like a lot of build artifacts for a 10 line program?

It deserializes a unicode string to a custom structure. Do not mistake it with C character-shuffling hello-world-programs. Edit: s/to JSON/to a custom structure/

I’m on mobile so I can’t check at the moment. But I’d be shocked if the equivalent go binary was anywhere near as big, or took anywhere near as long to build.

I’ll check later

Re: I sped up serde_json strings by 20%

#62
post #14
post #3

Serde json has 3gb of dependencies once you do a build for debug and a build for release. Use serde on a few active projects and you run out of disk space. I don’t know why json parsing needs 3gb of dependencies. I’m all for code reuse but Serde for json is a bit of a dogs breakfast when it comes to dependencies. all you need is an exploit in on of those dependencies and half of the rust ecosystem is vulnerable. Rust…

Rust emits unreasonable amount of debug information. It's so freakishly large, I expect it's just a bug. Anything you compile will dump gigabytes into the target folder, but that's not representative of the final product (after stripping the debug info, or at least using a toned-down verbosity setting).

> Rust emits unreasonable amount of debug information. It's so freakishly large, I expect it's just a bug.

Rust relies on the linker (via -ffunction-sections and -gc-sections) to delete functions that aren't ever used but the linker isn't capable of removing the corresponding debug info.

https://github.com/rust-lang/rust/issues/56068

Re: I sped up serde_json strings by 20%

#63
post #60
post #57

> Teaching to _think_ is just as important as teaching to code, but this is seldom done Oh, the arrogance of thinking that the other person doesn't think.

It isn't what author argues at all. It's about teaching how to think: that you have to do some research and it's not a step you can skip; having done your research doesn't free you from having to draw your own conclusions too. Skipping either of those steps is easy but wrong.

> is seldom done

It's this part

Re: I sped up serde_json strings by 20%

#65
post #60

Earlier quoted context omitted.

It isn't what author argues at all. It's about teaching how to think: that you have to do some research and it's not a step you can skip; having done your research doesn't free you from having to draw your own conclusions too. Skipping either of those steps is easy but wrong.

> is seldom done It's this part

Perhaps the wording was off on my part. What I meant is not that people don't think, it's that people seldom teach others to think, at least in web articles.

Most posts of such format I have seen are "we did this and got this", not "we tried this, it failed because of this, then we figured out something else might work and it worked after these modifications".

Re: I sped up serde_json strings by 20%

#66
post #29
post #16

Earlier quoted context omitted.

Dependency bloat is an issue with Rust in general. The dependency trees for any meaty Rust project quickly become pretty horrifying. Auditing all these dependencies is infeasible, and my level of confidence in a lot of them is fairly low. I worked with Rust for a few years, and with the benefit of a few years' experience, I don't think I'll be touching Rust again until the ecosystem matures a great deal (which will o…

> The dependency trees for any meaty Rust project quickly become pretty horrifying. s/Rust// This is really no different from any other language. At least Rust, with Cargo, makes it easy to scan your dependencies. And many notable Rust projects attempt to keep third party dependencies to a minimum. C++ gives you absolutely nothing to work with. Other languages with package managers don't keep dependency trees shallow…

> This is really no different from any other language.

There are languages with big standard libraries and first party frameworks.

I can build a complex web app in C# using only packages published by Microsoft in ASP.NET and EF.

Python ships with a lot of "batteries included".

Not saying I expect that from rust considering the funding/team size discrepancy and language targets - but I disagree that every language is same in this regard - JS/Node is notoriously bad, Rust is around C++ level, and plenty of higher level languages have first pary/standard library stacks.

Re: I sped up serde_json strings by 20%

#67
Very strong jart feel about this person's blog, that was a nice read

> We would need to reinvent the wheel, but this is quite neat if you think about it.

Is this real or ironic though? I read it and started laughing at the writer but the rest of the page seems quite heavy on self-deprecation

Re: I sped up serde_json strings by 20%

#68
post #42
post #29

Earlier quoted context omitted.

> The dependency trees for any meaty Rust project quickly become pretty horrifying. s/Rust// This is really no different from any other language. At least Rust, with Cargo, makes it easy to scan your dependencies. And many notable Rust projects attempt to keep third party dependencies to a minimum. C++ gives you absolutely nothing to work with. Other languages with package managers don't keep dependency trees shallow…

> And many notable Rust projects attempt to keep third party dependencies to a minimum. I don't think this is true. The only two major Rust crates that manage to keep their dependencies light are tokio and serde, and these are highly atypical projects. For a more typical example, look at something like axum (running `cargo tree` for project with a single dependency on axum returns 121 lines). > This is really no diff…

tokio and serde are certainly not the only ones. You can put almost all of my crates into that category too.

The problem with your framing is that you look at this as a "dependency addiction." But that doesn't fully explain everything. The `regex` crate is a good case study. If it were a C library, it would almost certainly have zero dependencies. But it isn't a C library. It exists in a context where I can encapsulate separately versioned libraries as dependencies with almost no impact on users of `regex`. Namely, it has two required dependencies: regex-syntax and regex-automata. It also has two optional dependencies: memchr and aho-corasick.

This isn't a case of the regex crate farming out its core functionality to other projects. Indeed, it started as a single crate. And I split its code out into separately versioned crates that others can now use. And this has been a major ecosystem win:

* memchr is used in all sorts of projects, and it promises to give you exactly the same implementation of substring search that the regex crate (and also ripgrep) use in your own projects. Indeed, that crate is used here! What would you do instead? If you were in C-land, you'd re-roll all of the specialized SIMD that's in memchr? For x86-64, aarch64 and wasm32 right? If you haven't done that sort of thing before, good luck. That'll be a long ramp-up time.

* aho-corasick is packaged as a stand-alone Python library that is quite a bit faster than pyahocorasick: https://pypi.org/project/ahocorasick-rs/ There's tons of other projects on crates.io relying on aho-corasick specifically, separately from how its used inside of `regex`.

* regex-syntax gives you a production grade regex parser. More than that, it gives you exactly the same parser used by the regex crate. People have used this for all sorts of things, including building their own regex engine without needing to re-create the parser (which is a significant simplification).

* regex-automata gives you access to all of the internal APIs of the regex engine. This is all the stuff that is too complex to put into a general purpose regex library targeting the 99% use case. As far as I know, literally no other general purpose regex engine has ever attempted this because most regex engines are written in C or C++ where you'd be laughed out of the room for suggesting it because dependency management is such a clusterfuck. Yet, this has been a big benefit to other folks. The Yara project uses it for example, and the Helix editor uses it to search discontiguous strings: https://github.com/helix-editor/helix/pull/9422 (Instead of rolling your own regex engine, which is what I believe vim does.)

This isn't dependency addiction. This is making use of separately versioned libraries to allow other projects to depend on battle tested components independent of their primary use case. Yet, if people repeat this kind of process---exposing internals like I did with the regex crate---then you wind up with a bigger dependency tree.

Good dependency management is a trade-off. One the one hand, it enables the above to happen, which I think is an objectively Good Thing. But it also enables folks to depend on huge piles of code so easily that it actively discourages someone from writing their own base64 implementation. But as should be obvious, it doesn't prevent them from doing so: https://github.com/BurntSushi/ripgrep/blob/ea99421ec896fcc9a...

Good dependency management is Pandora's box. It has been opened and it is never going to get closed again. Just looking on and calling it an addiction isn't going to take us anywhere. Instead, let's look at it as a trade-off.

Re: I sped up serde_json strings by 20%

#69
post #67

Very strong jart feel about this person's blog, that was a nice read > We would need to reinvent the wheel, but this is quite neat if you think about it. Is this real or ironic though? I read it and started laughing at the writer but the rest of the page seems quite heavy on self-deprecation

What does jart mean?

Re: I sped up serde_json strings by 20%

#70
post #10
post #3

Serde json has 3gb of dependencies once you do a build for debug and a build for release. Use serde on a few active projects and you run out of disk space. I don’t know why json parsing needs 3gb of dependencies. I’m all for code reuse but Serde for json is a bit of a dogs breakfast when it comes to dependencies. all you need is an exploit in on of those dependencies and half of the rust ecosystem is vulnerable. Rust…

I swear the target folder for literally any project of any scale is at least several GB in size.

I get a progress bar when I run `cargo clean` because it's so large.
Post reply on HN