Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I sped up serde_json strings by 20%

purplesyringa.moe

Post reply on HN
Print view View original

31–40 of 124 posts

Re: I sped up serde_json strings by 20%

#31

Post by duped »

38 wrote: post #8
zadokshi wrote: post #3

Serde json has 3gb of dependencies once you do a build for debug and a build for release. Use serde on a few active projects and you run out of disk space. I don’t know why json parsing needs 3gb of dependencies. I’m all for code reuse but Serde for json is a bit of a dogs breakfast when it comes to dependencies. all you need is an exploit in on of those dependencies and half of the rust ecosystem is vulnerable. Rust…

this 100%. serde is a bloated monster, its sad that its the popular JSON, because all it does is make Rust look bad in my opinion. here are some smaller options: https://lib.rs/crates/humphrey_json https://lib.rs/crates/rust_json https://lib.rs/crates/sj

The value prop isn't serde_json, it's automatically generated serializers and deserializers for structured data without needing an extra codegen step like with protobufs/capnproto, plus all that machinery decoupled from the actual data format you're reading.

It essentially generates a massive amount of code that you need to write anyway, at the cost of code size and compile time. And a lot of people are happy to make that trade off.

I wouldn't call that a "bloated monster" because of that. Also, none of those options are alternatives to serde_json, unless you restrict yourself to serde_json::Value - which no one does in practice.

Re: I sped up serde_json strings by 20%

#32

Post by runevault »

echelon wrote: post #29
troad wrote: post #16

Earlier quoted context omitted.

Dependency bloat is an issue with Rust in general. The dependency trees for any meaty Rust project quickly become pretty horrifying. Auditing all these dependencies is infeasible, and my level of confidence in a lot of them is fairly low. I worked with Rust for a few years, and with the benefit of a few years' experience, I don't think I'll be touching Rust again until the ecosystem matures a great deal (which will o…

> The dependency trees for any meaty Rust project quickly become pretty horrifying. s/Rust// This is really no different from any other language. At least Rust, with Cargo, makes it easy to scan your dependencies. And many notable Rust projects attempt to keep third party dependencies to a minimum. C++ gives you absolutely nothing to work with. Other languages with package managers don't keep dependency trees shallow…

I'm not sure languages without a package manager being the default are nearly as bad.

When it is trivial (like running a single command line or a single line in a file like cargo.toml), having a proliferation of dependencies is so easy it almost becomes guaranteed. But in languages like c++ where not everyone uses them (even if several PMs exist) adding a dependency to a library is a much bigger deal as you either have to use something like git submodules or manage them yourself OR make any user of your library go get the correct versions themselves.

Re: I sped up serde_json strings by 20%

#33

Post by SkiFire13 »

troad wrote: post #16
zadokshi wrote: post #3

Serde json has 3gb of dependencies once you do a build for debug and a build for release. Use serde on a few active projects and you run out of disk space. I don’t know why json parsing needs 3gb of dependencies. I’m all for code reuse but Serde for json is a bit of a dogs breakfast when it comes to dependencies. all you need is an exploit in on of those dependencies and half of the rust ecosystem is vulnerable. Rust…

Dependency bloat is an issue with Rust in general. The dependency trees for any meaty Rust project quickly become pretty horrifying. Auditing all these dependencies is infeasible, and my level of confidence in a lot of them is fairly low. I worked with Rust for a few years, and with the benefit of a few years' experience, I don't think I'll be touching Rust again until the ecosystem matures a great deal (which will o…

Let's be real about dependencies https://wiki.alopex.li/LetsBeRealAboutDependencies

Re: I sped up serde_json strings by 20%

#34

Post by cedws »

hermanradtke wrote: post #6
zadokshi wrote: post #3

Serde json has 3gb of dependencies once you do a build for debug and a build for release. Use serde on a few active projects and you run out of disk space. I don’t know why json parsing needs 3gb of dependencies. I’m all for code reuse but Serde for json is a bit of a dogs breakfast when it comes to dependencies. all you need is an exploit in on of those dependencies and half of the rust ecosystem is vulnerable. Rust…

Please show your work. I cannot reproduce "3gb of dependencies". Here is my test: Cargo.toml [package] name = "serde-test" version = "0.1.0" edition = "2021" [dependencies] serde = { version = "1.0.208", features = ["derive"] } serde_json = "1.0.127" src/main.rs use serde::Deserialize; #[derive(Deserialize)] struct Foo { bar: String, } fn main() { let foo: Foo = serde_json::from_str("\"bar\": \"baz\"").unwrap(); prin…

Jesus, 78MB is still a lot for such a simple program.

Re: I sped up serde_json strings by 20%

#35

Post by 38 »

duped wrote: post #31
38 wrote: post #8

Earlier quoted context omitted.

this 100%. serde is a bloated monster, its sad that its the popular JSON, because all it does is make Rust look bad in my opinion. here are some smaller options: https://lib.rs/crates/humphrey_json https://lib.rs/crates/rust_json https://lib.rs/crates/sj

The value prop isn't serde_json, it's automatically generated serializers and deserializers for structured data without needing an extra codegen step like with protobufs/capnproto, plus all that machinery decoupled from the actual data format you're reading. It essentially generates a massive amount of code that you need to write anyway, at the cost of code size and compile time. And a lot of people are happy to make…

> none of those options are alternatives to serde_json, unless you restrict yourself to serde_json::Value - which no one does in practice.

check your facts, all the above options have derive support, serde is not special in that.

Re: I sped up serde_json strings by 20%

#36

Post by ComputerGuru »

cedws wrote: post #34
hermanradtke wrote: post #6

Earlier quoted context omitted.

Please show your work. I cannot reproduce "3gb of dependencies". Here is my test: Cargo.toml [package] name = "serde-test" version = "0.1.0" edition = "2021" [dependencies] serde = { version = "1.0.208", features = ["derive"] } serde_json = "1.0.127" src/main.rs use serde::Deserialize; #[derive(Deserialize)] struct Foo { bar: String, } fn main() { let foo: Foo = serde_json::from_str("\"bar\": \"baz\"").unwrap(); prin…

Jesus, 78MB is still a lot for such a simple program.

That’s not the size of the program but the size of the build artifacts folder that includes all the intermediate files like .o in a C project and more.

Re: I sped up serde_json strings by 20%

#37

Post by what »

ComputerGuru wrote: post #36
cedws wrote: post #34

Earlier quoted context omitted.

Jesus, 78MB is still a lot for such a simple program.

That’s not the size of the program but the size of the build artifacts folder that includes all the intermediate files like .o in a C project and more.

That still seems like a lot of build artifacts for a 10 line program?

Re: I sped up serde_json strings by 20%

#38

Post by fyrn_ »

38 wrote: post #9
spense wrote: post #2

awesome that serde moves so quickly. i just ran across simdutf8 and realized the pr for simd-enabled uft8 parsing is coming up on 5 years: https://github.com/rust-lang/rust/issues/68455

https://wikipedia.org/wiki/Poe's_law

The parent is comparing the speed of Rust std improvement to the faster pace of Serde.

Re: I sped up serde_json strings by 20%

#39

Post by throwup238 »

runevault wrote: post #32
echelon wrote: post #29

Earlier quoted context omitted.

> The dependency trees for any meaty Rust project quickly become pretty horrifying. s/Rust// This is really no different from any other language. At least Rust, with Cargo, makes it easy to scan your dependencies. And many notable Rust projects attempt to keep third party dependencies to a minimum. C++ gives you absolutely nothing to work with. Other languages with package managers don't keep dependency trees shallow…

I'm not sure languages without a package manager being the default are nearly as bad. When it is trivial (like running a single command line or a single line in a file like cargo.toml), having a proliferation of dependencies is so easy it almost becomes guaranteed. But in languages like c++ where not everyone uses them (even if several PMs exist) adding a dependency to a library is a much bigger deal as you either ha…

> I'm not sure languages without a package manager being the default are nearly as bad.

The autoconf or cmake list of included libraries for many non-trivial C++ projects is usually just as long as stuff in Cargo, especially when breaking up Boost, Qt, or other megaframeworks into individual libraries.

They do "make any user of your library go get the correct versions themselves" but that has been far less of a problem this century thanks to OS package management.

Re: I sped up serde_json strings by 20%

#40

Post by commodoreboxer »

what wrote: post #37
ComputerGuru wrote: post #36

Earlier quoted context omitted.

That’s not the size of the program but the size of the build artifacts folder that includes all the intermediate files like .o in a C project and more.

That still seems like a lot of build artifacts for a 10 line program?

A 10 line program with two dependencies and all their transitive dependencies.
Post reply on HN