Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I sped up serde_json strings by 20%

purplesyringa.moe

Post reply on HN
Print view View original

81–90 of 124 posts

Re: I sped up serde_json strings by 20%

#81

Post by __s »

chipdart wrote: post #80
purplesyringa wrote: post #65

Earlier quoted context omitted.

Perhaps the wording was off on my part. What I meant is not that people don't think, it's that people seldom teach others to think , at least in web articles. Most posts of such format I have seen are "we did this and got this", not "we tried this, it failed because of this, then we figured out something else might work and it worked after these modifications".

> Most posts of such format I have seen are "we did this and got this", not "we tried this, it failed because of this, then we figured out something else might work and it worked after these modifications". That doesn't resemble anything remotely related to teaching how to think. You're just logging your trial and error process, which is exactly what each and every single developer goes through on a daily basis. What…

Again, they aren't saying developers don't think. They're talking about blogging

I had this issue at PeerDB where we'd blog about some dev, when I wrote it'd be a stream of consciousness trying to communicate the mood, frustration, & flailing process. It wouldn't get published, in favor of blogs with clearer product messaging

Re: I sped up serde_json strings by 20%

#82

Post by pezezin »

maccard wrote: post #61
loa_in_ wrote: post #58

Earlier quoted context omitted.

It deserializes a unicode string to a custom structure. Do not mistake it with C character-shuffling hello-world-programs. Edit: s/to JSON/to a custom structure/

I’m on mobile so I can’t check at the moment. But I’d be shocked if the equivalent go binary was anywhere near as big, or took anywhere near as long to build. I’ll check later

I don't know what you consider big and long, but on my computer (Ryzen 5700X) it took 7 seconds to build, and the resulting binary is 556 kB.

Re: I sped up serde_json strings by 20%

#83

Post by anta40 »

wiseowise wrote: post #51
haberman wrote: post #44

Earlier quoted context omitted.

> A medium-sized Rust project can easily tip the scales at 2-300 crates, which is still rather more dependencies than anything I’ve looked at here, but that’s explained by the simple fact that using libraries in C is such a monumental pain in the ass that it’s not worth trying for anything unless it’s bigger than… well, a base64 parser or a hash function. It's odd that this article spends so much time arguing that Ru…

Because Rust has Cargo which makes it trivial to include dependencies. C developers constantly reinvent because C dependency management is such a joke that no one bothers.

I wonder if Go tooling does the job umm... "better" here. At least faster build time and debug build is much smaller.

At my current company, we handle payment, transaction etc with Go (some Fiber, some Echo). None of the projects reach 100 MB, and my pkg folder size is around 2.5 GB-ish. Those are the dependencies of all of my Go codebase. Well not bad.

Compare it with building a Rust sqlite web API which easily eat 3 GB disk. 10 similar projects may eat at least 30 GB.... :D

Disclaimer: I don't use Rust for work... yet. Only for personal tinkering.

Re: I sped up serde_json strings by 20%

#84

Post by tpmoney »

chipdart wrote: post #80
purplesyringa wrote: post #65

Earlier quoted context omitted.

Perhaps the wording was off on my part. What I meant is not that people don't think, it's that people seldom teach others to think , at least in web articles. Most posts of such format I have seen are "we did this and got this", not "we tried this, it failed because of this, then we figured out something else might work and it worked after these modifications".

> Most posts of such format I have seen are "we did this and got this", not "we tried this, it failed because of this, then we figured out something else might work and it worked after these modifications". That doesn't resemble anything remotely related to teaching how to think. You're just logging your trial and error process, which is exactly what each and every single developer goes through on a daily basis. What…

In context they're very clearly talking about blogging/write-ups/presentation of technical things. A lot of the material about making / fixing things we're presented with in life are finished products, the results clean and tidy, and the steps to accomplish the result obvious with the benefit of someone else to tell you what they are. It's much less common to see even a glimpse of the effort it took to get there, or for someone to document the process, including dead ends and false starts.

Even here, we can imagine that had the author failed to actually make anything faster, they might not have written anything at all. And yet, wouldn't that still have had benefit to people? To see things attempted that didn't work, to understand why those things didn't work? Maybe it wouldn't have been as interesting to as wide an audience, but it's important to see failure. Both as a way of learning from others to not repeat the same efforts, but also because its really easy to fall into the trap of assuming you're incapable if you do fail when everyone around you always seems to be succeeding.

Or perhaps as an analogy, almost everyone creates some art in life, and certainly every artist struggles to create that art. Yet it would be a disservice to only ever present art to learning artists as complete master works and paint by numbers replications. We need to see the "happy little accidents" of Bob Ross, the sketch books of iterations on a design, the piles of failed clay firings. Not because no one experiences these things, but because they are instructive on their own in a way that only seeing success is not.

Re: I sped up serde_json strings by 20%

#85

Post by charrondev »

anta40 wrote: post #83
wiseowise wrote: post #51

Earlier quoted context omitted.

Because Rust has Cargo which makes it trivial to include dependencies. C developers constantly reinvent because C dependency management is such a joke that no one bothers.

I wonder if Go tooling does the job umm... "better" here. At least faster build time and debug build is much smaller. At my current company, we handle payment, transaction etc with Go (some Fiber, some Echo). None of the projects reach 100 MB, and my pkg folder size is around 2.5 GB-ish. Those are the dependencies of all of my Go codebase. Well not bad. Compare it with building a Rust sqlite web API which easily eat…

To me the size of the codebase after installing dependencies isn’t that relevant. Are you starved for 30GB for your work? My current company has projects in PHP, rust, and a bunch of frontend projects with all the modern build tooling.

The largest service we deploy is ~300Mb, maybe 200Mb if you exclude copy of Monaco that we ship in the Dist folder. That web server will have terabytes of database storage behind it and 32 or 64 Gb of Redis/Memcached behind it. If we add in the Elasticsearch we’ve got another terabyte and a ton of memory.

If those dependencies aren’t checked into version control or being shipped to prod does it really matter?

Re: I sped up serde_json strings by 20%

#86

Post by anta40 »

charrondev wrote: post #85
anta40 wrote: post #83

Earlier quoted context omitted.

I wonder if Go tooling does the job umm... "better" here. At least faster build time and debug build is much smaller. At my current company, we handle payment, transaction etc with Go (some Fiber, some Echo). None of the projects reach 100 MB, and my pkg folder size is around 2.5 GB-ish. Those are the dependencies of all of my Go codebase. Well not bad. Compare it with building a Rust sqlite web API which easily eat…

To me the size of the codebase after installing dependencies isn’t that relevant. Are you starved for 30GB for your work? My current company has projects in PHP, rust, and a bunch of frontend projects with all the modern build tooling. The largest service we deploy is ~300Mb, maybe 200Mb if you exclude copy of Monaco that we ship in the Dist folder. That web server will have terabytes of database storage behind it an…

For an application developer like me? No. It's technically not a dealbreaker. But probably more like a question for compiler devs.

Re: I sped up serde_json strings by 20%

#87

Post by uecker »

wiseowise wrote: post #51
haberman wrote: post #44

Earlier quoted context omitted.

> A medium-sized Rust project can easily tip the scales at 2-300 crates, which is still rather more dependencies than anything I’ve looked at here, but that’s explained by the simple fact that using libraries in C is such a monumental pain in the ass that it’s not worth trying for anything unless it’s bigger than… well, a base64 parser or a hash function. It's odd that this article spends so much time arguing that Ru…

Because Rust has Cargo which makes it trivial to include dependencies. C developers constantly reinvent because C dependency management is such a joke that no one bothers.

C has no dependency management. There are various other package management you can use with C though. I am quite happy with my Linux distribution package manager.

But I have to say it clearly: cargo is a supply chain disaster and ever changing dependencies are major problem for Rust. Rust programs having many dependencies if not a good thing.

Re: I sped up serde_json strings by 20%

#88

Post by 7bit »

wtetzner wrote: post #74
zorked wrote: post #57

> Teaching to _think_ is just as important as teaching to code, but this is seldom done Oh, the arrogance of thinking that the other person doesn't think.

I don't think there's any arrogance in the statement. It doesn't assume others don't think. It's simply observing that most blog posts and how-to articles show the final result, but not necessarily the steps that were needed to get there.

I don't see what one has to do with the other.

Re: I sped up serde_json strings by 20%

#89

Post by eesmith »

hsbauauvhabzb wrote: post #54
Sytten wrote: post #20

The utf-8 tricks make me very nervous since I have seen too many attacks with parser confusion. I for with serde for correctness not speed. I hope this was fuzzed all the way with a bunch of invalid utf-8 strings.

Any bugs you can point to that come to mind of this class?

https://en.wikipedia.org/wiki/UTF-8#Invalid_sequences_and_er...

> Many of the first UTF-8 decoders would decode these, ignoring incorrect bits and accepting overlong results. Carefully crafted invalid UTF-8 could make them either skip or create ASCII characters such as NUL, slash, or quotes. Invalid UTF-8 has been used to bypass security validations in high-profile products including Microsoft's IIS web server[26] and Apache's Tomcat servlet container.[27] RFC 3629 states "Implementations of the decoding algorithm MUST protect against decoding invalid sequences."

Re: I sped up serde_json strings by 20%

#90

Post by IggleSniggle »

vecplane wrote: post #69
s_Hogg wrote: post #67

Very strong jart feel about this person's blog, that was a nice read > We would need to reinvent the wheel, but this is quite neat if you think about it. Is this real or ironic though? I read it and started laughing at the writer but the rest of the page seems quite heavy on self-deprecation

What does jart mean?

HN username for an extremely talented software engineer and software-engineering communicator, justine.lol. Probably most known around here for her cross-platform C code (cosmopolitan) and relatedly redbean, a zip file and tool that is also an executable file-server-file that hosts itself and can produce other such self-hosting cross platform executable zip file servers.
Post reply on HN