An interesting library I've found that tackles this issue is
yieldable-json: https://developer.ibm.com/node/2017/11/13/yieldable-json-asy...
The cost of parsing JSON
291–300 of 308 posts
Re: The cost of parsing JSON
#292Earlier quoted context omitted.
Parsing 1GB of flat JSON data is equal to 1.61 metric cow farts.
Don't tell people that. Those who believe climate change is a hoax will then do it more out of spite against the alleged hoaxers. Think I'm joking?: https://www.scientificamerican.com/article/not-so-conservati...
Not having a sense of humour will hurt your cause more than anything.
Given the choice between a secret fool who has fun, and a joyless thought-policing jerk who happens to be right on an issue, people will choose the fool every time; and frankly I can't blame them.
Re: The cost of parsing JSON
#293Earlier quoted context omitted.
Are you including the cost of loading protobuf itself? You seem to be basing your argument on an assumed already present & loaded protobuf library. You need to benchmark starting from nothing at all. Your link that you seem to be basing this off of has a loaded and fully JIT'd protobuf. That's not the start state.
You can measure the impact on loading time, and the size of the protobuf implementation you're using probably has an impact on the threshold at which it becomes more efficient. I don't doubt that parsing a 500 character long JSON string is probably faster than loading a protobuf to do it instead. In fact, apparently this JSON parsing trick is only effective beyond 10K or so. But past a certain threshold memory bandwi…
You’re repeatedly missing the point. This is about optimizing startup time.
The comparison should be:
cost of downloading payload + runtime cost of parsing JSON
Vs
cost to download protobuf lib + parse and execute JS protobuf lib + download payload + runtime cost of parsing
Specifically, the article talks about how parsing JS is more costly than JSON - this cost will apply to the protobuf library which certainly far exceeds 10KB. There is no way the math will work on your favor until you get to MBs of data.
Re: The cost of parsing JSON
#294Earlier quoted context omitted.
For typical cases JavaScript protos (jspb) should use JSON wire format. You would only use binary wire format depending on whether your message type is suited to it, e.g., lots of internal byte arrays.
Lots of repetitive data with a relatively flat structure can be a good argument to get away from JSON, too. Let's set aside binary formats for a moment. I once sped up populating a large-ish table of data by an order of magnitude - and achieved a pretty decent reduction in data volume, too - just by switching the format to CSV.
Is CSV so different than JSON?
Before:
["my", "data", 1, 2, 3]
After:
my,data,1,2,3
Re: The cost of parsing JSON
#295Earlier quoted context omitted.
We're talking about JavaScript in the browser though... what other message format is more readily and performantly processed in-browser using JavaScript than JSON?
Once WebAssembly gains APIs to change the DOM quickly and the big JS frameworks switch to WebAssembly for their internal engines, you could make the case for usage of binary formats like protobuf. IMO this trend of piling on technology after technology to handle bloat instead of designing websites to be lean is wrong but it's certainly the direction we are walking into. Websites will become even more opaque and compl…
> Websites will become even more opaque and complex. Definitely not looking forward to it.
Why is that an issue? Do websites need to be open source? How many people, including software developers, will actually view the source for a 3rd party website and/or try to debug it? Beyond screen-scraping and learning purposes I don't see a use case for it.
I'd be quite happy with my browser(s) downloading and executing binary blobs if it means better usage of my devices' resources and bandwidth.
Re: The cost of parsing JSON
#296I was hoping this blog post would be about the Chrome team speeding up JSON parsing. On a website I made[1] with a ~1MB JSON ajax payload, Chrome takes about 18 seconds to parse the JSON, while Safari parses it pretty much instantaneously. [1] https://helpmepropose.live EDIT: Does anyone know if there's something I could do differently to speed this up in Chrome?
Why not use binary data? A typed array should work (unsure of the exact APIs, though).
That's a good idea, thanks! I'd previously thought typed arrays had to be initialized with data that's already in memory, but it looks like I was wrong there. This page is giving some hints that it can be done: https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequ...
Re: The cost of parsing JSON
#297Earlier quoted context omitted.
Why not just use promises? side note: legit question, I don't do web/app dev
Promises still run in the main thread. You could try to use a web worker, but then you run into the problem that they don't have shared memory, so you need to pass data back some other way.
IndexedDB runs on web workers, so that might be a good way to push the work off the main thread but still share the result.
Re: The cost of parsing JSON
#298Earlier quoted context omitted.
Parsing 1GB of flat JSON data is equal to 1.61 metric cow farts.
Don't tell people that. Those who believe climate change is a hoax will then do it more out of spite against the alleged hoaxers. Think I'm joking?: https://www.scientificamerican.com/article/not-so-conservati...
Everybody knows it's the nautical cow farts that contribute over 80% of JSO2N emissions. It's causing ACID compliant rain.
Re: The cost of parsing JSON
#299Earlier quoted context omitted.
To be honest, I understand the bit of backlash that I’ve received here and I think it’s well-deserved since I should’ve worded my statement better. Thank you for your comments. You all are correct re firing someone over mistakes and seemingly trivial matters. I was mostly referring to software engineers who make impactful decisions without good reason and/or without properly assessing the trade-offs. I think it’s fai…
I've found that the readability of fast code vs. slow code is often negligible - certainly it is in the specific example under discussion. I prefer to make a habit of using faster idioms in that case, so that when speed does matter I'm already covered. I don't consider that premature optimization.
Except it isn't, because most of your developers will be using an environment that includes syntax highlighting and probably some linting. Except within string literals.
Code inside string literals is less readable and more inclined to be wrong/buggy.
Re: The cost of parsing JSON
#300Earlier quoted context omitted.
Because JSON.parse blocks the thread it's in, and JS is single threaded [1]. So even if you put it behind a promise, when that promise actually runs, it will block the thread. In essence, using promises (or callbacks or timeouts or anything else like that) allows you to delay the thread-blocking, but once the code hits `JSON.parse`, no other javascript will run until it completes. And since no other javascript will r…
Another trick most people don’t realize is that not only is the fetch api is asynchronous but response.json() does the conversion in a background thread and is non UI blocking. If you have a large json object. You can use the fetch api to work with it. If you need to cache it, use the cache storage api. Unlike localStorage which will freeze the UI, cache storage wont. It’s slightly slower since it needs to talk to an…
This is a common misconception, but response.json() still blocks the main thread.
It looks like it doesn't, but the same exact symptoms will happen even while awaiting the fetch json().