101–110 of 159 posts
Author of the blog here. I had a great time writing this. By far the most complex article I've ever put together, with literally thousands of lines of js to build out these interactive visuals. I hope everyone enjoys.
I don't see a single visual. I don't use the web with javascript. Why not embed static images instead or in addition?
Author of the blog here. I had a great time writing this. By far the most complex article I've ever put together, with literally thousands of lines of js to build out these interactive visuals. I hope everyone enjoys.
Author of the blog here. I had a great time writing this. By far the most complex article I've ever put together, with literally thousands of lines of js to build out these interactive visuals. I hope everyone enjoys.
I don’t see any animations on Safari. Also, I’d much prefer a variable-width font, monospace prose is hard to read. While I can use Reader Mode, that removes the text coloring, and would likely also hide the visuals (if they were visible in the first place).
Earlier quoted context omitted.
"...has orders of magnitude less latency than..." [citation needed]. Local network access shouldn't be much different than local IPC.
> Local network access In what production scenarios do MySQL, Postgres, DB2, Oracle, et. al., live on the same machine as the application that uses them? I am pretty sure most of these vendors would offer strict guidance to not do that.
Earlier quoted context omitted.
> I've been advocating for SQLite+NVMe for a while now. Why SQLite instead of a traditional client-server database like Postgres? Maybe it's a smidge faster on a single host, but you're just making it harder for yourself the moment you have 2 webservers instead of 1, and both need to write to the database. > Latency is king in all performance matters. This seems misleading. First of all, your performance doesn't matt…
Until you hit the single-writer limitation in SQLite, you do not need to spend more CPU cycles on Postgres
Earlier quoted context omitted.
Sqlite doesn't work super well with parallelism in writing. It supports it, yes, but in a bit clunky way and it still can fail. To avoid problems with parallel writing besides setting a specific clunky mode of operations a trick of using a single thread for writing in an app can be used. Which usually makes the already complicated parallel code slightly more complicated. If only one thread of writing is required, the…
> If only one thread of writing is required, then SQLite works absolutely great. The whole point of getting your commands down to microsecond execution time is so that you can get away with just one thread of writing. Entire financial exchanges operate on this premise.
I love the visuals, and if it's ok with you will probably link them to my class material on block devices in a week or so. One small nit: > A typical random read can be performed in 1-3 milliseconds. Um, no. A 7200 RPM platter completes a rotation in 8.33 milliseconds, so rotational delay for a random read is uniformly distributed between 0 and 8.33ms, i.e. mean 4.16ms. >a single disk will often have well over 100,00…
Whoah, thanks for sharing the paper.
I’m still annoyed they didn’t include the drain time equation I used for calculating track width, which falls out of one of their equations.
Oh, and I’m very glad you showed differing track sizes across the platter. (BTW, did you know track sizes differ between platters? Google “disks are like snowflakes”)
Earlier quoted context omitted.
The entire point is to avoid the network hop. Application SQLite NVMe has orders of magnitude less latency than Application Postgres Client Network Postgres Server NVMe > You should be avoiding serial database queries as much as possible in the first place. I don't get to decide this. The business does.
"...has orders of magnitude less latency than..." [citation needed]. Local network access shouldn't be much different than local IPC.
Earlier quoted context omitted.
> I've been advocating for SQLite+NVMe for a while now. Why SQLite instead of a traditional client-server database like Postgres? Maybe it's a smidge faster on a single host, but you're just making it harder for yourself the moment you have 2 webservers instead of 1, and both need to write to the database. > Latency is king in all performance matters. This seems misleading. First of all, your performance doesn't matt…
Until you hit the single-writer limitation in SQLite, you do not need to spend more CPU cycles on Postgres
For small traffic, it's pretty simple to run it on the same host as web app, and unix auth means there are no passwords to manage. And once you need to have multiple writers, there is no need to rewrite all the database queries.
Earlier quoted context omitted.
I still measure 1-2ms of latency with an NVMe disk on my Desktop computer, doing fsync() on a file on a ext4 filesystem. Update: about 800us on a more modern system.
Not so sure that's true. This is single-threaded direct I/O doing a fio randwrite workload on a WD 850X Gen4 SSD: write: IOPS=18.8k, BW=73.5MiB/s (77.1MB/s)(4412MiB/60001msec); 0 zone resets slat (usec): min=2, max=335, avg= 3.42, stdev= 1.65 clat (nsec): min=932, max=24868k, avg=49188.32, stdev=65291.21 lat (usec): min=29, max=24880, avg=52.67, stdev=65.73 clat percentiles (usec): | 1.00th=[ 33], 5.00th=[ 34], 10.00…