Live data from Hacker News

D1: Our SQL database

blog.cloudflare.com

191–200 of 241 posts

Re: D1: Our SQL database

#191
post #159

Earlier quoted context omitted.

No one really means anything ill and this political correctness madness needs to stop.

Quoted post unavailable.

It is most probably not meaningfully painful for people, but it does seem to deflect responsibility from actually helping marginalized people, while giving status to assholes who enjoy harassing others instead of contributing anything.

In some cases such as the git branch name, it also actively lost us many hours.

Re: D1: Our SQL database

#192
post #183

Earlier quoted context omitted.

Because it is being used for manipulation, falsely. "Oh, oh, this hurts me! I cannot bear it!" Recollect Emory University, in 2015, when someone wrote "TRUMP 2016" in chalk on the sidewalk. "Traumatized" protestors shouted "You are not listening! Come speak to us, we are in pain!" It's manipulation, pretending to be hurt. These people weren't locked in some kind of perpetual seizure for the four years of the Trump pr…

Can you explain what incentive someone has to "manipulate" us into no longer using the term slave, if not genuine discomfort?

It makes petty people feel they have power over you. It’s not like they are that wrong. We like to reward norm enforcing, but we aren’t sufficiently inoculated against exploitative ones. The end game can be seen in any big religion.

Re: D1: Our SQL database

#193

For a Cloudflare article, this one is surprisingly light on technical details. And for the product where it most matters. I'm guessing this is a single master database with multiple read replicas. That means it's not consistent anymore (the C in ACID). Obviously reads after a write will see stale data until the write propogates. I'm a bit curious how that replication works. Ship the whole db? Binary diffs of the mast…

Yeah, nothing about WAL mode which is what most users will want for web apps.

sqlite is accessed via a socket? defeats the whole purpose of using sqlite.

Many here are mentioning using one sqlite file per customer but that sounds like a nightmare for migrations and analytics.

SQLite is great and all these new services and articles are nice but intentionally shadowing lots of complexity.

Re: D1: Our SQL database

#194

wow SQLite getting a lot of love these days https://tailscale.com/blog/database-for-2022 https://fly.io/blog/all-in-on-sqlite-litestream https://blog.cloudflare.com/introducing-d1

SQLite is great but it's way overhyped and abused on HN. People are very eager to turn SQLite into a durable, distributed database and it's really not meant for that, and by going down that road instead of using something like MySQL or Postgres you're missing out on lots of important functionality and tooling. I only say this because I have made this mistake at my previous startup. We built these really cool distribu…

So you didn't use SQLite then? Because RocksDB + Kafka is not similar at all.

Also databases all use the same fundamental primitives and it's up to you to choose the level of abstraction you need. For example, FoundationDB is a durable distributed database that uses SQLite underneath as the storage layer but exposes an ordered key/value API, but then allows you to build your own relational DB on top.

If you just needed distributed SQL because a single instance wasn't enough then there are already plenty of choices like CockroachDB/Yugabyte/TiDB/Memsql/etc that can serve the purpose instead of building your own.

Re: D1: Our SQL database

#195

Earlier quoted context omitted.

B2 to Cloudflare also does not incur egress fees: https://www.backblaze.com/blog/backblaze-and-cloudflare-part... Backblaze B2 customers will be able to download data stored in B2 to Cloudflare for zero transfer fees. This happens automatically once Cloudflare is configured to distribute your B2 files.

Does anyone remember when we had Net Neutrality?

Yes, like it was yesterday (or today). It was a strange time where the term was often used for things that had nothing to do with the original meaning of the term.

Re: D1: Our SQL database

#196

If SQLite gets you excited, I'm building a firebase alternative based on sqlite. I'm betting hard on sqlite so this get's me super excited!! https://javascriptdb.com CF people around, I would love to chat, if anyone is interested please reach out at: jp@javascriptdb.com I'll be applying to this beta for sure!

Super interesting! I really like the idea. I'll join the beta, email sent :)

Re: D1: Our SQL database

#197
post #59

wow SQLite getting a lot of love these days https://tailscale.com/blog/database-for-2022 https://fly.io/blog/all-in-on-sqlite-litestream https://blog.cloudflare.com/introducing-d1

Well I don't think it's a good fit for regular service, exactly how do you handle 2 replicas of the same service talking to the same DB? The fact that it's just a file on disk limits the usage.

Transactions, locks, queues, etc. No different than multiple app instances changing the same row in other databases.

Any state mutation is ultimately ordered in time and how that that ordering is accomplished depends on the abstractions you're using: in your app, network layer, database, etc.

Re: D1: Our SQL database

#198
post #59

wow SQLite getting a lot of love these days https://tailscale.com/blog/database-for-2022 https://fly.io/blog/all-in-on-sqlite-litestream https://blog.cloudflare.com/introducing-d1

Well I don't think it's a good fit for regular service, exactly how do you handle 2 replicas of the same service talking to the same DB? The fact that it's just a file on disk limits the usage.

Is think one way to think about this is to have one database being tied to one replica (replicas could handle more than one database). Where (importantly) the idea would be one database for each user. You horizontally scale for the number of users, but each user is only using one end node.

It’s interesting because you have to consider how to scale your database as well as your application. The fact that you don’t have one central database opens up more possibilities. But it doesn’t work for all instances (such as a shared read-write data source for all users). For example, this approach wouldn’t work for something like Twitter (at least the original architecture).

Re: D1: Our SQL database

#199
post #184

Earlier quoted context omitted.

It would really help if SQLite3 had a `MERGE`, or, failing that, `FULL OUTER JOIN`. In fact, I want it to have `FULL OUTER JOIN` even if it gains a `MERGE`. `FULL OUTER JOIN` is the secret to diff'ing table sources. `MERGE` is just a diff operation + insert/update/delete statements to make the target table more like the source one (or even completely like the source one). `FULL OUTER JOIN` is essential to implementin…

RIGHT and FULL JOIN are on the trunk branch of SQLite and will (very likely) appear in the next release. Please grab a copy of the latest pre-release snapshot of SQLite ( https://sqlite.org/download.html ) and try out the new RIGHT/FULL JOIN support. Report any problems on the forum, or directly to me at drh at sqlite dot org.

SWEEEEET!

Finally!

Thank you so much for this Mr. Hipp!

EDIT: Don't forget to edit the `omitted.html` page when you ship it!

Re: D1: Our SQL database

#200

If SQLite gets you excited, I'm building a firebase alternative based on sqlite. I'm betting hard on sqlite so this get's me super excited!! https://javascriptdb.com CF people around, I would love to chat, if anyone is interested please reach out at: jp@javascriptdb.com I'll be applying to this beta for sure!

Super interesting! I really like the idea. I'll join the beta, email sent :)

Any feedback on what do you find interesting would be awesome :) thanks!!
Post reply on HN