Live data from Hacker News

Database internals are becoming less important than developer experience

planetscale.com

31–40 of 76 posts

Re: Database internals are becoming less important than developer experience

#31
post #15

As a developer, I have to say that sqlite gives me the best experience. Everything else pales in comparison. Create a database? sqlite3 mydata.db Where is the database? In the current directory How is it structured on disk? It's a single file How do I backup the DB? cp mydata.db /my/backups/mydata.db Do I have to install a server? No Do I have to configure anything? No During setup and deployment I usually I dabble a…

Fully agree, but that comes with tradeoffs around concurrent writes, strong typing (though I hear that's coming), some other functionality, etc.

Re: Database internals are becoming less important than developer experience

#32
post #15

As a developer, I have to say that sqlite gives me the best experience. Everything else pales in comparison. Create a database? sqlite3 mydata.db Where is the database? In the current directory How is it structured on disk? It's a single file How do I backup the DB? cp mydata.db /my/backups/mydata.db Do I have to install a server? No Do I have to configure anything? No During setup and deployment I usually I dabble a…

Is there a wrapper to treat a database like its an sqlite database (a file) to simplify to the same level as Sqlite? This may at least simplify some issues. Doesn't matter whether its backed by a local database running in a container or by a remote one.

The biggest database unification effort tends to be focused around ODBC compatibility from what I've seen - and the operations laid out by ODBC are quite trivial and easy to comprehend I think.

Re: Database internals are becoming less important than developer experience

#33
post #15

As a developer, I have to say that sqlite gives me the best experience. Everything else pales in comparison. Create a database? sqlite3 mydata.db Where is the database? In the current directory How is it structured on disk? It's a single file How do I backup the DB? cp mydata.db /my/backups/mydata.db Do I have to install a server? No Do I have to configure anything? No During setup and deployment I usually I dabble a…

Unless you need to do something crazy, like run two nodes of your application. But only Google needs to do that, right?

Re: Database internals are becoming less important than developer experience

#35
post #22
post #11

Earlier quoted context omitted.

I can't imagine any scenario under which a reasonable person of at least median intelligence would perceive MongoDB as "easier to use" than Spanner, unless their entire experience with Mongo was they put one trivial JSON doc into an M0 cluster and got it back out later. Every practical aspect of MongoDB is a complete shitshow, from sharding to backup to failover.

"Sharding" "Backups" and "Failovers" are NOT "practical" aspects of any database. They're theoretical. Most databases are not big enough to need sharding. Most backups go unused. Most failover happens automatically, totally managed by your hosting provider. You know what is practical? Schema design. Query language. That's what made MongoDB super popular; no schemas to worry about. A query is just '{ firstName: "John"…

If the database itself can’t handle failover properly, there’s nothing your hosting provider can do except rewrite it (which is exactly what Amazon ended up doing with MongoDB). Also if you’ve spent any significant time being responsible for operating a database and came away with “backups aren’t practical” you’re insanely lucky (or I and everyone I know is insanely unlucky).

Re: Database internals are becoming less important than developer experience

#36
post #12

Earlier quoted context omitted.

Unfortunately MongoDB proved the opposite. You don't notice the data loss and schema problems until later.

That is a pretty bad experience in the long run.

Right, but in the long run you're more locked in than you were when you didn't know that.

Re: Database internals are becoming less important than developer experience

#38
post #22
post #11

Earlier quoted context omitted.

I can't imagine any scenario under which a reasonable person of at least median intelligence would perceive MongoDB as "easier to use" than Spanner, unless their entire experience with Mongo was they put one trivial JSON doc into an M0 cluster and got it back out later. Every practical aspect of MongoDB is a complete shitshow, from sharding to backup to failover.

"Sharding" "Backups" and "Failovers" are NOT "practical" aspects of any database. They're theoretical. Most databases are not big enough to need sharding. Most backups go unused. Most failover happens automatically, totally managed by your hosting provider. You know what is practical? Schema design. Query language. That's what made MongoDB super popular; no schemas to worry about. A query is just '{ firstName: "John"…

Backups are most definitely a necessary part of any persistent datastore that is accumulating value for your company. Sharding is something that becomes necessary at scale - having it in your toolbox is a really good idea - but it doesn't come up for most folks. Failovers are the least valuable of what you've mentioned IMO since downtime management is something that even very mature companies have a lot of flexibility with.

SQL is a pretty easy language to learn - you can get pretty much everything working right with some SELECT, FROM, WHERE, GROUP BY and subqueries alone - the more arcane dialect components of SQL (like HAVING - gosh I hate how easy to misunderstand HAVING is) are things you can grow into. But basic SQL - it works and has worked fine for decades - I strongly dislike tooling and languages that go on crusades to make a better SQL.

Re: Database internals are becoming less important than developer experience

#39
The author skips the first decade of database systems in the 1960s. The oldest databases were not relational! They were hierarchical or navigational.

Hierarchical databases were much like a filesystem, but for records instead of files.

Navigational databases allowed data to be linked in a network. Look up CODASYL for detail.

The relational database design was first proposed in the 1970s.

Re: Database internals are becoming less important than developer experience

#40
~hello everyone, author here~

I know posts with ThOuGhT LeaDeRshIp titles like this are usually annoying, but I thought it would be interesting to write down some of the lessons I've been gathering as I've spent more time covering and using specific databases. My background is in data science / analytics with a couple of years of more traditional full stack here and there.

Broadly we've seen this pattern with infrastructure in general – it's a lot easier to set up a server than it used to be, all things considered. Now obviously if you're a tiny startup, you're more comfortable outsourcing everything to Heroku, and if you're a hyperscale enterprise, you probably want more control on exactly what your database is doing.

The thesis here is that on the tail end (hyper scale), things are getting more under control and predictable, and developers there want the same "nice things" you get with platforms like Heroku. Elsewhere in the ecosystem, more and more parts of the stack are getting turned into "simple APIs" (Stripe for payments, Twilio for comms, etc.). And perhaps most interestingly, as serverless for compute seems kind of stuck (maybe?), it may be the case that serverless for databases – whatever that ends up meaning – is actually an easier paradigm for application developers to work with.

Post reply on HN