Live data from Hacker News

A Minimalist Guide to SQLite

tech.marksblogg.com

71–80 of 127 posts

Re: A Minimalist Guide to SQLite

#71
post #47

Earlier quoted context omitted.

For me, using PostgreSQL is like using static typing: a bit more work, but it catches a lot of bugs. SQLite favours a very permissive approach, while PostgreSQL favours locking everything down with strict types and strong integrity checks, and it's very easy to verify lots of details of your data before it's accepted into the database. That takes work to set up properly and maintain, but it catches bugs early and red…

Strictly speaking, SQLite is dynamically typed in a sense, because the database does not enforce the values in the rows match what the columns claim about the type. But from a developer experience point of view, the main difference between dynamic and static typing is the following: Dynamic typing requires a lot of extra checks to ensure everything is what you expect it to be, and a lot of annotations to document wha…

I agree with your point, but disagree with the example.

RE static vs dynamic typing, I'd say that it's dynamic typing that has higher cognitive tax - you as a programmer are fully responsible for ensuring types agree up everywhere, whereas with static typing, all of that job is done by compiler. It's easier to change stuff in the code when you know the compiler will catch your dumb mistakes. And I say that as someone who loves writing in Common Lisp.

But the point about SQLite vs a typical RDBMS is spot-on, IMO. SQLite is a library. It operates on files. That's it. It's entirely local. You can use it without having to become a sysadmin, without having to set up a whole service on the OS on which you might not even have root privileges anyway. You don't have to make global changes to the system just for your program. Moreover, if your product is of the distributable kind (desktop apps, self-hosting web stuff) you don't need to make your users become sysadmins, manage a system-wide service, acquire root rights they might not have, etc.

SQLite is local. Its data is local. That's, IMO, its strongest benefit.

Re: A Minimalist Guide to SQLite

#72
post #48
post #41

Earlier quoted context omitted.

> I mean Postgres is a fine piece of software but if your website gets 500 visits a day, you don't need Postgres; just use SQLite. On the flip side, setting up a PostgreSQL instance is so easy now with docker that you could go ahead and use it with a fraction of the admin overhead that it used to require: https://hub.docker.com/_/postgres/

Docker is its own nightmare :/ I just find it weird and confusing. Also, how do you setup a database in docker? Docker does not support persisting data as far as I know. So if you need to restart the thing, all the data will be wiped. At least that's how it seems to be.

Other HNers are making it more complicated than it strictly needs to be. Kubernetes and storage drivers are great and all, but we're talking hello world here.

So, if you want to keep it simple:

  docker run postgres --detach --volume /save/my/data/in/this/path:var/lib/postgresql/data
That's it. (The --detach is to start it in background. Otherwise it would run in your terminal session and stop when you CTRL-C.)

Even if you don't specify a path for the data, the container would save its data in a Docker volume (a file), which doesn't get deleted until you explicitly ask it to. If you stop the container and restart it, it'll reattach to the same volume and find the data. If you delete the container without explicitly deleting its volumes as well, you'll be able to create another postgres container and attach the old volume to it.

Re: A Minimalist Guide to SQLite

#73
post #33

Earlier quoted context omitted.

It's really weird to have to write on every request. If you're logging user visits for example, you can queue those up in a list (in the code) and flush it to the database every few seconds instead of flushing it on every visit.

> queue those up in a list (in the code) That won't work in the language that most server-side website code is written in. PHP processes are created and destroyed per HTTP request, and even the FastCGI implementation won't let one share data across requests. PHP does have a sessions feature, but that data is written to either the filesystem (by default) or a database on every request!

It's pretty common to write session data in memory eg. memcache/redis.

Re: A Minimalist Guide to SQLite

#74
post #43
post #29

Very nice! I'm in the middle of writing a book based on SQLite (the overall topic is data analysis with SQL, but SQLite is the medium) while I'm teaching it to students. I used to teach MySQL but it was just so goddamned hard to get it configured correctly on people's computers (I always hated when work had to be done on pre-configured computers lab rather than my own laptop), nevermind the server/client/daemon aspec…

I'm of course not suggesting you switch back to MySQL, but distributing preconfigured MySQL docker images perhaps might've helped with your configuration troubles.

Maybe now, but it didn't seem feasible in 2014, or at least I just wasn't familiar enough with it at that point. In any case, this was a humanities class, and I wanted to keep the tech as straightforward as possible.

Re: A Minimalist Guide to SQLite

#76

One of my favorite sqlite features is the vtable mechanism: https://sqlite.org/vtab.html It makes it possible to expose a custom, application specific database format as a sqlite table, and suddenly you can run SQL queries on your data!

Yes, this! Vtables are how [lnav](http://lnav.org) works its magic. Embedded SQLite makes so much sense for such use cases.

Re: A Minimalist Guide to SQLite

#77
post #11

Earlier quoted context omitted.

An sqlite db file is probably as light as your json or csv AND better formed/typed

It’s easier to compare it to CSV than JSON, because the former is tabular while the latter can have deep imbrications that aren’t well represented in SQL(ite).

`imbrications`: overlapping, as in roof tiles

(thought it was a typo, now I’m not sure)

Re: A Minimalist Guide to SQLite

#78

Earlier quoted context omitted.

Not an SQL expert but pretty sure two tables joined together is extremely normal and well represented in RDBMS

Yep, I can write an INNER JOIN faster than I can remember jq's arcane syntax. If you needed it all in one flat table for some reason, SQLite supports views.

Speaking of which, I've not grokked jq's functional approach to composing queries. Can anyone suggest a good approach to understanding jq please?

Re: A Minimalist Guide to SQLite

#79

One of my favorite sqlite features is the vtable mechanism: https://sqlite.org/vtab.html It makes it possible to expose a custom, application specific database format as a sqlite table, and suddenly you can run SQL queries on your data!

Postgres also has that concept, though calls it Foreign Data Wrappers: https://wiki.postgresql.org/wiki/Foreign_data_wrappers

And it goes beyond single tables e.g. PG-Strom uses FDW to implement GPGPU scans, joins, aggregations & projections (and provides a pg/CUDA while at it)

Re: A Minimalist Guide to SQLite

#80
post #66

Earlier quoted context omitted.

Instead of doing all of that, you could also do it without docker. Simply install postgres via your Linux distro and run e.g.: /usr/lib/postgresql/9.5/bin/postgres -D .mydatadir

Sure you can do that, but if your app needs to use other services (Redis, for example) you can easily throw that in the compose file as well, and then it's easy to maintain versions and have every developer be on the same page. This is especially helpful if some developers use OS X or Windows. I don't understand the "all of that" phrasing, as if what I just described was a large amount of work.

How many people run webwpps with changing version requirements for their _database_?

Docker is useful for running components of your own code or for isolation, not for managing versions of core applications like a database or a mail server for most people deploying ready-made apps.

Post reply on HN