Live data from Hacker News

SQLite the only database you will ever need in most cases (2021)

unixsheikh.com

361–370 of 378 posts

Re: SQLite the only database you will ever need in most cases (2021)

#361

Earlier quoted context omitted.

Ah, that brings back memories. Had 2 RAID 10 MySQL servers run for a decade without rebooting. One had an app db, the other a stats db, and the two replicated to each other. Spinning disks and all, I was terrified to reboot them and have the boot disk fail (which was not on RAID). The main disks failed once or twice which slowed the servers down considerably until rebuild of the raid finished. Very nervous time.

How did this situation come to an end? End of life for the service?

New machines with SSDs! Then took those guys out of service for good.

Re: SQLite the only database you will ever need in most cases (2021)

#362
post #354

Earlier quoted context omitted.

I know my way around Elixir or React, as I do around C or Rust, as I do around sysadmin (not only DevOps) or DBA work, as I do around low-level system code To what level do you know these tools? This is a challenging discussion because the idea of "knowing" or "being good at" a tool is so nebulous. As I mentioned elsewhere I'm using a definition that is essentially, "the level of skill a solid engineer would acquire…

I've used Elixir full time for the past 6 years. I've written Rust full time for the past 3 years. I've written C since I was 14 in 2001. I wrote a small operating system (up to reading and running a binary from ext2) around 2004 in C, so I know how a computer works at low level. I might still be able to write x86 assembly. I've been a MySQL DBA full time for 3 years. I've administered Linux systems since I was 14 in…

    I've used Elixir full time for the past 6 years.

    I've written Rust full time for the past 3 years.
What does this mean? You've been working 80+ hours a week for the last three years?

Re: SQLite the only database you will ever need in most cases (2021)

#363

Earlier quoted context omitted.

Fair enough I guess. I don’t think you need two servers to do zero downtime upgrades. And the other issues are, imo, beyond the 0.99 uptime threshold that most services realistically have when you add in breakage due to upgrades. I like your statelessness point. I suppose in your view it’s better to have the concentrated stateful core with stateless servers as opposed to just one stateful instance. Two instances mean…

> I don’t think you need two servers to do zero downtime upgrades Absolutely not and I can't understand why I keep hearing this argument. Doing zero downtime upgrades on a single server have been simple since basically forever, run another process on another port, change config, restart front balancer gracefully and there you go.

Sure, it can be done, but that alone isn't enough reason to give up redundancy.

Re: SQLite the only database you will ever need in most cases (2021)

#364
post #59

Earlier quoted context omitted.

HN users pride themselves on finding the least capable tool for the job that only just works for the task but no more. It’s not about logic or practicality. It’s that they feel some kind of mental pain using Postgres as it is too “bloated”.

Or maybe its because SQLite is just... easier to deploy? Cheaper? There are many reasons to choose it over a "fatter" solution.

definitely not easier to deploy, at least if you're talking about how most people build software these days (what sqlite managed service are you familiar with? for mysql or postgres there's hundreds of companies offering this)

Re: SQLite the only database you will ever need in most cases (2021)

#365

I like sqlite as much as the next guy but it's built-in datatypes are limited. Things like arrays, UUIDs, geometry stuff, JSON, etc. Sure you can store more advanced stuff as blobs or text but then you have to mess around with deserializing it in the host language and you lose the ability to query it directly in the db engine.

Lack of int64 is also a problem for some applications.

Oh my god how I want a uint64, including in Postgres.

The limited datatypes are so silly, and the limits seem pretty pointless. There's all kinds of weird datatypes, but no unsigned integers?

Of course, SQLite types are a special level of hell, where everything is stringly typed.

Re: SQLite the only database you will ever need in most cases (2021)

#366
post #157

Everytime I try to use SQLite I run into db locking issues where I seemingly have to try to run my query in a retry loop. Am I doing something wrong or does SQLite just not play nice in multi threaded contexts?

This is true for every SQL database. Push conflicting transactions hard enough and you'll need a retry loop. In Postgres you'll see row-level MVCC detecting a write conflict and have the exact same end result. SQLite's locking is just coarser grained, and tends to trigger with less load.

https://www.postgresql.org/docs/15/mvcc-serialization-failur...

Re: SQLite the only database you will ever need in most cases (2021)

#367
post #81

Earlier quoted context omitted.

My standard for any serious service is at least minimal redundancy for improved availability during failures. At least two webservers.

Does this practically improve the situation? The odds of two servers breaking at the same time for the same reasons seems very high. I actually can't think of a single example where the secondary sever would keep running. Regression via a code or dependency update? Full disk? DNS is down? Too much load? All of these would bring down both servers in quick succession. I guess something like a "once every 2 days" race c…

We use 3 node MSSQL and it happens all the time where the primary gets in a bad state (100% cpu, high latency etc)and simply failing over to another instance fully recovers.

It could be bad hardware, it could be bad query (left dangling/canceled on old instance), could be bad statistics and unlocks disk fragmentation etc etc.

Re: SQLite the only database you will ever need in most cases (2021)

#368

I wish sqlite made their terminal interface a bit more robust or emulated psql’s interface. Simple things like \d tablename would be great.

.schema tablename

The output is apples and oranges tho. Since I was downvoted by someone I'll added a simple example to show the difference between the two interfaces. I shouldn't have assumed anyone here was familiar with the respective representations.

Sample data:

    CREATE TABLE users (user_id serial, name text);
    CREATE TABLE comments (comment_id serial, user_id int, comment text unique);
    CREATE VIEW user_comment_view as select u.user_id, u.name, c.comment from users u, comments c where u.user_id = c.user_id;
    INSERT INTO users VALUES (1, 'Bob');
    INSERT INTO users VALUES (2, 'Sally');
SQLITE3 OUTPUT

    sqlite> .schema
    CREATE TABLE users (user_id serial, name text);
    CREATE TABLE comments (comment_id serial, user_id int, comment text unique);
    CREATE VIEW user_comment_view as select u.user_id, u.name, c.comment from users u, comments c where u.user_id = c.user_id
    /* user_comment_view(user_id,comment) */;


    sqlite> .schema users
    CREATE TABLE users (user_id serial, name text);


    sqlite> select * from users;
    1|Bob
    2|Sally

POSTGRESQL OUTPUT

    test=# \d
                    List of relations
     Schema |          Name           |   Type   |  Owner   
    --------+-------------------------+----------+----------
     public | comments                | table    | postgres
     public | comments_comment_id_seq | sequence | postgres
     public | user_comment_view       | view     | postgres
     public | users                   | table    | postgres
     public | users_user_id_seq       | sequence | postgres
    (5 rows)


    test=# \d users
                                Table "public.users"
     Column  |  Type   | Collation | Nullable |                Default                 
    ---------+---------+-----------+----------+----------------------------------------
     user_id | integer |           | not null | nextval('users_user_id_seq'::regclass)
     name    | text    |           |          | 


    test=# select * from users;
     user_id | name  
    ---------+-------
           1 | Bob
           2 | Sally
    (2 rows)

Postgres also supports adding + to commands to get additional extended information, eg, \d+. You can also filter by tables (\dt), filter by views (\dv), filter by functions (\df), etc. It's allows much more natural enumeration of the DB which I wish sqlite had as well.

Re: SQLite the only database you will ever need in most cases (2021)

#370

I agree. Most uses of databases definitely don't need to grow larger than, say, a single filesystem, or a single application, or a single host, or a single network, or a single geographical region, or a single customer, or a single organization, or a single global network of customers in organizations in regions on networks on hosts on applications on filesystems. There could not be any features of any other database…

Or you could have just said "SQLite is very useful but doesn't work well for all apps" and left it at that objective truth.
Post reply on HN