Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite the only database you will ever need in most cases (2021)

unixsheikh.com

Post reply on HN
Print view View original

171–180 of 378 posts

Re: SQLite the only database you will ever need in most cases (2021)

#171

Post by orthecreedence »

bravura wrote: post #124
cutler wrote: post #109

Earlier quoted context omitted.

Agreed. Too much reliance on AWS, Heroku etc. can leave you without vital Linux skills. Linux cli tools are not just for getting a job done, though they're great at what they do. No, they're part of the *nix culture which is what open source software is based on. Only recently have I become aware that it was better to have entered software development in or before the early 2000s, ie. before The Cloud took off. You w…

I used slackware linux. I wrote assembly code when I was a teenager. I use 'perl -i -pe' constantly. I still don't want to learn a bunch of arcane flags from a bunch of tools with a million gotchas. Sorry. I'd rather focus on the code craft that gives me joy these days. Sorry, I detest this attitude of: "Well I was hazed so why aren't you hazed too?" Did I mention that Russ Cox and I used to write C code with pen and…

> I used slackware linux. I wrote assembly code when I was a teenager.

Lol are you me?

I grew up with Slack/assembler/softice/etc and I used to host everything on Slack boxes on Linode (bless them for having a pre-built image). But I left it a while ago because I got sick of dealing with hand upgrades and compiling everything from scratch. I feel the same way about other things. I could probably out-CLI most of the people here saying "you should learn linux!" but sometimes I just need to get something done.

If I'm trying to get something out the door, I don't give a rat's ass about the details. It's fun and interesting when you're a teenager and it's 4am on a summer night and you've got nothing better to do, but these days I have strict time, limited energy, and I'm not going to spend more time fiddling with things than I need to.

Re: SQLite the only database you will ever need in most cases (2021)

#173

Post by erik_seaberg »

canadianfella wrote: post #122
irskep wrote: post #29

This sentiment pops up regularly on HN, and I've seen at least one article per month for the past few months, but the trouble is, none of them seem to help you actually deploy it. They assume you're comfortable spinning up public web servers. If you want to use a PaaS to deploy an app, because you don't want to spend your time learning to be a sysadmin, then all the tutorials are going to put you on the Postgres path…

Why do people use hosted databases instead of just installing them? I don’t understand it.

If you install your own copy, you are on call for it, and the colo is unlikely to offer much help. You also have to set up monitoring or you won’t even know when it fails. Then there’s replication. Backups. All this stuff is work that PaaS vendors are ready to automate away, if my time is expensive for the org.

For fun, sure, dink around and learn as long as there are no customers to affect.

Re: SQLite the only database you will ever need in most cases (2021)

#174

Post by cldellow »

sanderjd wrote: post #150
cldellow wrote: post #75

Earlier quoted context omitted.

You can run migrations from another process. Migrations are just writes, and SQLite supports writes from multiple processes. A single transaction that does a very large write will likely impact the reader -- the reader will be blocked while the write finishes. I use SQLite in a web scraper on my laptop. The scraper runs as 16 processes hammering the database, doing about 5,000 write transactions/sec. Occasionally, si…

> Migrations are just writes It is possible to change the schema of a SQLite db as "just a write"?

Yes. SQLite supports transactional DDL.

Re: SQLite the only database you will ever need in most cases (2021)

#175

Post by JohnBooty »

kcb wrote: post #144
JohnBooty wrote: post #134

Earlier quoted context omitted.

You don't need to be a full-blown sysadmin to know how to do basic deployments, and learning these things will make you a better developer. Yes. True. However, it is also true that we have a limited amount of time per week and a limited number of weeks on Earth. Time spent learning sysadmin-y stuff is less time spent mastering developer-y stuff. Think about what it means to be a "full stack" engineer in 2023: - Unix…

Unless you're one of the very few known as top in a field, mastery of a specific technology adds very little value and is always in danger of becoming more legacy than relevant. That's in comparison to someone with enough knowledge to assist in or handle any phase of getting a product into the the hands of customers and making money. Most things don't need master level work or knowledge to be successfully implemented…

    mastery of a specific technology adds very 
    little value
I disagree in the strongest possible terms but it's certainly possible that we're working with different definitions of "mastery" so I'll give you mine. My version is perhaps actually closer to "competency" rather than "deep under-the-hood knowledge." I'm talking about somebody who knows how to use a screwdriver without stabbing their own eyeballs out, and when to use a screwdriver, but isn't necessarily like... inventing new types of screwdrivers or innovating in the field.

Somebody with Postgres "mastery" should have a strong command of basic normalized table structures, they should know how to construct useful indexes, they should know the downsides of over-indexing, they should know how to find and optimize slow queries, and they should at least know of somewhat advanced topics such as partitioning, materialized views, foreign data wrappers, and so on. It is okay not to have a clue how to set up e.g. partitioning but you should know what it is so that you can learn about it and employ it when needed.

Somebody with "mastery" of an application framework such as Rails should be competent at basic OO design, they should be familiar with Ruby structures and idioms, they should understand what goes where in Rails' MVC paradigm, they should be able to write comprehensive and performant tests, and they should know how Rails' asset pipeline builds and emits front end code. I would not expect them to have mastery of Ruby's metaprogramming, but I would expect them to know what it is and have some idea of how Rails uses it to extend Ruby. (I realize Rails is controversial, but I'm not talking about anything Rails-specific. Feel free to substitute your language+framework of choice)

In short, the sort of working knowledge you might commonly gain with 1-3 years of real-world production experience with a given piece of the stack. Or perhaps less if you were really focused on it and just learning it full time.

Without guidance, folks with less knowledge than this tend to produce absolutely unscalable, unmaintainable spaghetti code for anything larger than a toy project. To return to the screwdriver analogy, they can kind of use a screwdriver but they're constantly stripping screwdriver heads at which point either the project is ruined or they need some help from more practiced craftsperson.

    always in danger of becoming more legacy than relevant
I would certainly agree that you can go too deep to be useful as a "full-stack" developer. For example, if you really understand Postgres internals and can write your own extensions and have a bunch of commits in Postgres itself, cool, but what happens when our next client wants us to work on their MySQL-based app? And while Postgres is unlikely to die anytime soon, what happens if you've achieved that level of expertise in a tool that does fall by the wayside? I've certainly "mastered" a few tools that are distant memories.

Re: SQLite the only database you will ever need in most cases (2021)

#176

Post by danielmarkbruce »

sanderjd wrote: post #158
danielmarkbruce wrote: post #116

Earlier quoted context omitted.

Third paragraph of the article says: "In contrast to many other database management systems, SQLite is not a client-server database engine, but you actually very rarely need that. If your application software runs on the same physical machine as the database, which is what most small to medium sized web applications does, then you probably only need SQLite." That's how we square it. It's right there in the article.

I've realized that this whole thing comes down to the word "most" doing too much duty here. I don't think it's true that most applications just run a single db-and-application node. I've never worked on such an application. You and the author do seem to think this is true. It would be difficult for either of us to support our intuition empirically, so this is where the divide comes from.

Yep, that's probably true.

Re: SQLite the only database you will ever need in most cases (2021)

#177

Post by berdon »

irskep wrote: post #164
berdon wrote: post #151

Earlier quoted context omitted.

Fwiw, I’ve used all of those in my career, extensively here and there. I’ve written many thousands of lines of bash scripts. Even written my own php-for-bash-script style code tags that support arbitrary shells. I’ve written my own log based distributed kv store, gone down the YouTube trail of writing my own db, gone through the angular and react iterations, deep dove in docker, docker compose, k8s, crds and custom h…

Sure, you can do all that, but have you also written video games, UI frameworks, compilers, Twitter bots, localization tools, and web-based whiteboards? Have you ever made a music video or gotten a novel published? If not, why not? I'd say it's because you had different interests and spent your time elsewhere, which is completely fine and not a problem at all. People are allowed to have their own interests and explor…

No whiteboard, music video, or a published novel yet.

But, come now, his point was he’d never worked with a full stack developer and that they don’t exist.

Re: SQLite the only database you will ever need in most cases (2021)

#178

Post by naniwaduni »

sanderjd wrote: post #158
danielmarkbruce wrote: post #116

Earlier quoted context omitted.

Third paragraph of the article says: "In contrast to many other database management systems, SQLite is not a client-server database engine, but you actually very rarely need that. If your application software runs on the same physical machine as the database, which is what most small to medium sized web applications does, then you probably only need SQLite." That's how we square it. It's right there in the article.

I've realized that this whole thing comes down to the word "most" doing too much duty here. I don't think it's true that most applications just run a single db-and-application node. I've never worked on such an application. You and the author do seem to think this is true. It would be difficult for either of us to support our intuition empirically, so this is where the divide comes from.

> I don't think it's true that most applications just run a single db-and-application node. I've never worked on such an application.

Rather than applications you've worked on (many of us spend years on end working on a narrow range of applications), consider software you use (most of us flip between multiple applications every day spanning the gamut of uses). Ignore, for a moment, whatever you know about their implementation, and focus on the user-facing functionality.

Do even half of them logically communication between application nodes?

(Consider that there is still a rich class of software that is fully capable of running locally on the "client"'s computer!)

Re: SQLite the only database you will ever need in most cases (2021)

#180

Post by [deleted] »

bogwog wrote: post #86
irskep wrote: post #29

This sentiment pops up regularly on HN, and I've seen at least one article per month for the past few months, but the trouble is, none of them seem to help you actually deploy it. They assume you're comfortable spinning up public web servers. If you want to use a PaaS to deploy an app, because you don't want to spend your time learning to be a sysadmin, then all the tutorials are going to put you on the Postgres path…

> I'm an application developer and I do not want to become a release engineer. I resent even having to learn Docker. :-) Sorry, but that's a terrible attitude to have. You don't need to be a full-blown sysadmin to know how to do basic deployments, and learning these things will make you a better developer.

[deleted]
Post reply on HN