Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How SQLite is tested

sqlite.org

Post reply on HN
Print view View original

71–80 of 87 posts

Re: How SQLite is tested

#71

Post by bastardoperator »

resonious wrote: post #56
bastardoperator wrote: post #8

Over a decade ago, the maintainer of SQLite gave a talk at OSCON about their testing practices. One concept that stood out to me was the power of checklists, the same tool pilots rely on before every flight. He also mentioned Doctors Without Borders, who weren't seeing the outcomes they expected when it came to saving lives. One surprising reason? The medical teams often didn't speak the same language or even know ea…

On the other hand, I think this narrative also causes a lot of useless red tape. There might be some survivorship bias here. Aviation, Doctors Without Borders, and SQLite have good checklists. Checklists are simple, so it's easy to think "oh I could do that too". But you never hear about the probably endless companies and organizations that employ worthless checklists that do nothing but waste people's time. I wish t…

WHO and Gawande emphasize iteration, that the draft is always wrong. They also claim good checklists are really coordination tools disguised as task lists.

Re: How SQLite is tested

#72

Post by chrisweekly »

Waterluvian wrote: post #69
bastardoperator wrote: post #8

Over a decade ago, the maintainer of SQLite gave a talk at OSCON about their testing practices. One concept that stood out to me was the power of checklists, the same tool pilots rely on before every flight. He also mentioned Doctors Without Borders, who weren't seeing the outcomes they expected when it came to saving lives. One surprising reason? The medical teams often didn't speak the same language or even know ea…

The thing that drives me absolutely mental about most developers I’ve worked with is just how much work they’ll do to avoid the easy thing, if the easy thing isn’t programmatic. I have tests and CI and all that, sure. But I also have a deployment checklist in a markdown document that I walk through. I don’t preserve results or create a paper trail. I just walk the steps one by one. It’s just so little work that I rea…

Yeah, checklists are great. Further, they're even precursors to automation.

Re: How SQLite is tested

#73

Post by [deleted] »

Waterluvian wrote: post #69
bastardoperator wrote: post #8

Over a decade ago, the maintainer of SQLite gave a talk at OSCON about their testing practices. One concept that stood out to me was the power of checklists, the same tool pilots rely on before every flight. He also mentioned Doctors Without Borders, who weren't seeing the outcomes they expected when it came to saving lives. One surprising reason? The medical teams often didn't speak the same language or even know ea…

The thing that drives me absolutely mental about most developers I’ve worked with is just how much work they’ll do to avoid the easy thing, if the easy thing isn’t programmatic. I have tests and CI and all that, sure. But I also have a deployment checklist in a markdown document that I walk through. I don’t preserve results or create a paper trail. I just walk the steps one by one. It’s just so little work that I rea…

[deleted]

Re: How SQLite is tested

#74

Post by dragonwriter »

Waterluvian wrote: post #69
bastardoperator wrote: post #8

Over a decade ago, the maintainer of SQLite gave a talk at OSCON about their testing practices. One concept that stood out to me was the power of checklists, the same tool pilots rely on before every flight. He also mentioned Doctors Without Borders, who weren't seeing the outcomes they expected when it came to saving lives. One surprising reason? The medical teams often didn't speak the same language or even know ea…

The thing that drives me absolutely mental about most developers I’ve worked with is just how much work they’ll do to avoid the easy thing, if the easy thing isn’t programmatic. I have tests and CI and all that, sure. But I also have a deployment checklist in a markdown document that I walk through. I don’t preserve results or create a paper trail. I just walk the steps one by one. It’s just so little work that I rea…

Manual checklists are often the best option for repeated tasks that can't be automated sufficiently reliably and sufficiently economically. But if they can be, then manual checklists are unnecessarily inefficent and/or unreliable. And the more frequently repeated the task is (ceteris paribus), the more up-front energy is justified in automating it. That said, to automate a process, you have to understand it enough to generate a checklist as a prerequisite (and, sure, you can develop that understanding in the course of automation, but doing so first will also go a long way to informing you if automation is likely to be worthwhile.)

That said, and without prejudice to SQLite’s use of checklists which I haven’t deeply considered, while the conditions that make checklists the best choice are definitely present in aviation and surgery in obvious ways, processes around software tend to lend themselves to efficient and reliable automation, with non-transitory reliance on checklists very often a process smell that, while not necessarily wrong, merits skepticism and inquiry.

Re: How SQLite is tested

#75

Post by amboar »

bastardoperator wrote: post #8

Over a decade ago, the maintainer of SQLite gave a talk at OSCON about their testing practices. One concept that stood out to me was the power of checklists, the same tool pilots rely on before every flight. He also mentioned Doctors Without Borders, who weren't seeing the outcomes they expected when it came to saving lives. One surprising reason? The medical teams often didn't speak the same language or even know ea…

I've become pretty enthusiastic about checklists.

In case it helps anyone else, I wrote a small tool that helps maintain and execute them:

https://github.com/amboar/checklists/

It's just a shell script wrapped around $EDITOR and git. The intent is to write checklists in the style of github-flavoured markdown. It has some tricks such as envsubst(1) interpolation and embedding little scripts whose execution is captured alongside the execution itself.

Here's an example checklist that's fairly well-worn (though is somewhat dated):

https://gist.githubusercontent.com/amboar/f85449aad09ba22219...

Where checklist entries are commands I just copy/paste them into a shell session. Usually this is in a tmux split with the checklist on one side with a shell on the other.

Could more of it be automated? Perhaps, though when some of the steps fail for various reasons, I find it's easier to recover or repair the situation if I have been invoking them myself sequentially. The embedded script support enables progressive automation where stability of the results is demonstrated over time.

Re: How SQLite is tested

#76

Post by rkagerer »

resonious wrote: post #56
bastardoperator wrote: post #8

Over a decade ago, the maintainer of SQLite gave a talk at OSCON about their testing practices. One concept that stood out to me was the power of checklists, the same tool pilots rely on before every flight. He also mentioned Doctors Without Borders, who weren't seeing the outcomes they expected when it came to saving lives. One surprising reason? The medical teams often didn't speak the same language or even know ea…

On the other hand, I think this narrative also causes a lot of useless red tape. There might be some survivorship bias here. Aviation, Doctors Without Borders, and SQLite have good checklists. Checklists are simple, so it's easy to think "oh I could do that too". But you never hear about the probably endless companies and organizations that employ worthless checklists that do nothing but waste people's time. I wish t…

I make checklists for myself and they're enormously helpful. Because my brain can't always remember every single little detail of every complex task every single time.

I've also seen checklists made by morons that are enormously unhelpful.

IMO it's paramount for whoever is making the checklist to have familiarity with the task at hand (both how to do it properly, and what steps people tend to miss or get wrong), investment (is this tool something you'd find indispensable for yourself if you were placed in the role of executing it?), a sense of pragmatism and conciseness.

The ability to recognize what things will be obvious or flow naturally from A to B helps eliminate redundant fluff. e.g. I train volunteer firefighters and in most canonical steps for calling a Mayday, one is basically "Tell the person on the other end what's wrong". You don't need a checklist item for that. When something goes seriously sideways and you need help, you will be very inclined to convey what is the matter.

Re: How SQLite is tested

#77

Post by harry8 »

bastardoperator wrote: post #8

Over a decade ago, the maintainer of SQLite gave a talk at OSCON about their testing practices. One concept that stood out to me was the power of checklists, the same tool pilots rely on before every flight. He also mentioned Doctors Without Borders, who weren't seeing the outcomes they expected when it came to saving lives. One surprising reason? The medical teams often didn't speak the same language or even know ea…

Likely the idea came from Atul Gawande, who is a literate surgeon who has done work popularising the idea.

https://atulgawande.com/book/the-checklist-manifesto/

Re: How SQLite is tested

#78

Post by jacquesm »

epolanski wrote: post #21
bastardoperator wrote: post #8

Over a decade ago, the maintainer of SQLite gave a talk at OSCON about their testing practices. One concept that stood out to me was the power of checklists, the same tool pilots rely on before every flight. He also mentioned Doctors Without Borders, who weren't seeing the outcomes they expected when it came to saving lives. One surprising reason? The medical teams often didn't speak the same language or even know ea…

I've always found an enormous amount of good practices (not just engineering ones) in aircraft operations and engineering that would be applicable to software engineering. I've always day dreamed of an IT organization that combined those with the decision-making procedures and leadership of modern armies, such as the US Army one. I've re-read multiple times FM22-100 which I find strikingly modern and inspiring: https…

It's all about balance, in the end. If you do too much of one thing your business will fail. If you don't do enough of another, your business will fail too. And they're the same thing...

The trick then is to do just enough of everything to avoid disaster and to move as fast as you can to get to a realm where you can actually afford to do it right. Most start-ups initially cut corners like it is crunch time at the circle factory, which then usually catches up with them at some point either killing them or forcing them to adapt a different pace. Knowing exactly when to put more or less attention on some factor is the recipe for success but nobody has managed to execute that recipe twice in a row without finding things that no longer work, so it remains a dynamic affair rather than one you can ritualize.

And that's where checklists shine: repeated processes that are well defined and where change is slow enough that the checklists become 'mostly static', they still change but the bulk of the knowledge condensed in them stays valid over multiple applications.

Re: How SQLite is tested

#79

Post by joshvm »

AlotOfReading wrote: post #50
taberiand wrote: post #33

Earlier quoted context omitted.

Overengineering is building a bridge that will stand 1000 years when 100 will do; it's excess rigor for marginal benefit. Juicero wasn't overengineering, it was building a crappy bridge to nowhere with a bunch of gaudy bells and whistles to try and hide its uselessness and poor design, that collapsed with the first people to walk over it

Have you looked at a Juicero teardown [0]? It's overengineered to the point where it's a genuinely astonishing bit of engineering art. It's also an incredibly stupid product. Those things are completely compatible. [0] https://blog.bolt.io/juicero/

AvE’s Juicero teardown is also a classic:

https://youtu.be/_Cp-BGQfpHQ?si=V-LLVn1cQmrMsiDZ

Re: How SQLite is tested

#80

Post by steve_adams_86 »

stouset wrote: post #45
steve_adams_86 wrote: post #40

Earlier quoted context omitted.

> Nobody ever complains about the person who’s leaving everything they touch a little cleaner than how they found it. This should be true, but it's not in my experience. Even small, clear improvements are rejected as off-mission or shunted to a backlog to be forgotten. Like, "cool, but let's hold off on merging this until we can be certain it's safe", or in other words "this is more work for me and I'd really rather…

Do it as part of ticket work you're already doing. There is always a way to leave things better than how you found them. I have worked across a wide gamut of roles (full-stack eng, infosec, deploy infra, devops, infra eng, sysadmin), companies (big and small, startups and huge multibillion-dollar players), and industries (finance, datacenters, security products, gaming, logistics, manufacturing, AI) over a thirty yea…

> There is always a way to leave things better than how you found them.

I agree, and that's something I do even if I get push back. I think it's essential in all aspects of life, or things typically get worse. People have to care.

My point is more so that it often—in my experience, at least—is met with friction because a lot of people see this kind of housekeeping as bad for bottom lines, or in the weeds, as a distraction, or what have you. I've encountered friction and push back more than acceptance let alone appreciation, I think.

A very common (and sometimes fair) form of push back is along the lines of "let's keep this ticket ONLY to the bug fix/feature/whatever and avoid any unnecessary changes". This is generally good practice, but I'll personally allow unrelated changes if they're relatively simple chores to leave things better than they were found.

If I were to summarize my experience, it's that caring to make things better or as good as they should be typically requires my own time and energy outside of regular work hours. That's a hard sell, especially after 20 years or so. I still do my best to make things better and I've come to expect and meet that friction with the necessary energy to overcome it, but... I mean, it's not always easy or necessarily rewarding. Many of my jobs could have often been dramatically easier if I cared less, but I'm not sure anyone would have even minded that.

Post reply on HN