Live data from Hacker News

How SQLite is tested

sqlite.org

31–40 of 87 posts

Re: How SQLite is tested

#31
I am surprised to see that there isn't a lot of information about performance regression testing.

Correctness testing is important but the way SQLLite is used, potential performance drops in specific code paths or specific type of queries could be really bad for apps that use it in critical paths.

Re: How SQLite is tested

#32
post #19

Over a decade ago, the maintainer of SQLite gave a talk at OSCON about their testing practices. One concept that stood out to me was the power of checklists, the same tool pilots rely on before every flight. He also mentioned Doctors Without Borders, who weren't seeing the outcomes they expected when it came to saving lives. One surprising reason? The medical teams often didn't speak the same language or even know ea…

A lot of these seem to be potentially automated - why aren’t they? Anyone know?

The stupid answer is that not everything that can be automated should be.

The real answer is of a more philosophical nature, if you manually had to check A, B, C... Z, then you will have a better understanding of the state of the system you work with . If something goes wrong, at least the bits you checked can be disregarded and free you to check other factors. What if your systems correctly report a faulty issue, yet your automatic checklist doesn't catch it?

Also, this manual checklist checks the operator.

You should be automating everything you can, but much care should be put into figuring out if you can actually automate a particular thing.

Automate away the process to deploy a new version of hn, what's the worst that can happen?

But don't automate the pre flight checklist, if something goes wrong while the plane is in the air, people are going to die.

I think a less verbose version of the above is that a human can detect a fault in a sensor, while a sensor can't detect it is faulty itself.

Re: How SQLite is tested

#33
post #13

Earlier quoted context omitted.

You can literally just do this. I’ve never gotten fired from a software engineering job for moving slower and building things that work well, work predictably, and are built to last. Over a career of working at it, you get dramatically better at higher levels of quality even in earlier passes, so the same level of added effort provides increasing levels of reward as you gain experience. Nobody ever complains about th…

You can definitely go overboard for work. If you want to do it as a hobby, go nuts, but there isn't a point in overengineering far beyond what is needed (recall the Juicero)

Overengineering is building a bridge that will stand 1000 years when 100 will do; it's excess rigor for marginal benefit. Juicero wasn't overengineering, it was building a crappy bridge to nowhere with a bunch of gaudy bells and whistles to try and hide its uselessness and poor design, that collapsed with the first people to walk over it

Re: How SQLite is tested

#34
post #31

I am surprised to see that there isn't a lot of information about performance regression testing. Correctness testing is important but the way SQLLite is used, potential performance drops in specific code paths or specific type of queries could be really bad for apps that use it in critical paths.

While I’ve worked in HFT and understand the sentiment, I can’t recall any open-source project I’ve used coming out with performance guarantees. Most use license language setting no guarantee or warranty. Are there notable projects that do include this consideration as their core mission?

Re: How SQLite is tested

#35

Over a decade ago, the maintainer of SQLite gave a talk at OSCON about their testing practices. One concept that stood out to me was the power of checklists, the same tool pilots rely on before every flight. He also mentioned Doctors Without Borders, who weren't seeing the outcomes they expected when it came to saving lives. One surprising reason? The medical teams often didn't speak the same language or even know ea…

That is really insightful regarding the ritual improving outcomes through better communication - something I see reflected in many meetings I turn up to now which involve an introduction round between participants, and anecdotally improves participation in the meeting.

It would be amazing if someone had a link to a page with the MSF story, as that is a great reference to have! My google-fu hasn’t helped me in this case.

Re: How SQLite is tested

#36
post #13

Earlier quoted context omitted.

You can literally just do this. I’ve never gotten fired from a software engineering job for moving slower and building things that work well, work predictably, and are built to last. Over a career of working at it, you get dramatically better at higher levels of quality even in earlier passes, so the same level of added effort provides increasing levels of reward as you gain experience. Nobody ever complains about th…

> I’ve never gotten fired from a software engineering job for moving slower and building things that work well, work predictably, and are built to last. In most companies, that’s not how it plays out. Once something works, you’re immediately moved on to the next task. If you’ve had the time and space to refine, polish, and carefully craft your code, you’ve been fortunate.

The person who signals that some task works and is finished is you. You have way more control over this than you are giving yourself credit for.

If you spend your career acquiescing to every request to “just ship it” then, yes, slowing down a second to do a quality pass will seem impossible. But you really can just do it.

Re: How SQLite is tested

#37

I love sqlite, it's a great piece of software. The website is full of useful information, rather than the slick marketing we are used to, even on open source projects. With that said, I find it strange how the official website seems to be making its way through the HN front page piecemeal.

If you wait here long enough, it happens again, and again, and again, and again...to the point you start wanting to skewer it. :)

EDIT: Haskell was early 2010s Zig, and Zig is in the traditional ~quarter-long downcycle, after the last buzzkill review post re: all the basic stuff it's missing, ex. a working language server protocol implementation. I predict it'll be back in February. I need to make a list of this sort of link, just for fun.

Re: How SQLite is tested

#38
post #36

Earlier quoted context omitted.

> I’ve never gotten fired from a software engineering job for moving slower and building things that work well, work predictably, and are built to last. In most companies, that’s not how it plays out. Once something works, you’re immediately moved on to the next task. If you’ve had the time and space to refine, polish, and carefully craft your code, you’ve been fortunate.

The person who signals that some task works and is finished is you . You have way more control over this than you are giving yourself credit for. If you spend your career acquiescing to every request to “just ship it” then, yes, slowing down a second to do a quality pass will seem impossible. But you really can just do it.

> The person who signals that some task works and is finished is you.

That's not how it works in most big companies. You can't take arbitrarily long to finish a project. Before the project is greenlit you have to give an estimate for how long the project will take. If your estimate is too big or seems unreasonable the project dies then and there (or given to someone else). Once the project starts you're held to the estimate, and if you're taking noticeably longer than your estimate you better have a good explanation.

Re: How SQLite is tested

#39
post #35

Over a decade ago, the maintainer of SQLite gave a talk at OSCON about their testing practices. One concept that stood out to me was the power of checklists, the same tool pilots rely on before every flight. He also mentioned Doctors Without Borders, who weren't seeing the outcomes they expected when it came to saving lives. One surprising reason? The medical teams often didn't speak the same language or even know ea…

That is really insightful regarding the ritual improving outcomes through better communication - something I see reflected in many meetings I turn up to now which involve an introduction round between participants, and anecdotally improves participation in the meeting. It would be amazing if someone had a link to a page with the MSF story, as that is a great reference to have! My google-fu hasn’t helped me in this ca…

For prosterity, the original report on the pilot program with the checklist including introducing names of participants (doi 10.1056/NEJMsa0810119).

Possibly popularised by Atul Gawande “The Checklist Manifesto”.

Meta-comment: LLMs continue to impress me in the capabilities with unearthing information from imprecise inputs/queries.

Re: How SQLite is tested

#40
post #13
post #5

Always makes me a bit envious as well as awestruck. What a joy it must be in a lot of ways to be able to grind and perfect a piece of software like this. Truly a work of craftsmanship.

You can literally just do this. I’ve never gotten fired from a software engineering job for moving slower and building things that work well, work predictably, and are built to last. Over a career of working at it, you get dramatically better at higher levels of quality even in earlier passes, so the same level of added effort provides increasing levels of reward as you gain experience. Nobody ever complains about th…

> Nobody ever complains about the person who’s leaving everything they touch a little cleaner than how they found it.

This should be true, but it's not in my experience. Even small, clear improvements are rejected as off-mission or shunted to a backlog to be forgotten. Like, "cool, but let's hold off on merging this until we can be certain it's safe", or in other words "this is more work for me and I'd really rather not".

Post reply on HN