41–50 of 56 posts
I remember, waaaaay back in the early days of PostgeSQL, I was using it for project and it crashed in a way that corrupted our data. There was no hardware problem; it was just a database crash. (This was quite some time ago. I don't recall what year it was but I'm 68 now and have been a programmer since my senior year in college.) I switched to MySQL for that project. I assume PostgreSQL is not remotely prone to anyt…
I was interested in learning how to created a postgres extension the other week. Not just bundling some SQL scripts, but a proper extension tying into their API. Trying to find any good information on how to go about this proved super difficult. Well, I wasn't having much luck and just gave up.
through writing extensions luckily isn't
lol, of course the first comment is essentially “it would probably be easier if it were written in Rust”.
I laughed at that, too, but the commenter, Greg Smith (disclosure: a former colleague) has been involved with Postgres for ages, and concludes that Rust would not actually be a magic bullet here.
In Rust it's much easier to create robust and performant abstractions that are near impossible to misuse, eliminating tons of potential bugs right away.
In Rust you don't need 10 years of experience and staring with a microscope and at every line to ensure it doesn't introduce issues. In Rust trivial changes and indeed trivial, so you have more time to think about actually difficult parts.
In Rust contributors don't need to learn custom implementation for collections, string and other basic primitives on every project.
And most of all, people actually want to learn and work with Rust, so your contributor pool is expanding, not shrinking.
There are some striking similarities to working on another large OSS codebase: Mozilla. (I am employed there.) We have struggled with all of these things for years, and we have a much larger pool of committers (and thus much higher variance in committer abilities). Things today are much better than they used to be, even if all of the same problems are still present to some degree. Some of what we did might translate…
Wouldn't the testing basically consist of executing large amounts of SQL against an initially empty database, and then executing more SQL to read the current state of the data and verifying it is like it should be? Such tests could perhaps be database-agnostic to a degree, verifying that the database behaves according to the SQL-standard?
I was thinking more like lots of concurrent operations, and backups/restores (again concurrent with other DB traffic), and replication, and incremental operations, and failover, and error handling in general. All while varying things that feed into scheduling, etc. Anything nondeterministic is good, though that's not all of it. (Annoyingly, that means that failures will quite often be intermittent, which is a whole can of worms of its own.)
I remember, waaaaay back in the early days of PostgeSQL, I was using it for project and it crashed in a way that corrupted our data. There was no hardware problem; it was just a database crash. (This was quite some time ago. I don't recall what year it was but I'm 68 now and have been a programmer since my senior year in college.) I switched to MySQL for that project. I assume PostgreSQL is not remotely prone to anyt…
It can still happen sometimes, but it's very rare that it crashes and even more rare that it corrupts as well. The good news, the chances of it corrupting your data silently was basically zero, then and now. However back then MySQL seemed like it went out of it's way to corrupt your data. The only "bonus" is, it did it all silently, so nobody ever noticed until they went looking. With MariaDB(the successor) it's pret…
Earlier quoted context omitted.
It can still happen sometimes, but it's very rare that it crashes and even more rare that it corrupts as well. The good news, the chances of it corrupting your data silently was basically zero, then and now. However back then MySQL seemed like it went out of it's way to corrupt your data. The only "bonus" is, it did it all silently, so nobody ever noticed until they went looking. With MariaDB(the successor) it's pret…
Could be; but it never happened to me with MySQL! Maybe it depends on on how far back in the day you go...? :)
Could it be that RDBMS is just inherently very complex? Reminds me of this classic comment about Oracle Database: https://news.ycombinator.com/item?id=18442941 To quote part of it > Oracle Database 12.2. > It is close to 25 million lines of C code. > What an unimaginable horror! You can't change a single line of code in the product without breaking 1000s of existing tests. Generations of programmers have worked on th…
In that sort of codebase, the only thing scarier than changing a line of code and breaking thousands of tests, is changing a line of code and not breaking any tests.
Could it be that RDBMS is just inherently very complex? Reminds me of this classic comment about Oracle Database: https://news.ycombinator.com/item?id=18442941 To quote part of it > Oracle Database 12.2. > It is close to 25 million lines of C code. > What an unimaginable horror! You can't change a single line of code in the product without breaking 1000s of existing tests. Generations of programmers have worked on th…
Coming at this from a naive outsider perspective, the central problem described in the post (commits to PostgreSQL frequently have serious defects which must be addressed in follow-up commits) seems like one that would ideally be addressed with automated testing and CI tooling. What kind of testing does the Postgres project have? Are there tests which must pass before a commit can be integrated in the main branch? Ar…
Most database companies run only a small amount of tests before committing. After committing, you run tests for thousands of hours. It sucks. You probably do this all day every day. You just run the tests on whatever you have currently committed. you kind of have to be careful about not adding more tests that make it take much much longer. See https://news.ycombinator.com/item?id=18442941