Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hacking on PostgreSQL is hard

rhaas.blogspot.com

Post reply on HN
Print view View original

31–40 of 56 posts

Re: Hacking on PostgreSQL is hard

#31

Post by sanxiyn »

ncann wrote: post #28

Could it be that RDBMS is just inherently very complex? Reminds me of this classic comment about Oracle Database: https://news.ycombinator.com/item?id=18442941 To quote part of it > Oracle Database 12.2. > It is close to 25 million lines of C code. > What an unimaginable horror! You can't change a single line of code in the product without breaking 1000s of existing tests. Generations of programmers have worked on th…

I think it is not RDBMS, rather combinatorial explosion of configuration/flag/option/platform is insidious and we, software engineering as a field, don't know how to do it well.

I think it is one of the largest impact problem in software engineering if it can be improved. Maybe a way to restrict flag interaction and reduce support and test matrix as a result.

Re: Hacking on PostgreSQL is hard

#32

Post by irjustin »

ncann wrote: post #28

Could it be that RDBMS is just inherently very complex? Reminds me of this classic comment about Oracle Database: https://news.ycombinator.com/item?id=18442941 To quote part of it > Oracle Database 12.2. > It is close to 25 million lines of C code. > What an unimaginable horror! You can't change a single line of code in the product without breaking 1000s of existing tests. Generations of programmers have worked on th…

> The only reason why this product is still surviving and still works is due to literally millions of tests!

Personally I consider this a good thing. It's a sign of a really mature codebase where lots of edge cases are known + accounted for.

Even if the underlying code was really well written, simply the number of edge cases hamstring any "quick hacks".

Complex, runs reliably, easy to hack - Pick two

Re: Hacking on PostgreSQL is hard

#33

Post by garyrob »

I remember, waaaaay back in the early days of PostgeSQL, I was using it for project and it crashed in a way that corrupted our data. There was no hardware problem; it was just a database crash. (This was quite some time ago. I don't recall what year it was but I'm 68 now and have been a programmer since my senior year in college.) I switched to MySQL for that project. I assume PostgreSQL is not remotely prone to anything like that anymore!

Re: Hacking on PostgreSQL is hard

#34

Post by paulddraper »

x0x0 wrote: post #29
paulddraper wrote: post #20

Earlier quoted context omitted.

PostgreSQL is an order of magnitude more complex and extensible than SQLite. Any comparison is basic at best.

I suspect it may be closer to 2 orders of magnitude than 1. eg look at the pg docs re: at least 5 different types of indices are implemented https://www.postgresql.org/docs/current/indexes.html Everything is like that. (I'm not bashing sqlite, it's great software, but it has a tiny fraction of the product footprint that pg has.)

*6 different index types.

And that's just core, not even including built-in extensions like bloom.

Re: Hacking on PostgreSQL is hard

#35

Post by zie »

garyrob wrote: post #33

I remember, waaaaay back in the early days of PostgeSQL, I was using it for project and it crashed in a way that corrupted our data. There was no hardware problem; it was just a database crash. (This was quite some time ago. I don't recall what year it was but I'm 68 now and have been a programmer since my senior year in college.) I switched to MySQL for that project. I assume PostgreSQL is not remotely prone to anyt…

It can still happen sometimes, but it's very rare that it crashes and even more rare that it corrupts as well. The good news, the chances of it corrupting your data silently was basically zero, then and now.

However back then MySQL seemed like it went out of it's way to corrupt your data. The only "bonus" is, it did it all silently, so nobody ever noticed until they went looking. With MariaDB(the successor) it's pretty rare that it silently corrupts your data these days.

Re: Hacking on PostgreSQL is hard

#36

Post by garyrob »

zie wrote: post #35
garyrob wrote: post #33

I remember, waaaaay back in the early days of PostgeSQL, I was using it for project and it crashed in a way that corrupted our data. There was no hardware problem; it was just a database crash. (This was quite some time ago. I don't recall what year it was but I'm 68 now and have been a programmer since my senior year in college.) I switched to MySQL for that project. I assume PostgreSQL is not remotely prone to anyt…

It can still happen sometimes, but it's very rare that it crashes and even more rare that it corrupts as well. The good news, the chances of it corrupting your data silently was basically zero, then and now. However back then MySQL seemed like it went out of it's way to corrupt your data. The only "bonus" is, it did it all silently, so nobody ever noticed until they went looking. With MariaDB(the successor) it's pret…

Could be; but it never happened to me with MySQL! Maybe it depends on on how far back in the day you go...? :)

Re: Hacking on PostgreSQL is hard

#38

Post by galaxyLogic »

sfink wrote: post #30

There are some striking similarities to working on another large OSS codebase: Mozilla. (I am employed there.) We have struggled with all of these things for years, and we have a much larger pool of committers (and thus much higher variance in committer abilities). Things today are much better than they used to be, even if all of the same problems are still present to some degree. Some of what we did might translate…

Wouldn't the testing basically consist of executing large amounts of SQL against an initially empty database, and then executing more SQL to read the current state of the data and verifying it is like it should be?

Such tests could perhaps be database-agnostic to a degree, verifying that the database behaves according to the SQL-standard?

Re: Hacking on PostgreSQL is hard

#39

Post by gzapp »

I was interested in learning how to created a postgres extension the other week. Not just bundling some SQL scripts, but a proper extension tying into their API.

Trying to find any good information on how to go about this proved super difficult. Well, I wasn't having much luck and just gave up.

Re: Hacking on PostgreSQL is hard

#40

Post by taberiand »

ncann wrote: post #28

Could it be that RDBMS is just inherently very complex? Reminds me of this classic comment about Oracle Database: https://news.ycombinator.com/item?id=18442941 To quote part of it > Oracle Database 12.2. > It is close to 25 million lines of C code. > What an unimaginable horror! You can't change a single line of code in the product without breaking 1000s of existing tests. Generations of programmers have worked on th…

In that sort of codebase, the only thing scarier than changing a line of code and breaking thousands of tests, is changing a line of code and not breaking any tests.
Post reply on HN