Live data from Hacker News

PostgreSQL 11.3 and 10.8

postgresql.org

131–140 of 162 posts

Re: PostgreSQL 11.3 and 10.8

#131

Earlier quoted context omitted.

> If you want to support several different expressions you need to create a new column each time Yes and no. Generated columns in MySQL can optionally be "virtual". An indexed virtual column is functionally identical to an index on an expression. > Additionally, an ALTER TABLE blocks access to the table. It depends substantially on the specific ALTER and version of MySQL. Many ALTERs do not block access to the table…

> What's the difference, functionally speaking? (Asking honestly, not being snarky -- I may not understand what you are saying / what the equivalent postgres feature is?) You create a single index, e.g: create index on the_table using gin(jsonb_column); And that will support many different types of conditions, e.g.: check if a specific key/value combination is contained in the JSON: where jsonb_column @> '{"key": "va…

Interesting, thanks! Indeed, there isn't an equivalent feature in MySQL yet.

Out of curiosity, how commonly is this used at scale? I'd imagine there are significant trade-offs with write amplification, meaning it would consume a lot of space and make writes slow. (vs making indexes on specific expressions, I mean. That said, you're right -- there are definitely use-cases where making indexes on specific expressions isn't practical or is too inflexible.)

Re: PostgreSQL 11.3 and 10.8

#132
post #130
post #33

Earlier quoted context omitted.

I quite like Postgres and use it preferentially, but your numbers are off. SQL Server Enterprise costs more like $7500/core; that $15K pack, as far as I am aware, comes with two core licenses. SQL Server Standard 2016 costs $931 for a license if you use CALs ($209 a pop), or $3700/core. Also bear in mind that almost nobody pays list price for any of this.

Having to spend time dealing with the MS licensing mess, rather than building software, seems wasteful and in my experience can be likened to sitting on a cactus for an extended period of time.

I agree. (My own stuff is pretty much all Postgres and the occasional MySQL.) But when you're in an organization where your needs might merit SQL Server, I'd hope you'd have somebody to deal with that for you. ;)

Re: PostgreSQL 11.3 and 10.8

#133

Earlier quoted context omitted.

The benefits are better defaults in terms of data reliability. Hard to say if migration is worth it to you without a lot more details, but I'd definitely recommend trying it in a new project.

Frankly, data reliability concerns with modern MySQL / InnoDB are very outdated FUD. Many of the largest tech companies rely on MySQL as their primary data store. They would not do so if it was unreliable with persistence. There are many valid reasons to choose Postgres over MySQL, or vice versa -- they have different strengths and weaknesses. But there are no major differences regarding data reliability today, nor h…

I haven't kept up, but I remember having constrains that were accepted by the parser, but ignored by the engine (using InnoDB, I don't remember what specific constrain was concerned).

Is it still the case?

Re: PostgreSQL 11.3 and 10.8

#134

Earlier quoted context omitted.

If there is an analytics db/replica, your data analysts will worship the ground you walk on if you migrate from MySQL to Postgres.

Interesting, can you elaborate, I'm considering a switch.

Please humor me if I sound a bit triggered in this response, as MySQL pre-v8.0 is like if the creators went to data analysts and asked: "what features could we remove, and what changes could we make to an SQL-standard-compliant DB (postgres), in order to slow you down as much as possible, hinder your ability to read and write queries, and generally make your life a living hell?"

I really get that MySQL is good for what it does, from an engineer's point of view. It is an absolute piss-poor excuse for a database, prior to v8.0.

So what's wrong with MySQL (again, prior to v8.0, but no one seems to use the damn current version)

-Not ANSI SQL compliant (unlike Postgres)

-No CTEs/WITH clause (?!)

-no WINDOW FUNCTIONS (?!?!?!?)

-"schemas are called databases" which makes for bizarre interpretation of `information_schema` queries, which behave the same across all other DBs except mySQL. What I mean to say is MySQL calls each schema it's own database. This results in having to connect the same DB multiple times to other programs/APIs/inputs which accept JDBC.

-Worse replication options than postgres, not default ACID compliant,

-Don't know the programming term for this... but the horrendous "select col1, col2, col3... colN, count() from table group by 1" implicit group by. Meaning the system takes your INVALID query, and does things underneath the hood to return a result. Systems should enforce correct syntax (you must group by all non-aggregation columns... mysql implicitly does this under the hood).

-on a tangentially related note to the prior one, MySQL returns null instead of a divide by zero error when you divide by zero. Divide by zero errors are one of the few things that should ALWAYS RETURN AN ERROR NO MATTER WHAT -mysql doesn't support EXCEPT clauses

-doesn't support FULL OUTER JOIN

-doesn't support generate_series,

-poor JSON support

-very limited, poor array/unnest support

-insert VALUES () (in postgres) not supported

-lack of consistent pipe operator concatenation,

-weird datatype suppport and in-query doesn't support ::cast

-doesn't support `select t1._* , t2.field1, t2.field2 from t1 join t2 on t1.id = t2.id` ; that is, you cannot select * from one table, and only certain fields from the other.

-case dependence in field and table names when not escape quoted (mysql uses backtick, postgres uses double quote for escaping names). What the fuck is this? SQL is a case-insensitive language, then the creators build-in case sensitivity?

-As I mentioned above, mysql uses backticks to escape names. This is abnormal for SQL databases.

-mysql LIKE is case-insensitive (what the hell, it's case-sensitive everywhere else). Postgres has LIKE, and ILIKE (insensitive-like).

-ugly and strange support for INTERVAL syntax (intervals, despite being strings, give a syntax error in mysql. Example: In postgres or redshift etc you would right `select current_timestamp - interval '1 week'. In MySQL, you'd have to do `select current_timestamp - interval 1 week` (the '1 week' could be '7 month' or '2 day'... it's a string, and should be in single quotes. MySQL doesn't do this)

-mysql doesn't even support the normal SQL comment of `--`. It uses a `#` instead. No other database does that.

-probably the worst EXPLAIN/EXPLAIN ANALYZE plans I've ever seen from any database, ever

-this is encapsulated in the prior points but you can't do something simple like `select , row_number() as rownum from table`. Instead you have to declare variables and increment them in the query

-did I mention it's just straight up not SQL standard compliant?

At least MySQL 8.0 supports window functions and CTEs (seriously it's a death knell to a data analyst not to have these). They are the absolute #1 biggest piece of missing functionality to an analyst in my opinion.

This entire post focused on "mySQL have-nots", rather than "Postgres-haves" so I do think there are actually _even more_ advantages to using Postgres over MySQL. I understand MySQL is very fast for writes, but to my understanding it's not even like Postgres is slow for writes, and on the querying side of the coin, it's a universe of difference.

If you ever use MySQL in the future and there will be a data analyst existing somewhere downstream of you, I implore you to use MySQL v8.0 and nothing older, at any cost, for their sake.

Re: PostgreSQL 11.3 and 10.8

#135

Earlier quoted context omitted.

I upgraded from 9.3 → 11.2 a few months ago using pg_upgrade[1], on a master+slave database with 150GB of data. I did a fair amount of testing, but the final procedure was very fast and smooth. 1. Test the upgrade: set up an additional secondary (9.3), break the replication link (promote it to a master). Test the upgrade on that. It was really fast, under 30 seconds to shut down the old DB, run the in-place upgrade,…

This is a nice way to do that, but you have a low volume of data, and you think 30 seconds is fast and 1 minute of downtime is acceptable. I question these assumptions. Consider the situation when you're adding thousands of new records per seconds, and the database is being used every second (quite literally: to compute per seconds statistics). A better solution is to have triggers on the old master, to do the same i…

That's a good plan for someone with a larger or busier database, but my approach was the correct solution for us. You can be satisfied that your taxes aren't wasted by us implementing anything more complicated :-)

(Most of the writes to our PostgreSQL server are batch processes, decoupled with a message broker, and the message consumers were easily paused during the upgrade.)

Re: PostgreSQL 11.3 and 10.8

#136
post #100

Question for PG happy users. How do you manage failover and replication? At my previous job this was done by a consultant. Is this doable on a self hosted setup? Thank you in advance.

One option for automated health monitoring and failover is pglookout: https://github.com/aiven/pglookout

We use this, pgbouncer, and a bash script to link the two, for completely automated failover.

Queries done through pgbouncer just pause as if the query is really really slow when the db goes down, then when pglookout does the failover, the bash script switches pgbouncer's config and those pending queries are sent immediately.

Re: PostgreSQL 11.3 and 10.8

#137
post #133

Earlier quoted context omitted.

Frankly, data reliability concerns with modern MySQL / InnoDB are very outdated FUD. Many of the largest tech companies rely on MySQL as their primary data store. They would not do so if it was unreliable with persistence. There are many valid reasons to choose Postgres over MySQL, or vice versa -- they have different strengths and weaknesses. But there are no major differences regarding data reliability today, nor h…

I haven't kept up, but I remember having constrains that were accepted by the parser, but ignored by the engine (using InnoDB, I don't remember what specific constrain was concerned). Is it still the case?

It was the case until very recently. Check constraints are now supported in MySQL 8: https://dev.mysql.com/doc/refman/8.0/en/create-table-check-c...

I believe MariaDB added support for them a couple years earlier, but am not certain.

More broadly, I would agree it's a very painful "gotcha" to have aspects of CREATE TABLE be accepted by the parser but ignored by the engine. However, in MySQL's defense, theoretically this type of flexibility does allow third-party storage engines to support these features if the engine's developer wishes.

Ideally, the engine should throw an error if you try using a feature it does not support, but in a few specific cases it does not (at least for InnoDB). This can be very frustrating, for sure. But at least it's documented. And no database is perfect; they all have similarly-frustrating inconsistencies somewhere.

Re: PostgreSQL 11.3 and 10.8

#138
post #92

Earlier quoted context omitted.

Be warned that in PostgreSQL, WITH is an optimization barrier, and is planned to remain that way to serve that purpose. If you can, prefer using views to enhance readability (and testability as a bonus). PostgreSQL views (unlike those in MySQL) do not prevent optimization across them.

No, CTEs are not planned to remain a barrier, this is already fixed in the next version which is in feature freeze right now. https://www.depesz.com/2019/02/19/waiting-for-postgresql-12-...

Oh wow, that is news to me! A welcome change.

Re: PostgreSQL 11.3 and 10.8

#139
post #13

For those stuck on older versions of Postgres, I highly recommend paying the downtime to upgrade. Going from 9.x to 11 will get you a measurably large performance gain for free.

I've just upgraded my hobby app from 9.6 to 11 and some of my old queries are now timing out :-| I'll update this thread when I find out why.

    analyze
fixed pretty much everything :)

Re: PostgreSQL 11.3 and 10.8

#140

Earlier quoted context omitted.

Interesting, can you elaborate, I'm considering a switch.

Please humor me if I sound a bit triggered in this response, as MySQL pre-v8.0 is like if the creators went to data analysts and asked: "what features could we remove, and what changes could we make to an SQL-standard-compliant DB (postgres), in order to slow you down as much as possible, hinder your ability to read and write queries, and generally make your life a living hell?" I really get that MySQL is good for wh…

I'm not sure what you mean by the "star from one table, specific columns from another" issue... if I'm understanding correctly I'm pretty sure MySQL allows just that.

Division by zero errors and non-"magical" GROUP BY have been the default mode of operation for a _little_ longer, since the 5.7 series.

Post reply on HN