Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New in PostgreSQL 10

wiki.postgresql.org

Post reply on HN
Print view View original

221–230 of 258 posts

Re: New in PostgreSQL 10

#221

Post by wjossey »

manigandham wrote: post #143
wjossey wrote: post #133

Earlier quoted context omitted.

I’m not sure I agree with this statement. BigQuery is not in memory, nor is redshift (not super familiar with snowflake). Whereas memsql is and would be prohibitively expensive to operate on datasets that are analytics based, because it is in memory based. It’s my understanding that citus is best suited to analytics workloads and parallel queries, and really isn’t comparable to memsql as it is attempting to provide s…

MemSQL has in-memory rowstores and disk-based columnstores. Every columnstore table has a rowstore to aid in fast data ingest. Rowstores are meant for access of individual rows, lots of data updates, and smaller reference tables while columnstores are for the bulk of data requiring large scans and analysis. We have tables with 20 billion rows that take up BigQuery streaming ingest also uses in-memory buffering into B…

Thanks for the education on MemSQL. I wasn’t aware of the analytics use case (had never come up in discussions I was a part of for that tech), so it was news to me.

This article had a nice overview. Somewhat outdated but mostly still up to date. https://lnkd.in/g4anCf5

I’ll say it seems precarious to me to mix and match row store in memory data with a columnar store. Is it never a performance issue to have these two systems potentially in contention with one another for resources?

Re: New in PostgreSQL 10

#222

Post by virmundi »

jeltz wrote: post #211
brlewis wrote: post #202

Earlier quoted context omitted.

Not for DBAs, but I also don't expect candidates to get all the answers in that comment. It's a decent data structure question to ask why databases generally use binary trees for indexes when hash lookups are faster. Non-DBAs usually have to think a little bit.

Databases do not use binary trees for indexes and tables, they use B-trees. B-trees are a family of self balancing tree data structures which usually are not binary because in a database you want the tree to have few levels.

Thank you. I remembe having to balance a B-Tree in my DB final. 5 nodes deep. Had to show all of the work. I think it ended up with 20 some revisions based on all the insert and delete statements.

Re: New in PostgreSQL 10

#223

Post by user5994461 »

jacques_chester wrote: post #198
user5994461 wrote: post #191

Earlier quoted context omitted.

And citus will be a nightmare to setup and maintain, for what elasticsearch could have done easier and better out of the box. Use the right tool for the job.

I was pointing out that 2TB/day is not a very high write volume for modern OSS RDBMSes, especially distributed ones. As for elasticsearch, I have found it to be temperamental. The "out of the box" experience was usually unhappy. For logging I've seen people move to other tools (splunk, stackdriver) more than once.

I have found elasticsearch to be one the easiest database I ever had to setup and maintain. Some tuning to do when you have TB on multiple boxes but nothing hardcore.

2 TB/day is fairly easy... as long as you have a bit of experience AND one of the database that can scale horizontally AND the company buys the amount of hardware that is required.

StackDriver/Sumologic/logentries are cloud solutions, so obviously one doesn't have to deal with the database ever.

Re: New in PostgreSQL 10

#224

Post by unixhero »

KingMob wrote: post #207
unixhero wrote: post #186

Earlier quoted context omitted.

This needs to become a thing. Where people produce writeups on Oracle To Postgres, and how much better it is; under something like #RunsMuchBetterWithPostgres ... It is strange that there aren't more writeups on Postgresql migrations. Yandex had a good one. Posted here a long while ago: https://news.ycombinator.com/item?id=12489055

> It is strange that there aren't more writeups on Postgresql migrations. It's probably because switching databases is very painful and rare.

I would think that not having to pay the Oracle/MS/Sybase/Teradata licenses are a good enough incentive...

Speaking of economic incentives.

That could be a thesis topic for my hereto unwritten PhD in Behavioral Economics:

"Choosing to pay" What are the behavioral aspects of top leadership - Why do people still chose proprietary" A quantitative case study."

Re: New in PostgreSQL 10

#225

Post by Ygg2 »

unixhero wrote: post #224
KingMob wrote: post #207

Earlier quoted context omitted.

> It is strange that there aren't more writeups on Postgresql migrations. It's probably because switching databases is very painful and rare.

I would think that not having to pay the Oracle/MS/Sybase/Teradata licenses are a good enough incentive... Speaking of economic incentives. That could be a thesis topic for my hereto unwritten PhD in Behavioral Economics: "Choosing to pay" What are the behavioral aspects of top leadership - Why do people still chose proprietary" A quantitative case study."

If I had to wager, it's support. Oracle/MS generally provide much better support that open source software.

Re: New in PostgreSQL 10

#226

Post by manigandham »

_jal wrote: post #209
manigandham wrote: post #129

Earlier quoted context omitted.

SQL Server does use it's own mini-OS layer and this what lets it virtualize and run on Linux too, so there are higher system requirements. It seems you're just guessing on that last part though - SQL Server has been in production for decades and already has lots of tooling and there are even new CLIs available for the Linux environment. Production SQL Server comes with support from MS, you're definitely not on your o…

"Assumptions and hyperbole"? That MSSQL isn't available as a service like RDS? That tooling and integration that's been building up for decades around PG (or Mysql/Maria for that matter) doesn't exist? These are just facts. I'm sure you're right that MS would be happy to sell support. What I don't see is a reason to care, unless I run in to a Windows app that can't use another DB that I suddenly need to run under Lin…

I'm not sure what you're arguing. MSSQL is available on RDS and to quote the parent comment:

> ...SQL Server on Linux for your production instances, but you're well off the beaten path...first advice you'll hear seems likely to be "try it on Windows and see if it reproduces there"...

You dont think this is hyperbole? Have you actually used SQL Server or Microsoft support before? They do not treat enterprise customers this way and both Linux and MSSQL have plenty of tooling available. There's nothing challenging here.

> What I don't see is a reason to care

You're not the target market then. MSSQL has some compelling features and a big ecosystem, adding cross-platform support and a much easier installation process only helps customers with more options for deployment.

Re: New in PostgreSQL 10

#227

Post by BOBOTWINSTON »

tatersolid wrote: post #201
Illniyar wrote: post #173

Earlier quoted context omitted.

I hope those are interview questions for dbas because it's way outside common knowledge for backend developers.

Why are back-end developers not supposed to have a deep understanding of their tools? I hear this sort of statement all the time, it’s infuriating. I once even had a dev team lead claim that it was not a dev’s responsibility at all if the app generates bad SQL; the DBA is supposed to make the queries fast (this person no longer works for me, but I consistently hear similar claims from consultants and others in the de…

I think there is a very large gap between being able to answer that interview question effectively and having decent understanding of how to not misuse your ORM.

Specifically with ORMs, It does seem that a lot of devs don't understand or care about the generated SQL... but that question was way above the knowledge level I would expect/want out of someone IMO.

Re: New in PostgreSQL 10

#228

Post by takeda »

ww520 wrote: post #169
takeda wrote: post #149

Earlier quoted context omitted.

I would strongly discourage this kind of solution. DRBD is to replicate disk. As pointed out is not Postgres specific, and it doesn't understand the data it replicates. One of nightmare of replication is that there's latency and data never makes it instantly to the other host. This means that there is a possibility that once the master fails, the database file on the other host might be in a corrupted state. If your…

DRBD's replication is synchronous. That means the block device's write() call on the primary won't return until the data is written onto the standby's disk. It's is as strong a guarantee as you can get. In case of the secondary not available, the data are queued up in the primary and sync up with the secondary when it comes back online. If the primary's disk is destroyed at that time, the failure case and data loss i…

I guess that's better, but it would just decrease performance.

I still think using binlogs is safer and more efficient though, since it transfers only what's needed to make an update.

Re: New in PostgreSQL 10

#229

Post by irrational »

KingMob wrote: post #207
unixhero wrote: post #186

Earlier quoted context omitted.

This needs to become a thing. Where people produce writeups on Oracle To Postgres, and how much better it is; under something like #RunsMuchBetterWithPostgres ... It is strange that there aren't more writeups on Postgresql migrations. Yandex had a good one. Posted here a long while ago: https://news.ycombinator.com/item?id=12489055

> It is strange that there aren't more writeups on Postgresql migrations. It's probably because switching databases is very painful and rare.

This. It was worth it, but it still took us the better part of a year.

Re: New in PostgreSQL 10

#230

Post by irrational »

patates wrote: post #183
irrational wrote: post #154

Earlier quoted context omitted.

Truly. We've recently moved from Oracle (after using it for 15 years) to Postgresql. It's like a breath of fresh air. The documentation for Postgres is unbelievably superior to Oracle. So far its performance is equal to or better than Oracle. We had to go through and rewrite thousands of queries, but the sql syntax of Postgres was always simpler and more logical than the equivalent in Oracle (I think Oracle has too m…

Do you plan to write more about the migration, like a blog post? That would be very interesting to read.

It would take a lot more than a single blog post. It would probably take dozens of blog posts (or a book). It took us the better part of a year. There are just so many things that have to be taken into consideration and learned.
Post reply on HN