Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PostgreSQL benchmarks under giant lock free DragonflyBSD

leaf.dragonflybsd.org

Post reply on HN
Print view View original

11–15 of 15 posts

Re: PostgreSQL benchmarks under giant lock free DragonflyBSD

#11

Post by dmarble »

gcp wrote: post #5
JoachimSchipper wrote: post #2

Summary/background: DragonflyBSD split from FreeBSD (partly) over multicore strategy, FreeBSD going with threads and fine-grained kernel locks and DragonflyBSD going with a microkernel and multiple processes (not entirely dissimilar to GNU Hurd.) This post shows a standard PostgreSQL benchmark on a big machine (2 cpus * 6 cores/cpu * 2 hardware threads/core, 96GB RAM) and the news is that Dragonfly is approaching Fre…

One addendum to this benchmark (somewhere else in that thread) shows OpenSolaris mostly outperforming Linux.

I think you mean OpenIndiana (http://openindiana.org/)

Benchmark results from the above thread: http://leaf.dragonflybsd.org/mailarchive/kernel/2011-11/pdf0...

From http://wiki.openindiana.org/oi/Frequently+Asked+Questions/:

Is OpenIndiana a "fork"?

Kind of.

OpenIndiana aims to be binary and package compatible with the upcoming Solaris 11 and Solaris 11 Express, and most of the operating system is built from source code that Oracle continues to make available. So in some ways, our relationship is similar to the way the CentOS project tracks Red Hat Enterprise Linux.

However there are differences, for example we intend to use illumos in place of the official OS/Net consolidation which is no longer open source, and we also intend to provide improvements on top.

What is the relationship between OpenIndiana and illumos?

OpenIndiana provides a complete, ready-to-use operating system (distribution) similar to the former OpenSolaris distribution.

Re: PostgreSQL benchmarks under giant lock free DragonflyBSD

#12

Post by gaius »

seclorum wrote: post #8
m_eiman wrote: post #6

Earlier quoted context omitted.

It was also the less stable. I discovered more than once the machine was completely frozen (blank screen, no keyboard or network response). Even a NMI couldn't change anything. Sounds like a rather hard crash to me.

Were database transactions continuing while the terminal was non-responsive? Yes. Thus, its not really crashed.

Hmm. I remember back in the day, when the display driver could crash on an NT 3.51 box, but SQL Server would still be up. We treated that as a crash.

A machine that was not responding on the network is getting its database transactions from where exactly? It's crashed, even if the disks are still spinning.

Re: PostgreSQL benchmarks under giant lock free DragonflyBSD

#13

Post by foobarbazetc »

When you're "crashing" Scientific Linux (which I presume using a non-modified RHEL 6 kernel) then you're doing something wrong.

We run far bigger PostgreSQL DBs than this, and we've never had a RHEL crash. Ever.

Maybe there's something wrong with the hardware?

Additionally, if fsync = on in this test, then you're mixing a file system test with a database test.

Re: PostgreSQL benchmarks under giant lock free DragonflyBSD

#14

Post by fdr »

foobarbazetc wrote: post #13

When you're "crashing" Scientific Linux (which I presume using a non-modified RHEL 6 kernel) then you're doing something wrong. We run far bigger PostgreSQL DBs than this, and we've never had a RHEL crash. Ever. Maybe there's something wrong with the hardware? Additionally, if fsync = on in this test, then you're mixing a file system test with a database test.

> Additionally, if fsync = on in this test, then you're mixing a file system test with a database test.

Which is reasonable, given that file systems of any reasonable performance level (read: not user-space file systems) still tend to be bundled with one or at most a handful of reasonable kernel implementations. Turning fsync off would highlight some interesting bottlenecks, but I'm not sure if that makes for the most sensible comparison either. Not only is the fsync/checkpointing period of a database's operation important to model in a broad-spectrum benchmark, but some amount of file system coupling is unavoidable. One example is metadata syscalls to the file system, which have been a point of poor concurrency before[0]

I agree the fsync = off benchmark would be interesting, but I don't think it's fair to say that the information of an fsync = on test is a) less valuable/clearly improperly performed b) necessarily indicative of the most important bottlenecks for database workloads. A kernel that takes a huge lock during fsync(), for example, should be penalized for it.

Finally, fsync = off doesn't accomplish the stated goal of "independence" from the file system. And as seen in Robert Haas's post, non-fsync file system calls can be the problem in high concurrency situations.

[0] http://rhaas.blogspot.com/2011/08/linux-and-glibc-scalabilit...

Re: PostgreSQL benchmarks under giant lock free DragonflyBSD

#15

Post by einhverfr »

m_eiman wrote: post #6
einhverfr wrote: post #4

Earlier quoted context omitted.

I don't know that the issues reported can be called "crashes." Rather it is locking up, probably because there are not enough resources reserved for the terminal and therefore the terminal becomes unresponsive while waiting for an opportunity to run.

It was also the less stable. I discovered more than once the machine was completely frozen (blank screen, no keyboard or network response). Even a NMI couldn't change anything. Sounds like a rather hard crash to me.

Not a hard crash because the computer eventually would recover on its own as load lightens up. It's just a matter of interactive tasks not getting the priority you want them to have.
Post reply on HN