Earlier quoted context omitted.
I haven't read their analysis yet (I will try to when I have some free time), but in general, I would argue that trying to compare a document database to a SQL one is always going to be somewhat misleading. I'd care more if they were comparing Clustrix to MSSQL, MySQL, PostgreSQL. If you are using MongoDB in a way that is similar to the way you would have used a SQL DB you are probably doing something wrong. Specific…
Sergei compares Clustrix to MongoDB because their target markets are very similar--not because the technology is similar. As a startup, it behooves them to attack the low-end database market, but I suspect they've found that the primary market for a highly scalable low-end database lies on the web, but that market has chosen to go cheap-and-dirty with NoSQL. So now they're in the middle ground between fast-and-and-lo…
Looking at a coarse, "they want money from people" granularity, the target market (people who need a database) may be similar. If we look closer, however, the target market has, more or less, two segments:
One is at the product-for-free pricepoint, where you make your money either by selling additional services (e.g. 10gen with MongoDB, Basho with Riak, MontyProgram with MariaDB and MySQL) or upselling them to an enterprise version (e.g. IBM's DB2 Express-C, whose 2GB limitation makes it perfect to hook people on a workload they would use MySQL for, Franz' AllegroStore, OpenLink's Virtuoso open source edition).
The try-our-product-for-free method means that all the cheap-and-dirty-folks have something in their grubby fingers to build the next blog, mom-and-pop online store, or whatever. The folks with actual money to spend can lower their initial risk by trying out a couple of different databases to see which one fits best, without even having to ask. Only when they're actually happy with what they've got they're going to fork over the cash.
The next tier is the "we'll have to qualify you before we send our sales engineer" tier where you play with Oracle and IBM, or Greenplum and Vertica, because your prospective customers already know that your product is good enough.
There's no real space in-between. Either you have an engineer with good knowledge but no discretionary budget (whose chooses something appropriate for the task, after testing on a real workload, and makes the DB a non-topic for everyone else), in which case a comparison that people cannot replicate is not going to help you, or you have a high-level decision taker with budget power but no time to try things out or risk a couple thousand on a startup that may have gone bancrupt when he most cares about it. This latter category will be thoroughly unimpressed by any benchmark that is not the TPC-C or similar. No lottery whatsoever involved, these are the rules of enterprise spending.
BTW, calling ColdFusion and PHP (which were a far superior alternative to writing Perl CGI scripts without templating or any kind of library support) shitty and Clustrix "the right approach" is something for which people in 2024 will just laugh at you, even if Clustrix manages to do extremely well end ends up as the second-shittiest solution to a common problem.