Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bullshit graph database performance benchmarks

maxdemarzi.com

Post reply on HN
Print view View original

111–118 of 118 posts

Re: Bullshit graph database performance benchmarks

#111

Post by HyperSane »

cerved wrote: post #45
abdullah2993 wrote: post #40

Earlier quoted context omitted.

They are useful specifically in the intelligence field like NSA(no wonder they have so much graph stuff opensourced). Let me share one obvious use case you have data on a lot of people like call data records, Facebook friends list, Twitter followers/following list and potentially a lot of other data as well. Now you have two targets person A and person B with graph databases it is a trivial one liner to find how thes…

I don't think recursive CTEs are that bad

Have you tried Cypher?

Re: Bullshit graph database performance benchmarks

#112

Post by makeitdouble »

LLcolD wrote: post #7
PreInternet01 wrote: post #3

While this seems to be a pretty egregious example of a vendor benchmark misleading through cherry-picked unrealistic results, I'm not sure I share the author's pessimism about how these kinds of stunts will hold back the graph database market. Why? Simple: pretty much any benchmark I've seen of anything, ever, was similar nonsense -- give people numbers to game and they'll do so, enthusiastically. Even supposedly gol…

Yes, running a benchmark on your data is the only way. I've taken a look at both benchmarks (the one from OP and the one from Memgraph). They seem like different types of benchmarks and different approaches. But I still find it interesting that although the numbers in OP's are not so much in favor of Memgraph it turns out that Memgrpah is faster than Neo4j in large number of benchmark queries. So yes, it all comes do…

He didn't run the Memgraph one, only took their numbers https://news.ycombinator.com/item?id=34368362

I'd assume it's what the disclaimer at the top was about: that the code is in Python which he's not familiar enough with. The license bit on not having the right to integrate it if you're building a competing product might not be decisive but a good enough reason to not invest more time into running the code.

Re: Bullshit graph database performance benchmarks

#113

Post by ramraj07 »

darksaints wrote: post #90
ramraj07 wrote: post #78

Earlier quoted context omitted.

Absolutely no one, but doesn’t mean you look down on one of the most popular languages as if it’s beneath you.

So what you're saying is that he is opinionated about popular languages just like every other developer in the world. You don't think you can trust him to write databases because he doesn't like a language that he would never use to write a database. Weird. Linus Torvalds also hates one of the most popular languages in the world: C++. He must not know anything about operating systems...

You can hate python or c, but he didn’t say he hated it, he said he didn’t KNOW it. Do you think Linus doesn’t know python or c++?

Re: Bullshit graph database performance benchmarks

#114

Post by taubek »

manv1 wrote: post #95

The real problem with these kinds of "benchmarks" is that either the company doesn't have anyone on staff that's calling "bullshit" on it or the marketing people don't care that it's bullshit. Either one is a bad sign if they're going to be a vendor. At that point how can you trust their SLAs and/or their presales team?

In this thread I've seen comments from Memgraph CTO and founder (mbuda), CEO and founder (dtomicevic), and one of the developers that worked on mbench (mapleeman). To me it seems that they are addressing all of the questions in comments section.

Re: Bullshit graph database performance benchmarks

#115

Post by SahAssar »

nemo44x wrote: post #75
college_physics wrote: post #51

my feeling is that graph databases face an uphill battle for mass adoption not because their architects or vendors doing anything wrong but some intrinsic aspects of information exchange in most current situations and use cases * information tends to be private and/or commercially sensitive, this severs the links that graph dbs are good at representing (and made the "node focused" SQL approach the ubiquitous model th…

I don’t know, they’re the fastest growing database segment by far according to DBEngines. I saw very niche use cases 10 years ago but have seen more and more common use cases recently. Knowledge Graphs in particular have a lot of use. I’d expect to continue to see explosive growth in graph over the next 10 years. As ubiquitous as SQL or document databases? Probably not. But very likely more and more use cases will be…

According to DBEngines MS Access is also more popular than SQLite. I wouldn't trust their methodology.

Re: Bullshit graph database performance benchmarks

#116

Post by darksaints »

ramraj07 wrote: post #113
darksaints wrote: post #90

Earlier quoted context omitted.

So what you're saying is that he is opinionated about popular languages just like every other developer in the world. You don't think you can trust him to write databases because he doesn't like a language that he would never use to write a database. Weird. Linus Torvalds also hates one of the most popular languages in the world: C++. He must not know anything about operating systems...

You can hate python or c, but he didn’t say he hated it, he said he didn’t KNOW it. Do you think Linus doesn’t know python or c++?

He has literally never talked about python. It's probably safe to assume he doesn't know it.

Re: Bullshit graph database performance benchmarks

#117

Post by mhio »

aeyes wrote: post #86
junon wrote: post #82

Earlier quoted context omitted.

I don't think that's right. That's not how I understood the article.

The author published the code and I only see adapters which interface with Neo4J: https://github.com/maxdemarzi/memgraph_benchmark/tree/main/s... The numbers for Memgraph match what is shown on their benchmark website.

The author confirmed only neo4j tests in a follow up comment.

Re: Bullshit graph database performance benchmarks

#118

Post by redbar0n »

maxdemarzi wrote: post #87

Author here to clear up a few questions: I did not run any benchmarks for Memgraph, just Neo4j on my machine and compared them to their numbers on their machine. My 8 faster cores to their 12 slower cores, so not apples to apples, but close enough to make the point that Memgraph is not 120x times faster than Neo4j. I used to work at Neo4j, then at AWS for Neptune, I work on my own graph database http://ragedb.com/ ,…

> If you want to be my hero, find a way to fix this problem: https://maxdemarzi.com/2023/01/09/death-star-queries-in-grap...

Let me (try to) be your hero, Marzi. (Insert favorite reference to famous cheezy pop music song, if you like.)

Couldn't you use GraphBLAS algorithms, like they do in RedisGraph (which supports Cypher, btw) to fix that problem with "death star" queries?

Those algorithms are based on linear algebra and matrix operations on sparse matrices (which are like compressed bitmaps on speed, re: https://github.com/RoaringBitmap/RoaringBitmap ). The insight is that the adjacency list of a property-graph is actually a matrix, and then you can use linear algebra on it. But it may require the DB is built bottom up with matrices in mind from the start (instead of linked lists like Neo4j does). Maybe your double array approach in RageDB could be made to fit..

I think you'll find this presentation on GraphBLAS positively mind-blowing, especially from this moment: https://youtu.be/xnez6tloNSQ?t=1531

Such math-based algorithms seem perfect to optimally answer unbounded (death) star queries like “How are you connected to your neighbors and what are they?”

That way, for such queries one doesn't have to traverse the graph database as a discovery process through what each node "knows about", but could view and operate on the database from a God-like perspective, similar to table operations in relational databases.

Further reading: https://graphblas.org/

Post reply on HN