Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bullshit graph database performance benchmarks

maxdemarzi.com

Post reply on HN
Print view View original

31–40 of 118 posts

Re: Bullshit graph database performance benchmarks

#31

Post by dtomicevic »

LLcolD wrote: post #7
PreInternet01 wrote: post #3

While this seems to be a pretty egregious example of a vendor benchmark misleading through cherry-picked unrealistic results, I'm not sure I share the author's pessimism about how these kinds of stunts will hold back the graph database market. Why? Simple: pretty much any benchmark I've seen of anything, ever, was similar nonsense -- give people numbers to game and they'll do so, enthusiastically. Even supposedly gol…

Yes, running a benchmark on your data is the only way. I've taken a look at both benchmarks (the one from OP and the one from Memgraph). They seem like different types of benchmarks and different approaches. But I still find it interesting that although the numbers in OP's are not so much in favor of Memgraph it turns out that Memgrpah is faster than Neo4j in large number of benchmark queries. So yes, it all comes do…

Memgraph compared the freely available open source editions of both databases. Neo4j Enterprise seems to have more performance optimizations compared to the community Edition.

Re: Bullshit graph database performance benchmarks

#32

Post by LLcolD »

alias_neo wrote: post #27
dwroberts wrote: post #23

The author works on RageDB ( https://ragedb.com/ ) and this doesn't seem to be disclosed in the article

I'm not sure why it would need to be "disclosed", other than to suggest the author knows what they're talking about due to "domain knowledge".

He does "disclose" in related blog post [1]: "I don’t work for Neo4j anymore, why am I here defending them? Well… that and the fact that I still have a dinghy load of vested shares I have to sell so I can buy a place in the Villages and begin a new life as a golf cart driving day drinker."

This seems like a series of post on benchmarking results from different vendors so if he "disclosed" it once I don't think that there is need for another one.

[1] https://maxdemarzi.com/2022/12/06/khop-baby-one-more-time/

Re: Bullshit graph database performance benchmarks

#33

Post by jojobas »

chaps wrote: post #29
alias_neo wrote: post #27

Earlier quoted context omitted.

I'm not sure why it would need to be "disclosed", other than to suggest the author knows what they're talking about due to "domain knowledge".

It's because he has reason to destroy other competitors. Nice read otherwise.

He "destroys" one competitor as much as he praises another. He's never comparing either with ragedb.

Re: Bullshit graph database performance benchmarks

#34

Post by rjh29 »

alfiedotwtf wrote: post #8

On a tangent, what Graph Database would people recommend in 2023? In particular, I would like something that's linked in like SQLite rather than a full blown service like MySQL etc

For simple cases, you can get pretty far storing relations 6 times in SQLite, or any old key/value store. (a-b-son, b-a-father, son-a-b, father-b-a, a-son-b, b-father-a)

Re: Bullshit graph database performance benchmarks

#35

Post by t0ddst4vn »

dwroberts wrote: post #23

The author works on RageDB ( https://ragedb.com/ ) and this doesn't seem to be disclosed in the article

Based on this non-emotional and calm post seems like an appropriate db name.

I can see he also worked for Neo4j. I suppose that with all the trouble neo is going through [1], he wanted to protect them.

[1] https://news.ycombinator.com/item?id=33916240

Re: Bullshit graph database performance benchmarks

#36

Post by mcv »

chillfox wrote: post #17
alfiedotwtf wrote: post #8

On a tangent, what Graph Database would people recommend in 2023? In particular, I would like something that's linked in like SQLite rather than a full blown service like MySQL etc

If you only need a few graph queries then you could just use SQLite, it’s capable of doing it (I have done it before). But writing graph queries in SQL is painful, so I wouldn’t do it if you need more than a handful.

Exactly. There's no need for a graph database if you're not going to be searching for complex relationships. And once you do, SQL is hell.

I don't know of any SQLite-like graph database. I'm still a fan of neo4j.

Re: Bullshit graph database performance benchmarks

#37

Post by robertlagrant »

junon wrote: post #21
mhio wrote: post #13

I get that this is trying to point out that neo4j shouldn't be that far behind, but why are the i7/gatling test numbers being directly compared to memgraphs g6 test results? The conclusion is a bit premature without the other half of the test... What performance does memgraph have on the newer, single socket hardware?

Yeah that was strange, it's my understanding that you can't compare benchmarks between different machines, especially if they're not 1:1 identical hardware. If you're referring to this line, then it struct me as very odd. > Instead of 112 queries per second, I get 531q/s. Instead of a p99 latency of 94.49ms, I get 28ms with a min, mean, p50, p75 and p95 of 14ms to 18ms. Alright, what about query 2? Same story. Otherw…

Author is just stating the differences between the benchmarketing hardware and his own. Not comparing new hardware and one DB with old hardware and other DB.

Re: Bullshit graph database performance benchmarks

#38

Post by robertlagrant »

LAC-Tech wrote: post #11

What are people using graph databases for, and what do your queries look like? I've read about them briefly but I have to admit my imagination fails me as to how it would look in the real world.

Graph databases are very cheap to traverse relationships between things, but slower to do per-item-type operations. So finding your friends of friends of friends is cheap, but finding the mean age of everyone in the database is slow.

Re: Bullshit graph database performance benchmarks

#39

Post by robertlagrant »

HyperSane wrote: post #15
LAC-Tech wrote: post #11

What are people using graph databases for, and what do your queries look like? I've read about them briefly but I have to admit my imagination fails me as to how it would look in the real world.

I use Neo4j to create a CMDB that pulls in data from Active Directory, File Shares, Cloudstrike API, Okta API, Windows Services, Processes, and TCP ports, VCenter, Cisco CDP , ARP tables, Routing Tables, and MAC address tables from routers and switches. Powershell get-foo commands combined with the ConvertTo-JSON makes it very easy to import data from Windows. A possible query would be match (host:ESXihost)-[:running…

That's an interesting idea. Having done CMDB stuff in a previous life and also used Neo4J in my last job, I appreciate that one. I don't know whether you'd gain much vs using Postgres with JSON fields, but I bet the ergonomics are better, and if you do need a big relationally recursive query then it'd work well.

Re: Bullshit graph database performance benchmarks

#40

Post by abdullah2993 »

LAC-Tech wrote: post #11

What are people using graph databases for, and what do your queries look like? I've read about them briefly but I have to admit my imagination fails me as to how it would look in the real world.

They are useful specifically in the intelligence field like NSA(no wonder they have so much graph stuff opensourced). Let me share one obvious use case you have data on a lot of people like call data records, Facebook friends list, Twitter followers/following list and potentially a lot of other data as well. Now you have two targets person A and person B with graph databases it is a trivial one liner to find how these two people are linked. They can be linked directly or they could have 5 people between them doing the same in SQL recursive CTE is a major PIA and takes a lot of time(see degrees of kevin bacon using graph database). There are very niche companies that are making big bucks by just selling libraries/softwares just to plot these graphs and most of their customers are government agencies with a lot of funds.
Post reply on HN