Live data from Hacker News

Bullshit graph database performance benchmarks

maxdemarzi.com

91–100 of 118 posts

Re: Bullshit graph database performance benchmarks

#91

The author works on RageDB ( https://ragedb.com/ ) and this doesn't seem to be disclosed in the article

The author disclosed it in the first line of the article, albeit in a joke about death row records. Looks like someone needs to brush up on their west coast rap discography.

Re: Bullshit graph database performance benchmarks

#93

Earlier quoted context omitted.

I'm not sure why it would need to be "disclosed", other than to suggest the author knows what they're talking about due to "domain knowledge".

Maybe because I don’t trust someone who allegedly writes databases but is proud about not knowing python.

Author here: I did not write that I was proud of not knowing Python. I just wrote that I don't know Python. The thought of trying to understand 2k lines of it looking to see where Memgraph 'cheated' to make their product look good and the other bad was beyond my current capabilities.

Re: Bullshit graph database performance benchmarks

#94

What are people using graph databases for, and what do your queries look like? I've read about them briefly but I have to admit my imagination fails me as to how it would look in the real world.

I'll give you an example of a graph database use case.

The police have a ton of data lying around, and the consensus in the industry is that the 80/20 rule applies to criminals as well ie: 20% of the population takes up 80% of the police resources. You could probably also posit that 20% of that 20% are "peak criminals."

Anyway, they would like to track interactions of "things."

Say a car is involved in an incident. They normally track the make, model, plate, and color of the car - on paper. There's a lot of other info they they track: who owns that car? Who's in the car? Where is the car? Where does the owner live? Where do the occupants live? What other incidents has that car been involved in? Given the addresses of the people involved, who else is known to be around them?

All this relationship information can give someone a better understanding of the relationship between criminal elements in an area. If a car is being used in lots of crimes, it's easier to find out using a DB than some cop going "I recognize that car." If lots of people are being picked up and all live in a 2 block area, it'll be easier to see that if it's in a DB than a cop recognizing that fact from multiple incident reports.

I actually tried doing this in SQL, and it's super slow because you have to iterate over your tables over and over. With the graph database this becomes, well, substantially easier if you model it correctly.

This product, BTW, is known as CopLink by IBM.

As an aside, fusion centers have this problem too but worse, because they're supposed to coordinate information between different police departments in a region...all of whom don't particularly give a shit.

Re: Bullshit graph database performance benchmarks

#95
The real problem with these kinds of "benchmarks" is that either the company doesn't have anyone on staff that's calling "bullshit" on it or the marketing people don't care that it's bullshit.

Either one is a bad sign if they're going to be a vendor. At that point how can you trust their SLAs and/or their presales team?

Re: Bullshit graph database performance benchmarks

#96

What are people using graph databases for, and what do your queries look like? I've read about them briefly but I have to admit my imagination fails me as to how it would look in the real world.

I’ve blogged about a couple of examples:

Understanding the spider web that is AWS IAM permissions: https://eng.lyft.com/iam-whatever-you-say-iam-febce59d1e3b,

Calculating whether a vuln was introduced from a parent image or from the service itself in a microservice arch: https://eng.lyft.com/vulnerability-management-at-lyft-enforc...

Re: Bullshit graph database performance benchmarks

#98

The author works on RageDB ( https://ragedb.com/ ) and this doesn't seem to be disclosed in the article

The author is discussing facts and providing replicable results. Disclosing their "conflict of interest" more clearly would be laudable, but even if they lied to us and pretended to be an independent journalist, that might sway our opinion of their character, but it would have no effect on the veracity of their writing.

What they say can be classified in three categories: Objectively right, objectively wrong, or subjective claims. Their conflict of interest only affects our evaluation of subjective claims.

Things that can be assessed as right are right even if they were said by Vladimir Putin; things that can be assessed as wrong are wrong even if they were said by Florence Nightingale. It is an ad hominem appeal to motive to suggest otherwise.

https://en.wikipedia.org/wiki/Appeal_to_motive

Re: Bullshit graph database performance benchmarks

#99

While this seems to be a pretty egregious example of a vendor benchmark misleading through cherry-picked unrealistic results, I'm not sure I share the author's pessimism about how these kinds of stunts will hold back the graph database market. Why? Simple: pretty much any benchmark I've seen of anything, ever, was similar nonsense -- give people numbers to game and they'll do so, enthusiastically. Even supposedly gol…

> Even supposedly gold-standard benchmarks like the TechEmpower framework benchmarks quickly devolve into "application server handling HTTP requests by responding with predefined strings", which is as fast as it's utterly useless in most people's version of the real world.

It sets an upper bound on a server's performance given that page generation completes instantly. Sure it won't reflect real world performance, but in this case the benchmark should be read as "higher requests per second = lower resource footprint for the server".

Engineering is about being able to understand what a benchmark or measure truly means, and what useable information it contains.

Re: Bullshit graph database performance benchmarks

#100
post #33
post #29

Earlier quoted context omitted.

It's because he has reason to destroy other competitors. Nice read otherwise.

He "destroys" one competitor as much as he praises another. He's never comparing either with ragedb.

Like another poster commented -- he has shares in that company that he praises.
Post reply on HN