Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeWitt Clause, or can you benchmark %database% and get away with it

cube.dev

Post reply on HN
Print view View original

61–70 of 93 posts

Re: DeWitt Clause, or can you benchmark %database% and get away with it

#61

Post by ChikkaChiChi »

jandrewrogers wrote: post #48
kstrauser wrote: post #27

Honestly, the presence of that clause screams to me "this app sucks and we'll sue you if you tell anyone how badly". That may not be the case whatsoever, but my first assumption is that they're trying to hide terrible performance.

While I would prefer if this clause was not a thing, I also understand why it exists even for great products. It is surprisingly difficult to reproduce many workload benchmarks and quite easy to engineer a benchmark that misrepresents real-world database performance. There are tools that exist to generate optimally pathological workloads that target specific database implementations, while looking completely reasonab…

Reproducible benchmarking can also inform the person/team that their configurations are suboptimal. How are you supposed to know how good your "best" is if you aren't allowed to talk about shakedowns?

Re: DeWitt Clause, or can you benchmark %database% and get away with it

#62

Post by a1369209993 »

igorlukanin wrote: post #19
jenny91 wrote: post #11

All the BSL/SSPL ones shouldn't be in an "open source" section. Just change the heading to "source available" or put them with the "vendors".

Author of the blog post in question here. Let me clarify: they shouldn't be there because they're not OSI-approved, right? Just wanna get your point here. (While I understand that BSL/SSPL lack certain liberties, I deemed it okay to mark them as "open source" for the purposes of this post.)

> they shouldn't be there because they're not OSI-approved, right

No, they shouldn't be there because they aren't open source; OSI approval has nothing to do with it. (Though it's still a good indicator, similar to how FDA approval doesn't change whether/how well a drug works/is safe.)

Re: DeWitt Clause, or can you benchmark %database% and get away with it

#63

Post by jrockway »

fragmede wrote: post #53
chmod775 wrote: post #8

It would be quite refreshing if we could have a story in which Oracle are the good guys for once. I'm sure they are at least purchasing some modern-day 'indulgences' by - for instance - donating food to starving north korean elites?

Oracle Virtualbox is pretty good, and free for personal use. They also make patches available for their Oracle Linux kernel - the UEK, and in a better format than RedHat.

They are aggressive about the "personal use" thing. I used to work for an ISP. Apparently our customers would download Virtualbox, Oracle would pull the IPs out of their logs, find us, and then email us and ask us to buy a license. (More threatening than asking nicely, IIRC.) We informed them that we're an ISP and those IP addresses are our customers', not our office. And no, we didn't give them the contact information of our customers.

I wonder how many of their sales reps have gone after Spectrum and Comcast since the whole work from home thing started.

Re: DeWitt Clause, or can you benchmark %database% and get away with it

#64

Post by lostcolony »

Beltalowda wrote: post #41
danielheath wrote: post #38

Earlier quoted context omitted.

For a smaller/younger SAAS: If a customer environment is suddenly running at 100% of some resource when it wasn’t before, that’s an important thing to alert on / investigate. For established players it’s lost in the noise, but if it were me I’d appreciate a heads up for big changes.

Sure, a heads-up is certainly nice, but I don't think that running a (reasonable) set of benchmarks is all that out of the ordinary, or any different from just taxing the service at 100% with some periodic batch job or the like. Paying for it is even stranger IMO. And for what it's worth, I did actually work for a few small SaaS businesses, but a few reasonable benchmarks wouldn't have been a problem. Of course, if y…

An occasional spike probably won't even be noticed. Redlining it when you usually hover much lower, without ceasing, -should- trigger alarms, and get engineers going "WTF is going on?!"

Give the vendor a heads up so the engineers can sleep.

Re: DeWitt Clause, or can you benchmark %database% and get away with it

#65

Post by geocar »

Nextgrid wrote: post #7

If you want to benchmark for internal reasons you don't publish the results and nobody knows. If you want to make a service to the community, run your benchmarks, download Tor and publish the results anonymously. I don't see what the big deal is? Is this only limited to marketing claims where you post it on your company's website?

> I don't see what the big deal is?

The big deal is that it's slanderous.

Oracle isn't making an attestation about performance that the benchmark addresses, the benchmark aims to make statements about Oracle for some other reason, and that's important: Rights to free-speech generally end when they cause harm, and bearing full costs in the defense of false benchmarks is certainly harm.

Look at it this way: The clause aims to prevent purchases (entering into agreement with Oracle) under false pretenses. Oracle sells software to solve business problems, not you-need-a-paper-that-makes-Oracle-look-bad problems, and I think vendors are wise to protect themselves from that.

On the other hand, if you actually bought Oracle to solve a problem, and it didn't do that, you're still free to make those benchmarks and sue the shit out of Oracle with them, and this agreement can't by itself prevent the benchmarks from reaching the public record at that point.

> Is this only limited to marketing claims where you post it on your company's website?

If your company makes X and your company website contains a benchmark saying Oracle is slower than X, you're not just making a statement that you observed Oracle was slower than X, you're also making an attestation that the benchmark is a fair representation of both Oracle and X. And judge and jury are going to be wondering if it's as fair as you say, or if it's unfair as Oracle says.

Now, if you're a university and you don't make X, you might be able to argue that even if it's unfair, it was done in good-faith, and judge and jury may believe that, but Oracle will ask, if you truly believed X was fair, why didn't you get our feedback before publishing? and you'd better have a good answer to that.

On the other hand, if you choose to be anonymous, you may be able to avoid the judge and jury, but the community has to wonder who you are, whether you are motivated by a relationship to a company or product that competes with Oracle, or an impatient researcher who can't meet the standard of professional publishing. The community will wonder, but they have lots of other things to wonder about too, so they probably will not wonder for very long. So what's the point? Techdudes already know what they think of Oracle, and nobody who writes code talking or Oracle thinks that Oracle was chosen for its benchmarks, so who is this anonymous benchmark for?

Re: DeWitt Clause, or can you benchmark %database% and get away with it

#66

Post by eastbound »

rckoepke wrote: post #4
igorlukanin wrote: post #3

Earlier quoted context omitted.

I think someone from Oracle would be more informed on that matter. JK. On a more serious note, who would dare to displease a multibillion corp with hundreds of lawyers (without being backed by a similar co & lawyers)?

Could you pirate the database, then hide behind the fifth amendment to not reveal that you're a pirate while simultaneously asserting that you never agreed to any EULA? I'm not sure what the legal rights are here. I'm certain someone in say, China or Russia, could pirate the database and run benchmarks on it with no repercussions. Surprising that this isn't a business model for an overseas technology analyst firm.

The Fifth only protects the innocents. It’s a fun twist of this amendment - if you are guilty you do not have a right to keep silent.

Re: DeWitt Clause, or can you benchmark %database% and get away with it

#67

Post by eastbound »

_lqaf wrote: post #14
igorlukanin wrote: post #3

Earlier quoted context omitted.

I think someone from Oracle would be more informed on that matter. JK. On a more serious note, who would dare to displease a multibillion corp with hundreds of lawyers (without being backed by a similar co & lawyers)?

These things become substantially easier when approached correctly. In this case, never run Oracle software. Not only will it vastly improve your mood during budget season; your developers will be less likely to stab you in your sleep and you will never worry about their primary line of business: lawsuits. And you don't care how they benchmark.

Here the Jal clause is born: You are now allowed to speak about what Oracle’s main line of business is.

Re: DeWitt Clause, or can you benchmark %database% and get away with it

#68

Post by car_analogy »

floren wrote: post #43

You can add Splunk to the list of companies with a similar clause. As a Splunk competitor it makes sales a bit harder initially (we can show our product's numbers, but nothing to compare them against), but if you can convince customers to set up a head-to-head proof-of-concept of their own , well, they tend to figure out why Splunk doesn't want you publishing benchmarks...

How do potential customers react to "These are our numbers. We would compare them with Splunk's, but their license forbids publishing benchmark results"?

Re: DeWitt Clause, or can you benchmark %database% and get away with it

#69

Post by jkaplowitz »

eastbound wrote: post #66
rckoepke wrote: post #4

Earlier quoted context omitted.

Could you pirate the database, then hide behind the fifth amendment to not reveal that you're a pirate while simultaneously asserting that you never agreed to any EULA? I'm not sure what the legal rights are here. I'm certain someone in say, China or Russia, could pirate the database and run benchmarks on it with no repercussions. Surprising that this isn't a business model for an overseas technology analyst firm.

The Fifth only protects the innocents. It’s a fun twist of this amendment - if you are guilty you do not have a right to keep silent.

That's true if you've been convicted and sentenced for the crime regarding which your testimony would self-incriminate, but not otherwise. Someone who has committed the crime but hasn't yet been convicted and sentenced still falls under its protection, assuming there isn't a grant of immunity from prosecution to force the testimony anyway.
Post reply on HN