11–20 of 58 posts
benchmark = marketing (more times than not)
Everyone likes to see how their tool of choice performs against other tools. And it's an important (albeit just one part) of deciding which product to use.
The risk of a colored benchmark is quite high when benchmark is done by owner of product or by "fan" of product. With the exception of a well explained, clear benchmark that everyone can understand and reproduce easily.
Benchmarking is hard and a lot of reports are bogus. However they are still very useful for a lot of developers. Benchmarking programming languages got better. E.g.: http://benchmarksgame.alioth.debian.org/ gives roughly idea about performance of programing languages. I wish something similar existed for databases. I think exact figures would be hard to get, but I believe there are many 2x 10x differences that we sho…
Benchmarking is hard and a lot of reports are bogus. However they are still very useful for a lot of developers. Benchmarking programming languages got better. E.g.: http://benchmarksgame.alioth.debian.org/ gives roughly idea about performance of programing languages. I wish something similar existed for databases. I think exact figures would be hard to get, but I believe there are many 2x 10x differences that we sho…
I wish we got this kind of benchmark for the altjs solutions.
Earlier quoted context omitted.
Everyone likes to see how their tool of choice performs against other tools. And it's an important (albeit just one part) of deciding which product to use.
I always say do your own benchmarking for your own use case. The risk of a colored benchmark is quite high when benchmark is done by owner of product or by "fan" of product. With the exception of a well explained, clear benchmark that everyone can understand and reproduce easily.
Perhaps, once you've winnowed the choices down to just a few, it might make more sense, but I think good public benchmarks can be a helpful thing for that selection process.
> In RethinDB, you have to create databases and tables manually and it will raise an exception if they already exist. Compared to MongoDB that could be an inconvenience for some(and me) - one of the things I find appealing in MongoDB is the fluid interaction with databases
... well at least now I don't feel so bad about having some old MySQL stuff still in production. MySQL already has too much "fluidity" in dealing with my data...
Earlier quoted context omitted.
I always say do your own benchmarking for your own use case. The risk of a colored benchmark is quite high when benchmark is done by owner of product or by "fan" of product. With the exception of a well explained, clear benchmark that everyone can understand and reproduce easily.
At least with a public benchmark, people can point out flaws. With something rolled out internally, you're still likely to get flaws, but no one will point out that you misconfigured Postgres or set up Mongo the wrong way, or any of the other errors you are just as likely to make by doing it yourself. Perhaps, once you've winnowed the choices down to just a few, it might make more sense, but I think good public bench…
An internal benchmark indeed requires knowledge on the subject, but most of the times people at the mailinglists are quite helpful when you explain what you are trying to do. Especially when you post a benchmark which is not in favorite of their product.
Benchmarking is hard and a lot of reports are bogus. However they are still very useful for a lot of developers. Benchmarking programming languages got better. E.g.: http://benchmarksgame.alioth.debian.org/ gives roughly idea about performance of programing languages. I wish something similar existed for databases. I think exact figures would be hard to get, but I believe there are many 2x 10x differences that we sho…
In this case we were presented with a benchmark setup that failed to perform the task it supposedly benchmarked. That's not hard to avoid, and it makes the benchmark completely useless and misleading.
I don't think that this problem can be generalized in the sense you do here, since the problem with benchmarks usually isn't a complete failure to perform the task to be benchmarked, but things like finding a set of tests that give a fair representation of what you'd typically use the subjects for, or performing the tasks in idiomatic and optimal ways.
Benchmarking is hard and a lot of reports are bogus. However they are still very useful for a lot of developers. Benchmarking programming languages got better. E.g.: http://benchmarksgame.alioth.debian.org/ gives roughly idea about performance of programing languages. I wish something similar existed for databases. I think exact figures would be hard to get, but I believe there are many 2x 10x differences that we sho…
They're neat and all and it is called the benchmarks game but I wish they'd remove the ones that end up getting gamed like that.