Wow... some of these tests are still pretty severely hobbled.
Is there some reason that you use built in json serialization for some frameworks and not others?
There is also a lot of heterogeneity in the implementation of the multiple queries test. For instance, even if I only look at... say... java frameworks, you seem to implement the exact same feature in very different ways between platforms. For instance, for servlets, you will store all of the results in a simple array... and then write them out when you are done. Like so:
final World[] worlds = new World[count];
final Random random = ThreadLocalRandom.current();
try (Connection conn = source.getConnection())
{
try (PreparedStatement statement = conn.prepareStatement(DB_QUERY,
ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY))
{
// Run the query the number of times requested.
for (int i = 0; i
But for other frameworks, like Vert.x, you use CopyOnWriteArray to store all of the results... and then write them out when you are done. Like so:
private final HttpServerRequest req;
private final int queries;
private final List worlds = new CopyOnWriteArrayList();
.
.
.
@Override
public void handle(Message reply)
{
final JsonObject body = reply.body;
if ("ok".equals(body.getString("status")))
{
this.worlds.add(body.getObject("result"));
}
if (this.worlds.size() == this.queries)
{
// All queries have completed; send the response.
// final JsonArray arr = new JsonArray(worlds);
try
{
final String result = mapper.writeValueAsString(worlds);
final int contentLength = result.getBytes(StandardCharsets.UTF_8).length;
this.req.response.putHeader("Content-Type", "application/json; charset=UTF-8");
this.req.response.putHeader("Content-Length", contentLength);
this.req.response.write(result);
this.req.response.end();
}
catch (IOException e)
{
req.response.statusCode = 500;
req.response.end();
}
}
}
In other words, you literally create a new array each time you add a result to that CopyOnWriteArray. In fact, not only are you creating a new array, but you are creating new copies of the data in the array as well. Seems a little strange??? DEFINITELY inefficient. Is there a reason that is implemented differently? It seems to me that, at the least, they should both use arrays... but maybe there is something more you guys are testing???
The Onion C based code is written in an even MORE efficient manner for the multiple queries test. It actually stores it's results in json format from the outset! Like so:
json_object *json=json_object_new_object();
json_object *array=json_object_new_array();
int i;
for (i=0;i
The equivalent java code would be something like:
private final HttpServerRequest req;
private final int queries;
// INSTEAD OF:
//private final List worlds = new CopyOnWriteArrayList();
// HAVE:
private final JsonArray worlds = new JsonArray();
.
.
.
@Override
public void handle(Message reply)
{
final JsonObject body = reply.body;
if ("ok".equals(body.getString("status")))
{
// INSTEAD OF:
//this.worlds.add(body.getObject("result"));
// HAVE:
this.worlds.addObject(body.getObject("result"));
}
if (this.worlds.size() == this.queries)
{
// All queries have completed; send the response.
// final JsonArray arr = new JsonArray(worlds);
try
{
// INSTEAD OF:
//final String result = mapper.writeValueAsString(worlds);
// HAVE:
final String result = worlds.encode();
final int contentLength = result.getBytes(StandardCharsets.UTF_8).length;
this.req.response.putHeader("Content-Type", "application/json; charset=UTF-8");
this.req.response.putHeader("Content-Length", contentLength);
this.req.response.write(result);
this.req.response.end();
}
catch (IOException e)
{
req.response.statusCode = 500;
req.response.end();
}
}
}
With a similar change for Servlets. According to the benchmark results, Onion comes out on top. It's the fastest. But how much of that's because it seems to be written correctly and other tests seem to be written without taking advantage of the same efficiencies.
Is it the case here that some people have sent you test code optimized for their own frameworks?
If that is so, you should add some tests that would not be so amenable to optimization. I'm not picking on Onion here by the way. In fact, the argument could be made that Onion is not actually 'optimized', so much as just written correctly, and the other frameworks have tests written incorrectly. But I just wanted to know if you guys actually intended to use these different implementations for some reason that I am unaware of? Do they make the tests more fair somehow???