Comparing a non-blocking API to two blocking APIs. A more fair comparison would be using Twisted for Python and perhaps libev(ent) for C.
But it's a overly small test anyway.
In-memory key-value store in C, Go and Python
21–30 of 58 posts
Re: In-memory key-value store in C, Go and Python
#22In order to perform such a comparison you can't write three small trow-away programs, you need to optimize each version at your best for weeks to start to be meaningful, and you need an expert in the three systems. Otherwise the test is still interesting but is: "what is the best language to write a memcached clone without being an expert in a given language, using a few hours", that still says something about how di…
Yeah, a brief look at his buffering code raised alarms all over the place for me. The use of dynamic memory allocation; the use of strlen to find out if the current buffer is empty - seriously? memset()'ing the entire buffer instead of bothering to zero out just the by after the end of the recv(). That's ignoring various bugs and other stuff.
I haven't looked at the rest of the code, but if it's anything like the buffering code this is nothing like what you'd expect to see in a decent C implementation.
It's fairy enough to demonstrate that the Go version makes it easy to write decent performing code, but the C version is atrocious (though to his credit it does at least buffer - I've seen so much C networking code that murders performance by doing small read()'s that I want to cry, including for the longest time the MySQL client library).
Of course part of his criticism is also down to not bothering to look for the plethora of C networking libraries that does this and does it well.
Re: In-memory key-value store in C, Go and Python
#23I hand coded an epoll implementation for Python.
https://github.com/rahulkmr/Key-Value-Polyglot/blob/master/m...
As far as raw benchmark goes, it runs faster than the go version on my machine:
# Go version. Changed test.py to 10000 gets and sets.
± $ time python test.py
python test.py 0.48s user 0.60s system 47% cpu 2.289 total
# Python epoll version. Changed test.py to 10000 gets and sets.
± $ time python test.py
python test.py 0.20s user 0.26s system 50% cpu 0.903 total
Standard disclaimer: Please note that this comparison is highly unscientific, and take the numbers with a grain of salt.
Re: In-memory key-value store in C, Go and Python
#24I changed memg.py to use sockfile.write()/.flush() instead of socket.sendall().
This makes the memg.py server > x100 faster. It outperforms a gevent-based implementation by 10%.
See https://github.com/codeape2/Key-Value-Polyglot/commit/cbc53a...
EDIT: It does not outperform the gevent-based implementation. More performance testing indicates that gevent is around 2x faster. But it outperforms the original version by an order of magnitude.
Re: In-memory key-value store in C, Go and Python
#25I changed memg.py to use sockfile.write()/.flush() instead of socket.sendall(). This makes the memg.py server > x100 faster. It outperforms a gevent-based implementation by 10%. See https://github.com/codeape2/Key-Value-Polyglot/commit/cbc53a... EDIT: It does not outperform the gevent-based implementation. More performance testing indicates that gevent is around 2x faster. But it outperforms the original version by a…
sock.sendall() isn't the same as `sock.send(); sock.flush()`
sendall will look something like:
def sendall(sock, msg):
totalsent = 0
MSGLEN = len(msg)
while totalsent Re: In-memory key-value store in C, Go and Python
#26I hand coded an epoll implementation for Python. https://github.com/rahulkmr/Key-Value-Polyglot/blob/master/m... As far as raw benchmark goes, it runs faster than the go version on my machine: # Go version. Changed test.py to 10000 gets and sets. ± $ time python test.py python test.py 0.48s user 0.60s system 47% cpu 2.289 total # Python epoll version. Changed test.py to 10000 gets and sets. ± $ time python test.py py…
I wonder how a barebones Erlang-implementation will perform on this. It doesn't look too hard to write :)
Re: In-memory key-value store in C, Go and Python
#27This comparison is rather unfair on C, where you have chosen to use a low level interface, against Go, where you have chosen to use a high level interface. It is irrelevant that these are the default interfaces - high level interfaces for sockets exist in C. You could even integrate into Nginx.
This is typical in the Go community.
Usually the supposedly Go advantages are presented in a way, as if the same are not present in other languages.
Re: In-memory key-value store in C, Go and Python
#28I changed memg.py to use sockfile.write()/.flush() instead of socket.sendall(). This makes the memg.py server > x100 faster. It outperforms a gevent-based implementation by 10%. See https://github.com/codeape2/Key-Value-Polyglot/commit/cbc53a... EDIT: It does not outperform the gevent-based implementation. More performance testing indicates that gevent is around 2x faster. But it outperforms the original version by a…
sock.sendall() isn't the same as `sock.send(); sock.flush()` sendall will look something like: def sendall(sock, msg): totalsent = 0 MSGLEN = len(msg) while totalsent
I don't use socket.send/flush. I use sockfile.write/flush.
Re: In-memory key-value store in C, Go and Python
#29Uhm, where are the numbers?
Re: In-memory key-value store in C, Go and Python
#30This comparison is rather unfair on C, where you have chosen to use a low level interface, against Go, where you have chosen to use a high level interface. It is irrelevant that these are the default interfaces - high level interfaces for sockets exist in C. You could even integrate into Nginx.
This is typical in the Go community. Usually the supposedly Go advantages are presented in a way, as if the same are not present in other languages.
This is actually typical in most programming language communities that I've observed.