Writing a very fast cache service with millions of entries in Go

I had the same feeling when reading this article. That's nothing like "high performance", especially the 5ms median latency. memcached is a whole two orders of magnitude lower in terms of median latency.

Can you expand? Not knowing anything about Go, what's an 'in-process solution'?

One of the points of Go seems to have been the idea of making a Go process a lot like a mini-Unix, with many small programs plumbed together and communicating through pipes, only strong-typed ones (and often by passing ownership of data using pointers, so that you don't have to actually copy stuff). Which uses zero HTTP and JSON traffic, which is faster than even the fastest HTTP and JSON library could ever be.

The reason why Go doesn't (currently?) have a super-performant HTTP server and a super-performant JSON library is that the designers probably didn't envision someone using the language like this.

Redis eats million of entries for breakfast, is pleasant to work with, has TTL expiration of keys built in and is available as a managed AWS ElastiCache service when you get into serious data sizes: up to 32 core 237 GiB nodes, and then you shard to add more. Redis is also super as a local cache, and simple to deploy and manage together with the app that uses it.

Since you obviously ran some quick benchmarks and concluded that running it locally over a unix socket (confused why you would mention "time needed on the network"... you tested with local sockets, right?) was too slow, you should at least let Antirez know you've run into a new mysterious performance bug ;) Writing a cache service can be a fun side project, but I doubt you gained anything by doing so except another homegrown part to maintain.

Golang's HTTP server is pretty performant: https://www.reddit.com/r/golang/comments/28so0e/go_networkin...

In this case, while that is a microbenchmark, it's a relevant one; both things are basically measuring "what's the minimal cost for a web request"? I could still quibble around the question of routing, but generally "within 2x of nginx" is good enough for most uses, and generally, for any non-trivial use of either nginx or Go's web server, you're going to dominate the cost of the HTTP request with your processing. (Considering how nasty the inside of nginx is and how nice the Go HTTP server looks, that's actually surprisingly good performance. And I don't mean that "nasty" as a criticism; it is what it is for good reason.)

(That said, if my back was against the wall performance-wise, I'd seriously consider looking at my incoming requests, seeing if there's a strong enough pattern in what's going on, and writing myself a psuedo-HTTP server that isn't actually an HTTP server, but just looks like one, skipping as much parsing as I can on the way in, and emitting a lot more hard-coded stuff as headers on the way out. I've never had to do this yet, but it's an option I'm keeping in my pocket.)

As for JSON, well, people generally conflate "parsing" and "marshalling" with JSON. JSON parsing is so drop-dead easy that one skilled in the art can write a decent parser in just a day or two; it's a good format that way. However, the task of converting a parsed JSON representation into local data types is actually surprisingly subtle, and any mature language will almost certainly have at least two if not more JSON marshallers that work in fundamentally different ways.

There will generally be at least one built for raw parsing speed, but will always hand you back a very generic data structure that has none of your application-specific types in it. There will be one built for really, really convenient marshalling and unmarshalling of your application-specific types, but it'll probably be significantly slower, and make certain decisions that will mean it can't be used safely on arbitrary JSON... i.e., if there's something that may be a string but may be an object, this library will range from inconvenient to impossible to use. And there are other valid cost/benefit points; the JSON marshaller that loads the JSON into memory and deparses it with nearly-0 additional overhead by re-using the original byte buffer intelligently, the JSON marshaller that can build up specialized parsers with code generation at compile time even if you're in a dynamic language, the JSON parser that for better or worse permits a certain amount of sloppiness to deal with sloppy emitters, etc.

Go's default encoding/json is the one built for convenience of application-specific types that can't handle or emit arbitrary JSON easily. As a sideline it can also do the generic raw parsing if you pass it the correct type to start with, but I believe it's paying some overhead vs. something custom written for that. I'm pretty sure they know they made this choice; all that stuff I described was pretty clear by the time Go was being written, that it is basically impossible to write the JSON library, so you might as well choose which one you're looking to ship. I think for a standard library it was the right choice, because it's a solid middle-of-the-ground choice... most JSON can be marshalled by it, because most JSON is still well-enough behaved for that to work. It's mostly good enough for most uses. But if you need the ultimate speed, or the ultimate flexibility, or the ultimate anything-else, you'll need to pick something else.