Writing a very fast cache service with millions of entries in Go
1–10 of 92 posts
Re: Writing a very fast cache service with millions of entries in Go
#2I wonder if your team considered leveraging Nginx with some good caching policy, for most use cases it should already have everything you need and it'll probably be tough to beat Nginx's performance.
Re: Writing a very fast cache service with millions of entries in Go
#3From the article:
> [Go] also has managed memory, so it looks safer and easier to use than C/C++.
But most of the post describes a sophisticated way to work around the garbage collector, totally reliant on a specific implementation detail of the current Go GC (skipping of pointer-free data types), documented in a GitHub issue. It seems easier to not have, or to not use, the GC in the first place for this specific project if most of the engineering effort is going to go into an elaborate workaround for it. In particular, I don't understand the reason for rejecting offheap: the article suggests "a cache which relied on those functions would need to be implemented", but surely this is less complex to implement than the cache that relies on hiding what are effectively pointers behind offsets so the GC won't think to scan them.
(This isn't a suggestion to not use Golang at all, to be clear. Nor do I mean to suggest that GCs are bad things in general.)
Re: Writing a very fast cache service with millions of entries in Go
#4[deleted]
Re: Writing a very fast cache service with millions of entries in Go
#5Manual memory management isn't an esoteric technology. If that's what's required to reduce GC overhead, it's an acceptable choice, especially when it's in a fairly simple system like a cache (a hash table with an eviction policy).
Re: Writing a very fast cache service with millions of entries in Go
#6So essentially, to meet their requirements, they had to work around the Go garbage collector and use a non-standard HTTP server and JSON parser. Why not just write it in C++?
Re: Writing a very fast cache service with millions of entries in Go
#7I admit I only skimmed the article, but I think the tradeoff of parting with the requirement of having an HTTP API with JSON as the message format for transporting IDs over the wire, and just using Redis instead and its wire protocol, would probably have been the more time-efficient way to arrive at an (at least) acceptable solution for the problem they were looking to solve. But yeah, where's the NIH in that? ;)
Re: Writing a very fast cache service with millions of entries in Go
#8From the article: > [Go] also has managed memory, so it looks safer and easier to use than C/C++. But most of the post describes a sophisticated way to work around the garbage collector, totally reliant on a specific implementation detail of the current Go GC (skipping of pointer-free data types), documented in a GitHub issue. It seems easier to not have, or to not use, the GC in the first place for this specific pro…
Maybe this type of program is better suited for a language like Rust.
However, while not having a GC, you have to take care of all memory issues in a way that you convince the compiler that your won't ever blow up.
It would be very interesting to have such a comparison, so we could see whether it's easier to work around the GC, or easier to write bullet-proof code with manual memory management.
I'd expect the Rust to be more robust with regard to performance, but the question is: Would it also pay off in term of code complexity?
Re: Writing a very fast cache service with millions of entries in Go
#9> Considering the first point we decided to give up external caches like Redis, Memcached or Couchbase mainly because of additional time needed on the network.
Uh... we regularly get performance better than what the OP has described in their "needs" with Redis. It can run in memory - on the same machine and has plenty of HTTP frontends you can work on (not a lot of networking).
Re: Writing a very fast cache service with millions of entries in Go
#10From the article: > [Go] also has managed memory, so it looks safer and easier to use than C/C++. But most of the post describes a sophisticated way to work around the garbage collector, totally reliant on a specific implementation detail of the current Go GC (skipping of pointer-free data types), documented in a GitHub issue. It seems easier to not have, or to not use, the GC in the first place for this specific pro…
Maybe this type of program is better suited for a language like Rust. However, while not having a GC, you have to take care of all memory issues in a way that you convince the compiler that your won't ever blow up. It would be very interesting to have such a comparison, so we could see whether it's easier to work around the GC, or easier to write bullet-proof code with manual memory management. I'd expect the Rust to…
I don't want to argue specifically that Rust is the solution here, because I think using Go with offheap is a perfectly fine solution.
But I do think that the code complexity for a trivial hash table with striped locking can't possibly be higher than the complexity of serializing everything into a giant byte array and maintaining offsets into that array into a map which you make sure doesn't have any pointers in it.