Writing a very fast cache service with millions of entries in Go • Hacker News Board

Re: Writing a very fast cache service with millions of entries in Go

#21

Post by bechampion » Wed, Mar 30, 2016, 9:57 AM UTC

matthewmacleod wrote: post #13

I wonder if I'm missing the point here, but why are Redis or Memcached—implementations of exactly this service which are battle-tested and well-used—not suitable due to "additional time needed on the network", but this service is suitable? Is it just down to the requirement for a HTTP API? One thing I've noticed is an extreme demand for making internal services available over HTTP. It has it's benefits, but the obvio…

I was wondering about the same

Re: Writing a very fast cache service with millions of entries in Go

#22

Post by jzwinck » Wed, Mar 30, 2016, 10:30 AM UTC

matthewmacleod wrote: post #13

I wonder if I'm missing the point here, but why are Redis or Memcached—implementations of exactly this service which are battle-tested and well-used—not suitable due to "additional time needed on the network", but this service is suitable? Is it just down to the requirement for a HTTP API? One thing I've noticed is an extreme demand for making internal services available over HTTP. It has it's benefits, but the obvio…

I had to read the relevant sections three times to sort this out. I believe what they are saying is that this service they made had to speak HTTP in some specific way, and since Redis doesn't speak that way directly they would have needed to proxy requests in their service to Redis which would mean one network hop to reach their HTTP service plus one hop to reach Redis.

Of course, Redis and Memcached support Unix domain sockets which do not use the network and do not suffer from the overhead of TCP. The authors do not address this at all, suggesting they weren't aware of UDS, nor the fact that TCP within one Linux host does not touch the network at all.

Adding to the confusion is that even given an in-memory cache library which overcame their objections to the "network" based ones, they still elected to write their own low-level cache.

So the comment about time needed on the network was either spurious or misinformed. And one thing it was not: measured.

Re: Writing a very fast cache service with millions of entries in Go

#23

Post by elcct » Wed, Mar 30, 2016, 10:41 AM UTC

very fast. much entries. so cache. such go. wow

Re: Writing a very fast cache service with millions of entries in Go

#24

Post by st0p » Wed, Mar 30, 2016, 10:49 AM UTC

Why not use Varnish? POST messages of 500 bytes could easily be rewritten / proxy'd to GET requests. That might not be 100% restfull but seems like a lot less work. On our production environment Varnish always responds in less then 2 ms. Even on my development VM I never see response times > 5 ms. It has all the other requirements they state.

Perhaps I'm prejudiced because Varnish has proven to be such an awesome caching mechanism (we still use Redis as a key/value store), but this seems like NIH.

Re: Writing a very fast cache service with millions of entries in Go

#25

Post by iveqy » Wed, Mar 30, 2016, 10:56 AM UTC

smt88 wrote: post #12

roel_v wrote: post #6

So essentially, to meet their requirements, they had to work around the Go garbage collector and use a non-standard HTTP server and JSON parser. Why not just write it in C++?

> Why not just write it in C++? No need. Someone already wrote Redis.

Redis is written in C, afaik.

Re: Writing a very fast cache service with millions of entries in Go

#26

Post by sulam » Wed, Mar 30, 2016, 10:56 AM UTC

Sorry, I stopped reading at 10K rps and p50 of 5ms. In this day and age these numbers are pretty bad, especially for a cache where presumably all accesses are constant time. Every single caching solution listed out-performs this handily.

Re: Writing a very fast cache service with millions of entries in Go

#27

Post by bluetech » Wed, Mar 30, 2016, 11:05 AM UTC

jzwinck wrote: post #22

matthewmacleod wrote: post #13

I wonder if I'm missing the point here, but why are Redis or Memcached—implementations of exactly this service which are battle-tested and well-used—not suitable due to "additional time needed on the network", but this service is suitable? Is it just down to the requirement for a HTTP API? One thing I've noticed is an extreme demand for making internal services available over HTTP. It has it's benefits, but the obvio…

I had to read the relevant sections three times to sort this out. I believe what they are saying is that this service they made had to speak HTTP in some specific way, and since Redis doesn't speak that way directly they would have needed to proxy requests in their service to Redis which would mean one network hop to reach their HTTP service plus one hop to reach Redis. Of course, Redis and Memcached support Unix dom…

Quoting redis docs (http://redis.io/topics/benchmarks):

> When the server and client benchmark programs run on the same box, both the TCP/IP loopback and unix domain sockets can be used. Depending on the platform, unix domain sockets can achieve around 50% more throughput than the TCP/IP loopback (on Linux for instance). The default behavior of redis-benchmark is to use the TCP/IP loopback.

Haven't measured myself though.

Re: Writing a very fast cache service with millions of entries in Go

#28

Post by Maascamp » Wed, Mar 30, 2016, 11:05 AM UTC

pcwalton wrote: post #3

From the article: > [Go] also has managed memory, so it looks safer and easier to use than C/C++. But most of the post describes a sophisticated way to work around the garbage collector, totally reliant on a specific implementation detail of the current Go GC (skipping of pointer-free data types), documented in a GitHub issue. It seems easier to not have, or to not use, the GC in the first place for this specific pro…

I confronted a similar caching requirement (in my case the cache needs to be much larger) in Java recently and chose to implement off-heap for some of the reasons you mention. It avoids GC and heap size concerns entirely and makes it easy to tune the rest of the application's GC profile. Systems handles 45k writes/sec and about double that for reads with very low latency minimal CPU. Implementing concurrent writes/eviction without typical Java concurrency controls was a bit tricky though.

Project is here for those interested: https://github.com/Maascamp/fohlc

Re: Writing a very fast cache service with millions of entries in Go

#29

Post by NicoJuicy » Wed, Mar 30, 2016, 11:11 AM UTC

I found the mention of ffjson interesting, a faster serializer then the standard buildin json serializer ( 2x - 3x as fast) ==> https://github.com/pquerna/ffjson

Re: Writing a very fast cache service with millions of entries in Go

#30

Post by mhd » Wed, Mar 30, 2016, 11:12 AM UTC

vog wrote: post #8

pcwalton wrote: post #3

From the article: > [Go] also has managed memory, so it looks safer and easier to use than C/C++. But most of the post describes a sophisticated way to work around the garbage collector, totally reliant on a specific implementation detail of the current Go GC (skipping of pointer-free data types), documented in a GitHub issue. It seems easier to not have, or to not use, the GC in the first place for this specific pro…

Maybe this type of program is better suited for a language like Rust. However, while not having a GC, you have to take care of all memory issues in a way that you convince the compiler that your won't ever blow up. It would be very interesting to have such a comparison, so we could see whether it's easier to work around the GC, or easier to write bullet-proof code with manual memory management. I'd expect the Rust to…

I'm not sure how the current implementation of Go handles it, but its spiritual relatives Modula-3/Oberon handled this quite well, with a GC for most occasions and ways to bypass this with "unsafe" modules that allowed for untracked allocations/deallocations and pointer arithmetic. It's not really an either/or situation by (language) definition...