Writing a very fast cache service with millions of entries in Go • Hacker News Board

Re: Writing a very fast cache service with millions of entries in Go

#31

Post by nl » Wed, Mar 30, 2016, 11:34 AM UTC

mixedbit wrote: post #16

Artemis2 wrote: post #14

Why did they have to invent "a very fast cache service with millions of entries"? Are they the first company to ever need one so they had to write it? Can't Redis or Memcached be fast (despite the "additional time needed on the network" — even though Redis uses raw TCP for transactions, while this uses HTTP + JSON)? As others pointed out, Go is also a very poor choice if you need to work around every part of the lang…

Probably because they wanted full control and understanding of the source code. Large companies often prefer to develop things in-house, even if there already exist good alternatives.

Large companies often prefer to develop things in-house, even if there already exist good alternatives.

Fair point. Except they are a small consulting company.

Re: Writing a very fast cache service with millions of entries in Go

#32

Post by potatosareok » Wed, Mar 30, 2016, 11:49 AM UTC

Maascamp wrote: post #28

pcwalton wrote: post #3

From the article: > [Go] also has managed memory, so it looks safer and easier to use than C/C++. But most of the post describes a sophisticated way to work around the garbage collector, totally reliant on a specific implementation detail of the current Go GC (skipping of pointer-free data types), documented in a GitHub issue. It seems easier to not have, or to not use, the GC in the first place for this specific pro…

I confronted a similar caching requirement (in my case the cache needs to be much larger) in Java recently and chose to implement off-heap for some of the reasons you mention. It avoids GC and heap size concerns entirely and makes it easy to tune the rest of the application's GC profile. Systems handles 45k writes/sec and about double that for reads with very low latency minimal CPU. Implementing concurrent writes/ev…

Wondering what your use case is for such a large cache that is not clustered (unless I misunderstood the code). Both in terms of single process dying = lost 10gb of data and also what single process needs a 10gb cache. I do see ability to read from disk listed.

How would you compare your cache to open source data grids that also provide off heap (infinispan, geode/gemfire, ignite) or just other general cache solutions (redis/memcache)?

Re: Writing a very fast cache service with millions of entries in Go

#33

Post by osweiller » Wed, Mar 30, 2016, 11:55 AM UTC

As a user and advocate of Go (in my case primarily as glue code that is beneficial because it's easy and efficient to get to results), articles like this do the platform a disservice.

This implementation is far from fast (two magnitudes better performance and it would be credible as "very fast"), and it is non-idiomatic, specifically doing things to avoid the benefits of Go.

As an aside -- HTTP and serialization are both costly. In many, many cases where I've seen them in effect, they were a significant expense for little to no architectural gain.

Re: Writing a very fast cache service with millions of entries in Go

#34

Post by serafin_allegro » Wed, Mar 30, 2016, 11:58 AM UTC

nl wrote: post #31

mixedbit wrote: post #16

Earlier quoted context omitted.

Probably because they wanted full control and understanding of the source code. Large companies often prefer to develop things in-house, even if there already exist good alternatives.

Large companies often prefer to develop things in-house, even if there already exist good alternatives. Fair point. Except they are a small consulting company.

Nope. We are not :) We are e-commerce platform - part of Naspers Group: http://www.naspers.com/page.html?pageID=3

Re: Writing a very fast cache service with millions of entries in Go

#35

Post by nly » Wed, Mar 30, 2016, 12:01 PM UTC

osweiller wrote: post #33

As a user and advocate of Go (in my case primarily as glue code that is beneficial because it's easy and efficient to get to results), articles like this do the platform a disservice. This implementation is far from fast (two magnitudes better performance and it would be credible as "very fast"), and it is non-idiomatic, specifically doing things to avoid the benefits of Go. As an aside -- HTTP and serialization are…

> HTTP and serialization are both costly.

But can be done very fast (introducing very little latency) and are essential pure operations, so can be parallelized very well (for throughput). Of course, this doesn't account for dev costs, and doesn't make your architectural point invalid.

Re: Writing a very fast cache service with millions of entries in Go

#36

Post by jakub_h » Wed, Mar 30, 2016, 12:30 PM UTC

Maascamp wrote: post #28

pcwalton wrote: post #3

From the article: > [Go] also has managed memory, so it looks safer and easier to use than C/C++. But most of the post describes a sophisticated way to work around the garbage collector, totally reliant on a specific implementation detail of the current Go GC (skipping of pointer-free data types), documented in a GitHub issue. It seems easier to not have, or to not use, the GC in the first place for this specific pro…

I confronted a similar caching requirement (in my case the cache needs to be much larger) in Java recently and chose to implement off-heap for some of the reasons you mention. It avoids GC and heap size concerns entirely and makes it easy to tune the rest of the application's GC profile. Systems handles 45k writes/sec and about double that for reads with very low latency minimal CPU. Implementing concurrent writes/ev…

Now that I think about these issues, I'm wondering now how performant AllegroCache is.

Re: Writing a very fast cache service with millions of entries in Go

#37

Post by tyingq » Wed, Mar 30, 2016, 12:33 PM UTC

I'm curious why they didn't use something like mmap. That would have skipped the off heap approach, and also allowed for management, statistics, etc, to run as a separate process.

Edit: Apparently the offheap package does use mmap if you pass a path to their Malloc.

Re: Writing a very fast cache service with millions of entries in Go

#38

Post by endymi0n » Wed, Mar 30, 2016, 12:36 PM UTC

I still don’t get why people try writing their own data store, especially in a language that's simply not very well suited to that task (and we're an almost 100% Golang shop here). Seems to be a rite of passage.

The requirements are literally Public service announcement: Don't write your own data store. Repeat after me: Don't write your own data store, except if you want to experimentally find out how to build data stores. It's arbitrarily hard and gets even more so at every layer. Plus, you leave behind an unmaintainable mess for the people after you. There's already a great OSS data store optimized for every use case and storage medium I could possible imagine.

Re: Writing a very fast cache service with millions of entries in Go

#39

Post by Maascamp » Wed, Mar 30, 2016, 12:38 PM UTC

potatosareok wrote: post #32

Maascamp wrote: post #28

Earlier quoted context omitted.

I confronted a similar caching requirement (in my case the cache needs to be much larger) in Java recently and chose to implement off-heap for some of the reasons you mention. It avoids GC and heap size concerns entirely and makes it easy to tune the rest of the application's GC profile. Systems handles 45k writes/sec and about double that for reads with very low latency minimal CPU. Implementing concurrent writes/ev…

Wondering what your use case is for such a large cache that is not clustered (unless I misunderstood the code). Both in terms of single process dying = lost 10gb of data and also what single process needs a 10gb cache. I do see ability to read from disk listed. How would you compare your cache to open source data grids that also provide off heap (infinispan, geode/gemfire, ignite) or just other general cache solution…

The use case in mainly deduplication of a high volume data stream (though it's got a few other uses). The write volume is fairly stable so it's sized in such away that we'll never emit dupes even when the upstream source crashes and needs to be rebuilt from backups (for this case that means > a billion cache entries). Something like the opposite of a bloom filter (https://www.somethingsimilar.com/2012/05/21/the-opposite-of-...) didn't work because we don't want false negatives either. Since the cache is fed by a Kafka log HA is achieved simply by having multiple consumers individually populating their own cache instance. The persistence mechanisms are to allow for code deployments that don't blow away the cache, not HA.

We actually experimented with grid caches (ignite in particular since it offers off-heap in memory storage as well), but the performance simply isn't there. At the volume we're writing even millisecond latency is a non-starter. We did explore both memcached and redis, but we need strict FIFO and both of those solutions provide nondeterministic LRU.

Re: Writing a very fast cache service with millions of entries in Go

#40

Post by jakub_h » Wed, Mar 30, 2016, 12:39 PM UTC

roel_v wrote: post #6

So essentially, to meet their requirements, they had to work around the Go garbage collector and use a non-standard HTTP server and JSON parser. Why not just write it in C++?

Correction: they thought they had to. I suspect that an in-process solution would avoid the HTTP and JSON issues, and better implementation of the store itself would avoid GC issues.