Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing a very fast cache service with millions of entries in Go

allegro.tech

Post reply on HN
Print view View original

61–70 of 92 posts

Re: Writing a very fast cache service with millions of entries in Go

#61

Post by atomic77 »

markbnj wrote: post #56
endymi0n wrote: post #38

I still don’t get why people try writing their own data store, especially in a language that's simply not very well suited to that task (and we're an almost 100% Golang shop here). Seems to be a rite of passage. The requirements are literally Public service announcement: Don't write your own data store. Repeat after me: Don't write your own data store, except if you want to experimentally find out how to build data s…

Sort of came here to say this. Interesting article, especially some of the design choices (such as http for the interface), but I am most curious as to why redis, or memcached, or some other k/v store was not good enough?

I wanted to like this article. But this:

> Considering the first point we decided to give up external caches like Redis, Memcached or Couchbase mainly because of additional time needed on the network.

Even mysql+innodb can easily handle 10k read/write queries against a simple pk table with "millions of entries" on desktop-grade hardware.

If the article was "here's a fun experiment to make a cache server in Go", fair enough.

Re: Writing a very fast cache service with millions of entries in Go

#62

Post by atomic77 »

chris_va wrote: post #59
im_down_w_otp wrote: post #55

Earlier quoted context omitted.

There are two datastores I would really like that don't exist. 1) An efficient, high-performance distributed persistence store for arbitrarily large CRDT's. 2) A Kafka-like highly-available distributed binary log w/ cheap topics, that doesn't require external coordination, and doesn't lose acknowledged writes (which I'll happily give up any shape of linearization guarantee for).

(1) Spanner does exist, though not really available outside of Google

CockroachDB is probably the closest system to spanner that I've come across (and built in Go)

https://github.com/cockroachdb/cockroach

Re: Writing a very fast cache service with millions of entries in Go

#63

Post by im_down_w_otp »

chris_va wrote: post #59
im_down_w_otp wrote: post #55

Earlier quoted context omitted.

There are two datastores I would really like that don't exist. 1) An efficient, high-performance distributed persistence store for arbitrarily large CRDT's. 2) A Kafka-like highly-available distributed binary log w/ cheap topics, that doesn't require external coordination, and doesn't lose acknowledged writes (which I'll happily give up any shape of linearization guarantee for).

(1) Spanner does exist, though not really available outside of Google

Yes, Spanner does exist. Spanner also does not satisfy what I said in #1.

Re: Writing a very fast cache service with millions of entries in Go

#64

Post by im_down_w_otp »

endymi0n wrote: post #57
im_down_w_otp wrote: post #55

Earlier quoted context omitted.

There are two datastores I would really like that don't exist. 1) An efficient, high-performance distributed persistence store for arbitrarily large CRDT's. 2) A Kafka-like highly-available distributed binary log w/ cheap topics, that doesn't require external coordination, and doesn't lose acknowledged writes (which I'll happily give up any shape of linearization guarantee for).

can't help with 1), but as for 2), maybe Kudu is an option for you? http://blog.rodeo.io/2016/01/24/kudu-as-a-more-flexible-kafk...

Uses Raft. Not highly-available enough unfortunately. Availability > Consistency for the use case that matters to me (basically a service message bus where all ACK'd writes will eventually be read by some consumer(s), but I don't care when or in what order, just that they all eventually get consumed, and where as long as any node in the cluster is up, it'll ACK a write).

Re: Writing a very fast cache service with millions of entries in Go

#66

Post by krenoten »

Rezo wrote: post #47

Redis eats million of entries for breakfast, is pleasant to work with, has TTL expiration of keys built in and is available as a managed AWS ElastiCache service when you get into serious data sizes: up to 32 core 237 GiB nodes, and then you shard to add more. Redis is also super as a local cache, and simple to deploy and manage together with the app that uses it. Since you obviously ran some quick benchmarks and conc…

redis is single threaded, so 32 cores doesn't mean much without sharding.

I generally prefer memcache unless you have a super locked-down infrastructure (no engineers to deploy a KEYS operation that destroys a shard and all the systems that rely on the data inside until it's finished). Multithreaded + simpler API is great for multitenancy when you have to provide infrastructure to engineers who don't want to learn about infrastructure.

Re: Writing a very fast cache service with millions of entries in Go

#67

Post by krenoten »

Maascamp wrote: post #51
endymi0n wrote: post #38

I still don’t get why people try writing their own data store, especially in a language that's simply not very well suited to that task (and we're an almost 100% Golang shop here). Seems to be a rite of passage. The requirements are literally Public service announcement: Don't write your own data store. Repeat after me: Don't write your own data store, except if you want to experimentally find out how to build data s…

Not sure if this applies to their use case since they mention FIFO in the context of it being a simple eviction policy, but if you _require_ FIFO semantics then both Redis and memcached are out of the question since they use nondeterministic LRU policies (memcached's LRU is particularly egregious in its nondeterminism due to the way slab allocation works).

this is rarely a concern for things that are ephemeral in nature to begin with. it's also a 20 line code change to add a similar ttl buffer into memcached.

Re: Writing a very fast cache service with millions of entries in Go

#69

Post by Xorlev »

im_down_w_otp wrote: post #64
endymi0n wrote: post #57

Earlier quoted context omitted.

can't help with 1), but as for 2), maybe Kudu is an option for you? http://blog.rodeo.io/2016/01/24/kudu-as-a-more-flexible-kafk...

Uses Raft. Not highly-available enough unfortunately. Availability > Consistency for the use case that matters to me (basically a service message bus where all ACK'd writes will eventually be read by some consumer(s), but I don't care when or in what order, just that they all eventually get consumed, and where as long as any node in the cluster is up, it'll ACK a write).

> just that they all eventually get consumed

Unless you have some sort of replication or consensus, this is still just best-effort delivery. For example, the message could be acknowledged & fsynced followed by drive death. At scale, this would happen.

However, with asynchronous best-effort replication, broker death would be much less likely to lead to a loss of messages.

In general, it's all about playing the odds. Acknowledged writes can still be lost in the best of systems if the entire DC catches fire.

If you tune Kafka settings, you essentially get this behavior. You have to ensure ZK is up, but it will allow all topics to be published to so long as a single broker is alive.

Re: Writing a very fast cache service with millions of entries in Go

#70

Post by maleck13 »

endymi0n wrote: post #38

I still don’t get why people try writing their own data store, especially in a language that's simply not very well suited to that task (and we're an almost 100% Golang shop here). Seems to be a rite of passage. The requirements are literally Public service announcement: Don't write your own data store. Repeat after me: Don't write your own data store, except if you want to experimentally find out how to build data s…

Can you explain further why golang is bad for data stores, do you mean cache in particular? Seems contrary to many of the new datastores that I see being developed: etcd[1] , cockroachdb[2], influxdb[3] just to name a couple that are written in go.

[1]https://github.com/coreos/etcd [2]https://github.com/cockroachdb/cockroach [3]https://github.com/influxdata/influxdb

Post reply on HN