Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Faster – Fast key-value store from Microsoft Research

github.com

Post reply on HN
Print view View original

11–20 of 84 posts

Re: Faster – Fast key-value store from Microsoft Research

#11

Post by staticassertion »

fein wrote: post #6

It would be nice to know why I would use this instead of something tried and tested like Redis, but the sparse description doesn't really help.

Redis is like 0 for 3 from a CAP perspective as far as I understand it. Fine for a cache, but if you want a KV store you can rely on for some properties, it does not seem viable.

So redis is 'tried and true' but may not meet your constraints.

https://www.quora.com/What-is-Redis-in-the-context-of-the-CA...

I haven't read the Faster paper yet (planning to) so I don't know that it provides better guarantees. But I personally would like a simple-as-redis KV store that provides better guarantees.

edit: Ah, yeah, Faster doesn't seem to even really be directly comparable to Redis - seems more like rocksdb, and not a distributed system.

Re: Faster – Fast key-value store from Microsoft Research

#12

Post by kodablah »

mooman219 wrote: post #3

"What differentiates FASTER are its cache-optimized index that achieves very high performance — up to 160 million operations per second when data fits in memory;" I really dislike when papers make performance claims like this in the introduction. That "160 million" number is so meaningless at face value because everything from the runtime environment to the hardware is going to play a huge role in ops. I rather see h…

And: "FASTER achieves higher throughput than current systems, by more than two orders of magnitude, and scales better than current pure in-memory data structures, for in-memory working sets."

Looking at 7.2 of the paper, they probably mean "more than 2x", definitely not exponentially faster in most cases. Still nice work though.

Re: Faster – Fast key-value store from Microsoft Research

#14

Post by kodablah »

fein wrote: post #6

It would be nice to know why I would use this instead of something tried and tested like Redis, but the sparse description doesn't really help.

On the surface, I'd guess the statically linked vs separate daemon differences apply (both have obvious pros and cons depending upon requirements and deployment scenarios).

Also, MS research is just that, a research wing and many of their libs go stale/unsupported after written.

Re: Faster – Fast key-value store from Microsoft Research

#15

Post by fizx »

If you want concrete benchmarks, they compare to RocksDB and Redis around page 10 of their academic paper. (https://www.microsoft.com/en-us/research/uploads/prod/2018/0...)

TL;DR:

I find their choice of benchmarks to be very convenient.

They tested on in-memory 8 byte payloads and were way faster than RocksDB and Redis. They then tested against only different configurations of themselves for configurations that hit disk. They also tested an embedded version of their software vs a Redis that hits loopback (rather than e.g. running against the raw redis data structure implementation, which would have been a more fair comparison).

Re: Faster – Fast key-value store from Microsoft Research

#16

Post by dlahoda »

atombender wrote: post #7

For those wondering what this is, it is not a client/server app, from what I can tell, but an embedded engine. It looks like it's intended to be a library, and it's been implemented in two languages (C# and C++). To get something like Redis or Riak you would have to build API, clustering, etc. on top of it. So it's more analogous to libraries like RocksDB, BoltDB, BDB etc. Paper: https://www.microsoft.com/en-us/resea…

c++ part is one liner intrinsic which may be supported directly with new .net and several io methods which would be heavy on pinvoke calls like 5 per method if in c#, but these are very simple. so c++ part could be done in c and easily ported to unix. i guess c# should allow for pinvoke strategy like in lua to replace c++ more.

Re: Faster – Fast key-value store from Microsoft Research

#17

Post by baybal2 »

I'm sceptical. Modern top tier in mem key value storages like LMDB are just within 20% 30% away from CPU maximum IO throughput on server class hardware. There must be some tricks in their metric.

Do they actually fetch data or they just measure how many memory pointers per second they can dump?

Re: Faster – Fast key-value store from Microsoft Research

#18

Post by mgraczyk »

haney wrote: post #8
mooman219 wrote: post #3

"What differentiates FASTER are its cache-optimized index that achieves very high performance — up to 160 million operations per second when data fits in memory;" I really dislike when papers make performance claims like this in the introduction. That "160 million" number is so meaningless at face value because everything from the runtime environment to the hardware is going to play a huge role in ops. I rather see h…

It'd be awesome if they provided a comparison of other tools on the same hardware like, "on an AWS M4.xlarge instance we were able to achieve 160m/ops/sec when the dataset fit into memory where as redis only did X" Lacking that I agree it's a pretty meaningless stat.

Check out the linked paper for a detailed performance comparison.

Re: Faster – Fast key-value store from Microsoft Research

#20

Post by hashrate »

This is really interesting.

This is usually what people would do when using NoSQL database to reduce latency by caching elements with an in-memory database.

Basically they are mixing up Redis with RocksDB , which is what devs usually do to get even higher throughput by storing IDs in Redis to save a call to RocksDB.

Now what bother me is the look of repository , it looks completely rushed out.

No logo , unclear description of the tech...

Hence , as other mentioned it's just "an engine" , it doesn't actually contain the network layer, the clustering mechanism etc...

I guess it's probably the tech that is powering their flagship engine : "CosmosDB".

Post reply on HN