Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An FPGA-based In-line Accelerator for Memcached (2013) [pdf]

hotchips.org

Post reply on HN
Print view View original

11–20 of 34 posts

Re: An FPGA-based In-line Accelerator for Memcached (2013) [pdf]

#11

Post by andrewcanis »

dormando wrote: post #5
andrewcanis wrote: post #4

Our startup is working on accelerators using FPGAs on AWS including memcached. Using a single AWS F1 (FPGA) instance, our Memcached accelerator achieves over 11 million ops/sec at less than 300 microsecond latency. Compared to ElastiCache, the AWS-managed CPU Memcached server, our Memcached accelerator offers 9X better throughput, 9X lower latency, and 10X better throughput/$. We need to batch multiple requests per E…

How is elasticache so slow? what instances does it run on? edit: r4.4xlarge as per the link. 16vcpu? You should be able to beat on latency but beating on throughput means elasticache is misconfigured, likely. Or you're putting on way too much set traffic (think I saw you set the bench to 1:1 ratio of gets to sets?)

I wouldn't characterize Elasticache as running slow, a single instance in this case is handling 1.3M request/sec. But we can be 9X faster by batching multiple requests per packet and then offloading the TCP network stack and memcached operations to the FPGA. The FPGA allows us to handle the requests at network line-rate, even with small 100-byte requests. On Elasticache, past a certain point these small requests start to overload the CPU.

The interesting part is the FPGA could still do much more computation (for example, compression or encryption) while maintaining the same throughput due to hardware pipelining. We described this concept further in the blog post I linked to.

Re: An FPGA-based In-line Accelerator for Memcached (2013) [pdf]

#12

Post by ksec »

dormando wrote: post #3

This seems old (2013ish?). There're newer "key/value on FPGA" papers that're more modern. If memcached papers have one thing in common, it's an uncanny ability to get the comparison software to run as slowly as possible. 100k ops/sec/core is what you get when using a single client connection with blocking I/O. Using more clients (as per a normal prod setup) or pipelining queries is more like 1m ops+/core, with writes…

Especially when we have Xeon -D that goes up to 16 Core. AMD EPYC that gives more Core per dollar. And more IPC soon in Zen 2. 7nm and 10nm from AMD and Intel next year on Server. Not to mention it now Support ARMv8

So Excuse my ignorance, apart from AWS or Azure Scales, what would anyone uses Memcached on FPGA?

* I wouldn't mind if the system was simple plug and play and has all the benefits, cost saving without the headache. But very rarely are any technology deployment without any headache or hassle free.

Re: An FPGA-based In-line Accelerator for Memcached (2013) [pdf]

#13

Post by andrewcanis »

brooksbp wrote: post #10
rjeli wrote: post #6

Earlier quoted context omitted.

really wish AWS supported Ethernet on fpga

There aren't Ethernet interfaces (transceivers, PCS/PMA, MAC) on the AWS FPGAs?

Although there are Ethernet transceivers on the AWS FPGAs (these are Xilinx UltraScale+ VU9P FPGAs) they are unused and not connected directly to the AWS network. Instead the FPGAs are connected over PCIe to a host server, which has a standard NIC. This required us to use DPDK to bypass the kernel and pass raw network traffic directly to/from the FPGA over PCIe.

We have been told by cloud providers that the FPGA cannot be directly connected due to network security concerns. Since there is no easy way to control how the arbitrary hardware programmed by users on the FPGA will interact with their network. Microsoft has been using FPGAs directly connected to their network (called a bump-in-the-wire architecture) for the FPGAs used in their datacenter (see Project Catapult for details). But these FPGAs are not programmable by Azure users yet.

Re: An FPGA-based In-line Accelerator for Memcached (2013) [pdf]

#14

Post by andrewcanis »

baybal2 wrote: post #7
andrewcanis wrote: post #4

Our startup is working on accelerators using FPGAs on AWS including memcached. Using a single AWS F1 (FPGA) instance, our Memcached accelerator achieves over 11 million ops/sec at less than 300 microsecond latency. Compared to ElastiCache, the AWS-managed CPU Memcached server, our Memcached accelerator offers 9X better throughput, 9X lower latency, and 10X better throughput/$. We need to batch multiple requests per E…

About to congratulate you that you are about to make competition to Alibaba.com. On our side, maxing the IOPS was the easy part. The hard one was to marry the protocol with converged/deterministic Ethernet with RDMA. We were split in between "one request, one frame/packet burst" vs "all requests are somehow smartly aligned with frames by stateful logic." The first one was surprisingly susceptible to performance artif…

Sounds interesting, are there any papers or public details that you could point me to with more technical information about this Alibaba RDMA-based memcached project? Alibaba also has FPGA instances available and we have been investigating their cloud offering.

Re: An FPGA-based In-line Accelerator for Memcached (2013) [pdf]

#15

Post by baybal2 »

andrewcanis wrote: post #14
baybal2 wrote: post #7

Earlier quoted context omitted.

About to congratulate you that you are about to make competition to Alibaba.com. On our side, maxing the IOPS was the easy part. The hard one was to marry the protocol with converged/deterministic Ethernet with RDMA. We were split in between "one request, one frame/packet burst" vs "all requests are somehow smartly aligned with frames by stateful logic." The first one was surprisingly susceptible to performance artif…

Sounds interesting, are there any papers or public details that you could point me to with more technical information about this Alibaba RDMA-based memcached project? Alibaba also has FPGA instances available and we have been investigating their cloud offering.

Invite only till at least autumn, and they have no English speaking support (it is for their Chinese only DCs.)

Google Alibaba China F1 or F2.

As for the offering, the idea is that end the clients will only have to deal with SDK and libs on instances, not raw RDMA or anything related to internal infrastructure. This is how much I am allowed to say besides the fact of its existence.

From their current experience, and that of other hosting providers, not many people who go with F1, F2, or other FPGA instance actually do reap any benefit, and some drop mid-way. That's why they want to get more people using them pass the "toying with it stage." The "Herokuification" (god, Heroku sounds beyond hilarious in Russian) is there to let people use the common APIs while getting benefit from FPGA performance, without dealing with things outside of average webdev area of expertise.

Re: An FPGA-based In-line Accelerator for Memcached (2013) [pdf]

#16

Post by glangdale »

dormando wrote: post #3

This seems old (2013ish?). There're newer "key/value on FPGA" papers that're more modern. If memcached papers have one thing in common, it's an uncanny ability to get the comparison software to run as slowly as possible. 100k ops/sec/core is what you get when using a single client connection with blocking I/O. Using more clients (as per a normal prod setup) or pipelining queries is more like 1m ops+/core, with writes…

Can confirm this eerie ability of FPGA and h/w folks from another domain (regular expressions - I'm the designer of Hyperscan, a s/w solution).

Every corner case that could be found in s/w was always the topic of a excited benchmark. Also, the old trick of 'hey, let's drop all the matches on the floor in our h/w or FPGA, while getting a huge number of matches in s/w and making the s/w guys look ridiculous'.

Every time I read a paper touting a great new speedup on FPGA (over some crap s/w implementation) I'm reminded of that old joke about the Texan visiting Israel and telling the owner of some small farm that "he can get on a tractor and ride for days without getting the the boundary of his property." The Israeli nods sympathetically and says "Yes, I too used to have a tractor like that".

Re: An FPGA-based In-line Accelerator for Memcached (2013) [pdf]

#17

Post by dormando »

andrewcanis wrote: post #11
dormando wrote: post #5

Earlier quoted context omitted.

How is elasticache so slow? what instances does it run on? edit: r4.4xlarge as per the link. 16vcpu? You should be able to beat on latency but beating on throughput means elasticache is misconfigured, likely. Or you're putting on way too much set traffic (think I saw you set the bench to 1:1 ratio of gets to sets?)

I wouldn't characterize Elasticache as running slow, a single instance in this case is handling 1.3M request/sec. But we can be 9X faster by batching multiple requests per packet and then offloading the TCP network stack and memcached operations to the FPGA. The FPGA allows us to handle the requests at network line-rate, even with small 100-byte requests. On Elasticache, past a certain point these small requests star…

I characterize it as slow because I know it can saturate the packet rate AWS gives it with software memcached. If the packet rate were much higher then you might win out.

The only reason why you can claim 9x latency is because you've saturated the worker threads. You should still win on latency even if it were properly bottlenecking on the network, but 9x throughput and 9x latency is completely false as a capacity limit in this test.

The other issue is 100 bytes isn't typical. It's common but almost every user has a varied workload. Deploying FPGA's for the larger cache values ends up being a waste. I designed a new storage system based off of offloading larger cold keys to flash, even.

Re: An FPGA-based In-line Accelerator for Memcached (2013) [pdf]

#18

Post by analognoise »

andrewcanis wrote: post #13
brooksbp wrote: post #10

Earlier quoted context omitted.

There aren't Ethernet interfaces (transceivers, PCS/PMA, MAC) on the AWS FPGAs?

Although there are Ethernet transceivers on the AWS FPGAs (these are Xilinx UltraScale+ VU9P FPGAs) they are unused and not connected directly to the AWS network. Instead the FPGAs are connected over PCIe to a host server, which has a standard NIC. This required us to use DPDK to bypass the kernel and pass raw network traffic directly to/from the FPGA over PCIe. We have been told by cloud providers that the FPGA cann…

I wonder if you could bound the problem with formal verification so that a custom design could only expose a certain activity profile, or an embedded hardware kill switch that would eliminate access after some detectable malicious interaction (at the hardware level).

Would that even be possible, or does it boil down to the halting problem (how can you know a sequence of transactions is malicious)?

Re: An FPGA-based In-line Accelerator for Memcached (2013) [pdf]

#19

Post by rjeli »

glangdale wrote: post #16
dormando wrote: post #3

This seems old (2013ish?). There're newer "key/value on FPGA" papers that're more modern. If memcached papers have one thing in common, it's an uncanny ability to get the comparison software to run as slowly as possible. 100k ops/sec/core is what you get when using a single client connection with blocking I/O. Using more clients (as per a normal prod setup) or pipelining queries is more like 1m ops+/core, with writes…

Can confirm this eerie ability of FPGA and h/w folks from another domain (regular expressions - I'm the designer of Hyperscan, a s/w solution). Every corner case that could be found in s/w was always the topic of a excited benchmark. Also, the old trick of 'hey, let's drop all the matches on the floor in our h/w or FPGA, while getting a huge number of matches in s/w and making the s/w guys look ridiculous'. Every tim…

I don’t understand the joke?

Re: An FPGA-based In-line Accelerator for Memcached (2013) [pdf]

#20

Post by namibj »

rjeli wrote: post #19
glangdale wrote: post #16

Earlier quoted context omitted.

Can confirm this eerie ability of FPGA and h/w folks from another domain (regular expressions - I'm the designer of Hyperscan, a s/w solution). Every corner case that could be found in s/w was always the topic of a excited benchmark. Also, the old trick of 'hey, let's drop all the matches on the floor in our h/w or FPGA, while getting a huge number of matches in s/w and making the s/w guys look ridiculous'. Every tim…

I don’t understand the joke?

I found https://www.reddit.com/r/Jokes/comments/2uj2ot/a_texas_ranch... And it seems to be not that good of a joke.
Post reply on HN