How often OS/network/context overhead is the bottleneck? In my experience most of the time it's the DB. Even if it fits in RAM, complex queries always take most of the time. (Web dev here).
Russ’ 10 Ingredient Recipe for Making 1 Million TPS on $5K Hardware
21–30 of 45 posts
Re: Russ’ 10 Ingredient Recipe for Making 1 Million TPS on $5K Hardware
#22Earlier quoted context omitted.
Some of the larger combats (5000+ ships) in EVE Online might do that. (I single EVE out because other MMOs generally shard by user-cohort, so having that number of people on one shard is impossible. EVE, meanwhile, shards by location within the virtual world (each star system is a shard), so the entire player-base can "gather" on a single shard for a confrontation.)
> In February 2013, EVE Online reached over 500,000 subscribers. So what, the entire game userbase needs to be awake and in the same spot? :0)
Perfect timing: http://www.theverge.com/2013/7/28/4565558/eve-online-biggest...
Re: Russ’ 10 Ingredient Recipe for Making 1 Million TPS on $5K Hardware
#23Ingredient 0: redefine “transaction” to mean nothing remotely like what it means to everyone else.
Which part of the article makes you think that the operations being made are not ACID?
Re: Russ’ 10 Ingredient Recipe for Making 1 Million TPS on $5K Hardware
#24Earlier quoted context omitted.
Intrusion protection systems must legitimately handle upwards of 15M queries/sec when under attack (on a 10 Gbps link).
I'm curious, can you elaborate on this? What would they be querying? What kind of intrusion and what kind of attack?
TCP flow state, IP shunning & reputation, client rate limiting, others I'm probably missing.
Specifically during DDoS attacks, an IPS must usefully distinguish bad from good traffic at packet rates saturating a link. This inevitably involves maintaining lots of per-client and per-IP state. Caching is of little help precisely due to the distributed nature of the attack, and you can't shed load since that only helps the attacker.
In this situation, memory stalls become your biggest bottleneck – each one can eat on the order of 10% of your processing budget in a run-to-completion (RTC) design. The only solution (beyond tricksier data layouts) is memory latency hiding via micro- or hyperthreading (kernel context switches are just too slow). Rearchitecting a RTC design into a micro-threaded model is a lot of work, and bug-prone. Hyperthreading gives you latency hiding "for free", if the silicon supports it.
Generally you want between 2-4 micro- or hyperthreads. A second micro/hyperthread will generally just help keep the pipeline busy outside memory stalls; hence you can eke out extra performance with a third or a fourth. Intel chips only support two hyperthreads (when they do, and the OS supports it). Some more specialized processors support more.
Re: Russ’ 10 Ingredient Recipe for Making 1 Million TPS on $5K Hardware
#25> context switches passing tcp-packets back and forth from the operating system were taking up 35% You could get rid of this (and in doing so, double your TPS) by switching to a memory-polling-based network driver like PF_RING [1] (and obviously, keeping the kernel on its own core like you are doing). > Lookup data in memory (this is fast enough to happen in-thread) > I knew I had it right when I watched the output o…
Of course, if you use Netmap/DPDK/PF_RING you have to bring your own TCP stack, which is more than many app developers are comfortable with.
Re: Russ’ 10 Ingredient Recipe for Making 1 Million TPS on $5K Hardware
#26> context switches passing tcp-packets back and forth from the operating system were taking up 35% You could get rid of this (and in doing so, double your TPS) by switching to a memory-polling-based network driver like PF_RING [1] (and obviously, keeping the kernel on its own core like you are doing). > Lookup data in memory (this is fast enough to happen in-thread) > I knew I had it right when I watched the output o…
Of course, if you use Netmap/DPDK/PF_RING you have to bring your own TCP stack, which is more than many app developers are comfortable with.
Excellent point which I missed :) Can you tell I live in an L3 bubble?
I am surprised (from a quick Google) there is no open-source user-space PF_RING-aware TCP stack. Am I missing something?
Re: Russ’ 10 Ingredient Recipe for Making 1 Million TPS on $5K Hardware
#27Ingredient 0: redefine “transaction” to mean nothing remotely like what it means to everyone else.
Which part of the article makes you think that the operations being made are not ACID?
> 100% in-memory workload. Don’t even think about hitting disk for 0.0001% of these requests.
They also discuss how these are 1M read requests.
Re: Russ’ 10 Ingredient Recipe for Making 1 Million TPS on $5K Hardware
#28Earlier quoted context omitted.
Of course, if you use Netmap/DPDK/PF_RING you have to bring your own TCP stack, which is more than many app developers are comfortable with.
Excellent point which I missed :) Can you tell I live in an L3 bubble? I am surprised (from a quick Google) there is no open-source user-space PF_RING-aware TCP stack. Am I missing something?
When you're doing packet capture that normally costs one system call per packet, Netmap/DPDK/PF_RING are a clear win. But with TCP a single system call can send or receive many packets and TSO/LRO offloads help even more. I haven't seen any numbers about the benefits of kernel bypass for TCP; maybe SolarFlare has some.
Re: Russ’ 10 Ingredient Recipe for Making 1 Million TPS on $5K Hardware
#29T in TPS means Transactions, right? What kind of a transaction is that one which uses no persistent storage?)
1 million network-to-memory writes, well, that is quite possible, but, please, do not call this a transaction in the way it meant in TPS.)
What was meant in the old days by transaction, was an atomic operation which completes after storing the data in a persistent (usually direct-access, which means no buffering by an OS kernel) storage, so it could be read without any corruption if a power loss will occur the very next second.
Re: Russ’ 10 Ingredient Recipe for Making 1 Million TPS on $5K Hardware
#30How often OS/network/context overhead is the bottleneck? In my experience most of the time it's the DB. Even if it fits in RAM, complex queries always take most of the time. (Web dev here).
Pretty often, if you're doing simple queries. From there, you can either increase the size of the query, or make it more complex (which increases its computational size) to amortize away the overhead. It always depends on the workload but cases like this, where the overhead is significant, are fairly common.