The fall of datacenter software • Hacker News Board

Re: The fall of datacenter software

#21

Post by majke » Thu, Aug 25, 2022, 2:57 PM UTC

“In a few years, datacenter networks will grow an order of magnitude from 40Gb to 400Gb. Systems researchers, including myself, have been preparing for this new world for almost a decade [1,2]. Demikernel is the first fully-featured nanosecond-scale OS: 100ns to get a packet from the NIC to the app in Demikernel’s DPDK TCP stack, processing 1 million packets per second per core, etc [3].”

Meanwhile a typical server is doing couple of gbps effective bandwidth, running hundreds of interconnected internal services from redis via zookeeper, ceph, dozen of docker containers, prometheus, some virtualization (kvm?), dozen of sqlite instances, consul and many many many more small random things.

“Datacenter” i work with is not “datacenter” of doing 1mpps per core.

Re: The fall of datacenter software

#22

Post by jeffbee » Thu, Aug 25, 2022, 3:16 PM UTC

majke wrote: post #21

“In a few years, datacenter networks will grow an order of magnitude from 40Gb to 400Gb. Systems researchers, including myself, have been preparing for this new world for almost a decade [1,2]. Demikernel is the first fully-featured nanosecond-scale OS: 100ns to get a packet from the NIC to the app in Demikernel’s DPDK TCP stack, processing 1 million packets per second per core, etc [3].” Meanwhile a typical server i…

All progress happens at the limit. You may just be running a lot of crud that loafs along, but out there somewhere there is an operator of something that sends a million packets per second per core and they'd rather get 2 million. You don't use your SSD but there is some organization that finds 1 million IOPS to be a little lower than they'd like.

You do benefit from these advances. Your searches get faster, your video streaming remains affordable, etc.

Re: The fall of datacenter software

#23

Post by ElevenLathe » Thu, Aug 25, 2022, 4:16 PM UTC

qsort wrote: post #3

sp0ck wrote: post #2

And at the end of all - this tailored and optimized to ns path there will be python app ;-)

Well, if you're deploying on an architecture with very specialized machinery, even up to JSON parsing and REST API handling, a scripting language isn't going to lose that much over a compiled one. Maybe not python, but the future the article talks about is definitely one where "js everywhere" is a compelling approach.

In a way it harkens back to 80s 8-bit home micros. BASIC on these boxes is incredibly slow, but you can still easily do realtime sprite-based games because all it is doing is manipulating some magic memory locations that are mapped to dedicated video-drawing hardware. Instead of the software needing to blit every frame, all it needs to do is say "Sprite 3 moved left by 3 pixels" every frame. Instead of doing collision detection in software, you just read a magic memory location that tells you which sprites have collided since you last looked.

If you're writing the kind of game that is possible to do in C64 Basic, you don't actually gain much by writing it in assembler instead. What you get from writing in assembler is an expansion of the universe of possible games.

Re: The fall of datacenter software

#24

Post by jayd16 » Thu, Aug 25, 2022, 4:21 PM UTC

EricE wrote: post #20

"They cannot continue focusing on software running on general-purpose CPUs but must re-orient themselves towards specialized hardware." In other words, mainframes. This trend has already started in general purpose computing - just look at what Apple is doing with their SOCs - dedicated cores for specific functionality as well as their own I/O engines for storage, all integrated into their SOC. Typing this on an M1 Ma…

Why is mainframe the metaphor? Aren't GPUs and sound cards a much more approachable analog?

Re: The fall of datacenter software

#25

Post by lanstin » Thu, Aug 25, 2022, 4:37 PM UTC

BirAdam wrote: post #12

sp0ck wrote: post #2

And at the end of all - this tailored and optimized to ns path there will be python app ;-)

You can be assured that the Rust zealots will rewrite the Python or JS application in Rust. They won’t sleep properly until all the world is rewritten in Rust.

I always expectrd this rewrite but so far as i can tell they, unlike GNU in the 80s, are not doing drop in replacements, pushing their adoption curve out decades. It is such a strong truth that people prefer to do new stuff rather than reimplement existing tools.

Re: The fall of datacenter software

#26

Post by crmd » Thu, Aug 25, 2022, 5:46 PM UTC

jeffbee wrote: post #22

majke wrote: post #21

“In a few years, datacenter networks will grow an order of magnitude from 40Gb to 400Gb. Systems researchers, including myself, have been preparing for this new world for almost a decade [1,2]. Demikernel is the first fully-featured nanosecond-scale OS: 100ns to get a packet from the NIC to the app in Demikernel’s DPDK TCP stack, processing 1 million packets per second per core, etc [3].” Meanwhile a typical server i…

All progress happens at the limit. You may just be running a lot of crud that loafs along, but out there somewhere there is an operator of something that sends a million packets per second per core and they'd rather get 2 million. You don't use your SSD but there is some organization that finds 1 million IOPS to be a little lower than they'd like. You do benefit from these advances. Your searches get faster, your vid…

How do you find these limit users? My startup makes low cost block and file storage on AWS that does 2M IOPS per volume with 200 uSec latency, and we’re surprised to find most customers are perfectly happy using gp3 with 3k IOPs and 1-2 ms latency.

Re: The fall of datacenter software

#27

Post by daveguy » Thu, Aug 25, 2022, 6:15 PM UTC

crmd wrote: post #26

jeffbee wrote: post #22

Earlier quoted context omitted.

All progress happens at the limit. You may just be running a lot of crud that loafs along, but out there somewhere there is an operator of something that sends a million packets per second per core and they'd rather get 2 million. You don't use your SSD but there is some organization that finds 1 million IOPS to be a little lower than they'd like. You do benefit from these advances. Your searches get faster, your vid…

How do you find these limit users? My startup makes low cost block and file storage on AWS that does 2M IOPS per volume with 200 uSec latency, and we’re surprised to find most customers are perfectly happy using gp3 with 3k IOPs and 1-2 ms latency.

What's your startup? I'd be interested to see the cost. Persistence gotchas, encryption / security.

Re: The fall of datacenter software

#28

Post by bombcar » Thu, Aug 25, 2022, 6:16 PM UTC

crmd wrote: post #26

jeffbee wrote: post #22

Earlier quoted context omitted.

All progress happens at the limit. You may just be running a lot of crud that loafs along, but out there somewhere there is an operator of something that sends a million packets per second per core and they'd rather get 2 million. You don't use your SSD but there is some organization that finds 1 million IOPS to be a little lower than they'd like. You do benefit from these advances. Your searches get faster, your vid…

How do you find these limit users? My startup makes low cost block and file storage on AWS that does 2M IOPS per volume with 200 uSec latency, and we’re surprised to find most customers are perfectly happy using gp3 with 3k IOPs and 1-2 ms latency.

I have a suspicion that many of them are not running on AWS, since they'd like to remove as much cloud/network latency as possible.

Re: The fall of datacenter software

#29

Post by bogomipz » Thu, Aug 25, 2022, 6:27 PM UTC

The author states:

>"Existing hardware resource isolation is primitive and there are no hardware mechanisms for scheduling CPU cycles (interrupts are too clunky) or memory (likewise for page faults)."

Don't Linux cgroups and cpu shares handle exactly this? Or is acknowledging this and meant to be a critique of CFS and its uses throttling when there is contention?

Re: The fall of datacenter software

#30

Post by oneplane » Thu, Aug 25, 2022, 6:49 PM UTC

crmd wrote: post #26

jeffbee wrote: post #22

Earlier quoted context omitted.

All progress happens at the limit. You may just be running a lot of crud that loafs along, but out there somewhere there is an operator of something that sends a million packets per second per core and they'd rather get 2 million. You don't use your SSD but there is some organization that finds 1 million IOPS to be a little lower than they'd like. You do benefit from these advances. Your searches get faster, your vid…

How do you find these limit users? My startup makes low cost block and file storage on AWS that does 2M IOPS per volume with 200 uSec latency, and we’re surprised to find most customers are perfectly happy using gp3 with 3k IOPs and 1-2 ms latency.

If your application has to go to disk to fetch something it's already accepted that it'll take time. Things that need to be really fast go to RAM or cache.

For everything else you're already getting into niches.