Serverless: slower and more expensive

I'll start with the initial setup: We can see m1 and t1 instances in there, as well as an m3. These first two are almost certainly running paravirtualized AMIs, and there is a chance the m3 is as well. PV AMIs are going to run slower in most cases to begin with (see: http://www.brendangregg.com/blog/2017-11-29/aws-ec2-virtuali... ) and are running on old instance types that have poor cost/performance ratios compared to modern ones - m1 and t1 are ancient. So, they could likely get even better performance by switching to modern instance types, getting away from old PV style virtualization on Xen, to the more modern Nitro stuff - t3 should be able to replace all of those instances, and you gain access to SR-IOV based drivers for disk and network. Even just switching to HVM instances and going to previous generation t2 would likely be a big win vs. t1.micro and m1.small.

And this brings me to my first point in favor of serverless: With Lambda, you don't have to worry about this. You don't have to keep up on which EC2 instance has the best price/performance ratio, you don't need to bother with migrating AMIs from Xen PV to Xen HVM (or PVHVM) or KVM compatible with NVMe and ENA drivers and all of that. You just manage your code. There's a mountain of other things to worry about when managing your own EC2 instances as well, even beyond this - just general patching, keeping up to date with new kernel versions, worrying about deprecation/EOL of the OS running in your instance, making sure your code works with new versions of glibc and all the other packages when you move to a new major release, etc. You might be able to skate by for a while ignoring all of this, but eventually, it's likely to cause problems. I know what I charge people to help manage all of that sort of stuff, and it's way more than the cost difference between the two setups.

Second: There's no indication if the latency calculation includes cold-start response times. They would likely be enough to drag the average up quite a bit for lambda, and there's lots of documented ways to eliminate them from a production lambda setup. (Though, they'd also likely see a latency decrease moving to t3 for their EC2 based setup as well, as previously mentioned.)

Third: API Gateway might not be the best tool for the job, especially as the author notes they don't make much use of all of the API gateway features. You can just use an ALB instead (or depending on your use case, potentially skip any sort of front end altogether, though it sounds like ALB is probably a better fit for what they're doing) and save on complexity and cost.

All in all, this comparison isn't really showing a setup that is well optimized on either the EC2/Beanstalk or Serverless sides - there's major wins on both the performance and cost sides for both implementations. Without both being optimally setup, it's really hard to draw a good conclusion on which is a better solution from a technical standpoint, and the discussion also completely ignores the cost in managing EC2, even with Beanstalk vs. a serverless setup.

[0] https://github.com/aws/chalice

The good news; you should be able to accomplish most testing locally, in memory. The bad news; your test code is probably going to be much larger and your going to have to be very aware of the data model representing the interface to the lambda and you're going to have to test the different possible states of that data.