Does this count network egress costs (for s3, lambda)? Those are hard to calculate but can add up. Also, you can uses sqs for queues, that has a pretty big free tier.
Finally, I think that bending the architecture of your app around what's free is not the best idea, especially if you are trying to get something out there to see if there is customer demand. (And doubly of you haven't built a cloud native app like is outlined here.)
In my mind the best way to build software for a startup is to build it using what you know as fast as you can. Avoid technical risk, because you have a boatload of business risk.
For me, that'd be using rails on heroku, which is still under $200/year for a fully functional dyno and database. For others it might be some varient of a mvc framework on a hosting provider. For others it might be WordPress (gasp!). For others it might be a cloud native app, as this post describes.
As long as you aren't spending extravagantly, time is more important than money when figuring out what your customers need.
This is true both in companies that have raised money and bootstrapped companies, for different reasons. For the first, you took money and need to figure out your product market for or scaling strategy ASAP. For the second, your time is super valuable because it is tied to your motivation. Doing work directly tied to customer value is a great motivator. (Doing other fun technical things that don't deliver customer value is a good way to learn things, but a bad way to run a bootstrapped business.)