For my first serverless project, I built a service that manages an advertising account, by analysing ads and optimising the spend spread for a configured goal. I was surprised to find that one of the most difficult aspects, was in throttling outgoing API requests. Basically, I needed to avoid abusing a 3rd party service by hammering it with parallel or rapid fire requests.
I discovered all kinds of weird solutions:
- Push a "please wait" message onto an SQS queue, and set a Cloud watch alarm to fire when the queue length was >= 1 for at least one second. Have a variety of lambdas respond to the alert by checking if the have any work to do, and then racing to pop the queue and do one operation. I think I needed SNS in there too, for some reason. Fanout?
- Build a town clock on an EC2 and have it post "admit one" tokens to SNS. Nice and simple, but I was evaluating serverless, and adding a server felt like defeat. Might as well just run the whole thing on that EC2.
- Have on lambda pop request descriptions from a queue, and sleep for one second after sending each one. The request responses would have been pushed onto another queue. All of the other lambdas could just handle the events they cared about as quickly as data became available. Sleeping in a lambda just felt wrong, but for one second, it probably would have been fine.
In the end, I was still experimenting and exploring when my estimate loomed near, so I just wrapped up all of the application code and piped it all together with delays in loops that made API requests. Deployed it to Heroku in a hurry with minimal fuss.
What I learned:
- There are bad use cases for serverless, and I had started on one (SLS doesn't wait on no one)
- Serverless does encourage a clean architecture, from a code perspective: I was able to quickly transform it into a completely different type of application.
- AWS's services have lots of little surprises in their feature set: SQS doesn't fire any kind of event. SNS can't schedule events in the future (maybe I should have expected that one). Cloud watch events can only trigger a single lambda.
- There is a need for a town clock / scheduler as a service in AWS'. How many people have built that themselves?