Handling 1M Requests per Minute with Go
1–10 of 109 posts
Re: Handling 1M Requests per Minute with Go
#2Go astroturfing continues ...
Re: Handling 1M Requests per Minute with Go
#3Arguably, UploadToS3 should not be a method of *Payload.
Suggestion: Make an S3-uploader package with _internal_ connection pooling, upload queueing and concurrency handling.
Re: Handling 1M Requests per Minute with Go
#4Other than the worker/concurrency mechanism being part of the language, what's the difference to a RabbitMQ-Worker architecture? You might even argue that the lack of persistence (in the given example) is a potential source for data loss.
Re: Handling 1M Requests per Minute with Go
#5Go astroturfing continues ...
Indeed, think the solution probably did most of the work here, although Go's channels would have made it a little easier to implement.
Re: Handling 1M Requests per Minute with Go
#6>> But since the beginning, our team knew that we should do this in Go because during the discussion phases we saw this could be potentially a very large traffic system
I don't get this reasoning.
Re: Handling 1M Requests per Minute with Go
#7Other than the worker/concurrency mechanism being part of the language, what's the difference to a RabbitMQ-Worker architecture? You might even argue that the lack of persistence (in the given example) is a potential source for data loss.
I'm guessing persistence is not a priority.
The difference is management of a simple process (behind elastic load balancer, of course) vs a more complicated architecture with three distinct, load balanced process types (webserver -> queue -> worker).
Re: Handling 1M Requests per Minute with Go
#8>> But since the beginning, our team knew that we should do this in Go because during the discussion phases we saw this could be potentially a very large traffic system I don't get this reasoning.
They're going from Ruby, so Go is a pretty easy transition with respect to skill set and helped them accomplish scale (as seen from the analysis).
I agree that other languages could accomplish the same effect, but I'm not surprised by their sentiment given where they were coming from.
Re: Handling 1M Requests per Minute with Go
#916k requests per second isn't fast.
Re: Handling 1M Requests per Minute with Go
#10Go astroturfing continues ...
Based on the write-up? Regardless of language choice, I thought the analysis was pretty honest and the data suggests they did indeed engineer a solution worth being proud of. Did you get beyond the title?