when i read it "blah blah with Go", i know for sure people is going to discuss about Node =))
Handling 1M Requests per Minute with Go (2015)
21–30 of 39 posts
Re: Handling 1M Requests per Minute with Go (2015)
#22Earlier quoted context omitted.
A random result from Google http://blog.digg.com/post/141552444676/making-the-switch-fro... . My day job is writing large scale node services that service over million concur. clients doing fairly intensive data acquisition/processing so I am not hating on Node :)
Since Node is single threaded how do you make that scale on multi core without something that spawn multiple Node app and thus using 2x 3x 4x the memory ...?
It's obviously application specific in our case data is written to Kafka and a bunch of workers are reading it off and do processing
Re: Handling 1M Requests per Minute with Go (2015)
#23Since when did per minute become a thing? 16K/sec is typical for a node app on fairly avg server.
16k r/s is not trivial to get for a non-hello-world app on NodeJS even on a big server, but if you read the article you'd see it's actually about a quarter of that. ... Yeah basically it is an article about using concurrency.
I was not obviously commenting on the content but on a trend of coming up with measurements that make for a catchy title.
Re: Handling 1M Requests per Minute with Go (2015)
#24Earlier quoted context omitted.
16k r/s is not trivial to get for a non-hello-world app on NodeJS even on a big server, but if you read the article you'd see it's actually about a quarter of that. ... Yeah basically it is an article about using concurrency.
I was not obviously commenting on the content but on a trend of coming up with measurements that make for a catchy title.
> 16K/sec is typical for a node app on fairly avg server.
This certainly doesn't read like "I'm talking about the title" to me.
Also, in other parts of the thread you take performance questions seriously, further diluting the impression your post was about the title or phrasing.
Re: Handling 1M Requests per Minute with Go (2015)
#25Earlier quoted context omitted.
I was not obviously commenting on the content but on a trend of coming up with measurements that make for a catchy title.
> 16K/sec is typical for a node app on fairly avg server. This certainly doesn't read like "I'm talking about the title" to me. Also, in other parts of the thread you take performance questions seriously, further diluting the impression your post was about the title or phrasing.
The title is presenting handling 16K/sec as something special my only original point was that the author is using per minute metrics to make numbers look more impressive. It's very possible that I did a very poor job of making that point :). My comment that I am sure Go more performant than node was beign downvoted so that took things in a diff. direction.
Re: Handling 1M Requests per Minute with Go (2015)
#26They're uploading each POST payload to S3 at a rate of up to 1M uploads a minute? They're going to go broke from S3 operational fees. PUT fees are $0.005 per 1k, or $5/minute, or $7200/day
S3 is an absolutely terrible financial choice for systems that need to store a vast number of tiny files.
Re: Handling 1M Requests per Minute with Go (2015)
#27They're uploading each POST payload to S3 at a rate of up to 1M uploads a minute? They're going to go broke from S3 operational fees. PUT fees are $0.005 per 1k, or $5/minute, or $7200/day S3 is an absolutely terrible financial choice for systems that need to store a vast number of tiny files.
Storing millions of tiny files in any filesystem is a terrible choice.
Re: Handling 1M Requests per Minute with Go (2015)
#28They're uploading each POST payload to S3 at a rate of up to 1M uploads a minute? They're going to go broke from S3 operational fees. PUT fees are $0.005 per 1k, or $5/minute, or $7200/day S3 is an absolutely terrible financial choice for systems that need to store a vast number of tiny files.
Storing millions of tiny files in any filesystem is a terrible choice.
Fair point, I was mostly focused on the absurd cost for that specific implementation. What would you suggest as an alternative? A document-oriented database?
Re: Handling 1M Requests per Minute with Go (2015)
#29They're uploading each POST payload to S3 at a rate of up to 1M uploads a minute? They're going to go broke from S3 operational fees. PUT fees are $0.005 per 1k, or $5/minute, or $7200/day S3 is an absolutely terrible financial choice for systems that need to store a vast number of tiny files.
They're batching the requests into larger files on S3. The 1M refers to the number of HTTP requests hitting their server.
Re: Handling 1M Requests per Minute with Go (2015)
#30Earlier quoted context omitted.
Storing millions of tiny files in any filesystem is a terrible choice.
Fair point, I was mostly focused on the absurd cost for that specific implementation. What would you suggest as an alternative? A document-oriented database?
An underused option is actually SQLite. That gives you a surprisingly feature-rich system with very low overhead. In fact, you may see benefits: faster access and less disk usage https://www.sqlite.org/fasterthanfs.html
A key-value store would probably work well, depending on how well its storage layer is architected.