Live data from Hacker News

Making 1M requests with Python-aiohttp

pawelmhm.github.io

21–30 of 84 posts

Re: Making 1M requests with Python-aiohttp

#21

I really keep wishing that there would be benchmark comparisons of asyncio/aiohttp with gevent/python2 . Performance would be a killer reason to migrate immediately to Py3. What I suspect though is that asyncio is not all that better than gevent. Can someone correct me on this?

Is there anything inherent to Python3 that is slower than Python2? Or is it just some of the performant packages still have not been ported to Python3?

Re: Making 1M requests with Python-aiohttp

#22
post #10

Earlier quoted context omitted.

Why you say is not amazing? Honestly curious here :)

Because it's trivial. I would be interested in anything doing 10,000+ req/sec on a cheap VPS. 320 is nothing. People achieve 2 million requests/second with C++ on EC2: https://medium.com/swlh/starting-a-tech-startup-with-c-6b5d5...

Absolutely excellent article. I always keep C++ at the back of my mind in case I need it some day, so I think the list of libraries they used will be useful for me in the future. Thanks.

Re: Making 1M requests with Python-aiohttp

#23

1,000,000 requests in 52 minutes is just 320 req/sec. Am I missing something? What's so amazing about this? I just deployed some production feed that serves at 1955 requests/second on a cheap VPS in freaking PHP, one of the slowest languages out there.

> Am I missing something? What's so amazing about this?

The article is not about testing performance of a web server, but showcasing performance differences between synchronous and asynchronous code using asyncio. So, not about serving requests, but consuming.

Re: Making 1M requests with Python-aiohttp

#24

Does anyone enjoy doing async work in python? I've done a few hobby projects and honestly I was yearning for javascript + async lib after awhile. As great as python is maybe we should yield async programming to the languages designed for it?

This is a genuine question: in what ways is Python's async implementation lacking? Could it have been baked in a better way? In what ways do languages that were supposedly designed for async programming different than Python? Python is definitely lacking an elegant interface for async programming.

simple reason - there is NO framework that is built ground up for nodejs style async programming.

Obviously there is Twisted and Tornado - but gevent or asyncio are actually the paradigms that people are using now. If there were a Flask like framework that was ground up built to leverage async (rather than bolting it on) and included all the batteries for web development.. then python would have a serious edge over node.

Re: Making 1M requests with Python-aiohttp

#25
post #7

Earlier quoted context omitted.

This is a genuine question: in what ways is Python's async implementation lacking? Could it have been baked in a better way? In what ways do languages that were supposedly designed for async programming different than Python? Python is definitely lacking an elegant interface for async programming.

I think that Python 3.5 now has a very elegant interface for async programming. I prefer Tornado to the standard library's asyncio, but the new keywords are nice for both packages (disclaimer: I'm the maintainer of Tornado). The downsides have nothing to do with the design of the language. The problem is that introducing a new concurrency model late in a language's life splits the ecosystem. Most existing packages ar…

>> The problem is that introducing a new concurrency model late in a language's life splits the ecosystem.

Really you could say that about a number of features/changes in python 3, not just the new async syntax. Python 3 itself was an ecosystem-splitting instrument.

Re: Making 1M requests with Python-aiohttp

#26

I really keep wishing that there would be benchmark comparisons of asyncio/aiohttp with gevent/python2 . Performance would be a killer reason to migrate immediately to Py3. What I suspect though is that asyncio is not all that better than gevent. Can someone correct me on this?

Is there anything inherent to Python3 that is slower than Python2? Or is it just some of the performant packages still have not been ported to Python3?

i keep looking for a reason to switch to python 3 and cant find one. Plus if I want to use the cool stuff in Pypy.. then I better not !

overall - very less reason to consider Py3 at all. Performance would have been one - if there were a comparison between gevent and asyncio.

Re: Making 1M requests with Python-aiohttp

#27

1,000,000 requests in 52 minutes is just 320 req/sec. Am I missing something? What's so amazing about this? I just deployed some production feed that serves at 1955 requests/second on a cheap VPS in freaking PHP, one of the slowest languages out there.

I don't care for PHP as much as the next guy, but it's usually in the top 25 of the web framework benchmark (most of the other top langs are Java, Go and C++): https://www.techempower.com/benchmarks/

Just curious, what's up with this Ur language at both position 1 and 4? Never heard of it, and probably not experienced enough to make sense of it, but how is it that a language that doesn't even have a full official tutorial to its name beat out java, C++ and Go in those rankings by a factor of >2?

I'm genuinely curious.

Re: Making 1M requests with Python-aiohttp

#28
post #8

Earlier quoted context omitted.

I guess I don't know how JS was any more "designed for" async than python was.

From https://developer.mozilla.org/en-US/docs/Web/JavaScript/Even... : JavaScript has a concurrency model based on an "event loop". This model is quite different than the model in other languages like C or Java. ... A very interesting property of the event loop model is that JavaScript, unlike a lot of other languages, never blocks. Handling I/O is typically performed via events and callbacks, so when the application…

I think that quoted statements are not very good.

The other languages don't have a builtin concurrency model. For C, Java and others event loop libraries and applications that are built on top of them can be found (nginx, netty, ...). As well as there are libraries that build on top of synchronous IO.

The eventloop was also not tied to Javascript in the older standards. Only the introduction of Promises and other stuff required the existence of an eventloop in order to define when continuations should run.

"The event loop model never blocks" is also only true as long as you (and all the libraries that you use) do not block it. There is no automatic "does not block" guarantee.

Re: Making 1M requests with Python-aiohttp

#29
post #22

Earlier quoted context omitted.

Because it's trivial. I would be interested in anything doing 10,000+ req/sec on a cheap VPS. 320 is nothing. People achieve 2 million requests/second with C++ on EC2: https://medium.com/swlh/starting-a-tech-startup-with-c-6b5d5...

Absolutely excellent article. I always keep C++ at the back of my mind in case I need it some day, so I think the list of libraries they used will be useful for me in the future. Thanks.

If you want to write fast C++ Web services I recommend a look at Seastar: http://www.seastar-project.org/
Post reply on HN