Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Making 1M requests with Python-aiohttp

pawelmhm.github.io

Post reply on HN
Print view View original

41–50 of 84 posts

Re: Making 1M requests with Python-aiohttp

#41

Post by rdtsc »

sandGorgon wrote: post #26
riyadparvez wrote: post #21

Earlier quoted context omitted.

Is there anything inherent to Python3 that is slower than Python2? Or is it just some of the performant packages still have not been ported to Python3?

i keep looking for a reason to switch to python 3 and cant find one. Plus if I want to use the cool stuff in Pypy.. then I better not ! overall - very less reason to consider Py3 at all. Performance would have been one - if there were a comparison between gevent and asyncio.

I see you've been downvoted. Interesting, you said that you haven't found a reason to switch and someone looked at it and thought "How dare you not find a reason to switch, here let's teach you a lesson".

But in large I agree. 3 hasn't provided enought of a carrot and 2 hasn't been enough of a pain for many people to want to switch. Especially when it comes to existing stable code bases. For new development, yes, many can and should pick Python 3. But if say Python 3 brought even a 20% speed improvement overwall, the move would have been a lot faster.

I find people are ok with accepting some breakages due to re-writes if either existing stuff is very broken, or new stuff is so much better.

Re: Making 1M requests with Python-aiohttp

#42

Post by [deleted] »

sago wrote: post #38
mrweasel wrote: post #34

Earlier quoted context omitted.

>i keep looking for a reason to switch to python 3 and cant find one. Unicode? Not having to deal with encoding all over the place has been well worth switch to Python 3. If performance is a a huge issue, I honestly don't know why you would stay on Python (regardless of version) I wouldn't want to switch back to Python 2.7 is I can avoid it. There's honestly no reason not to go with 3.4 or 3.5 at this point, unless y…

This is a superficially trivial bit of syntactic sugar, but an example of the way small tweaks can provide big impact. This: > do_something(*some_args, *some_more_args) is rocking my world right at the moment. That's a massive time saving feature I've been waiting for and worth the price of a 3.5 upgrade.

[deleted]

Re: Making 1M requests with Python-aiohttp

#43

Post by kenneth_reitz »

sago wrote: post #38
mrweasel wrote: post #34

Earlier quoted context omitted.

>i keep looking for a reason to switch to python 3 and cant find one. Unicode? Not having to deal with encoding all over the place has been well worth switch to Python 3. If performance is a a huge issue, I honestly don't know why you would stay on Python (regardless of version) I wouldn't want to switch back to Python 2.7 is I can avoid it. There's honestly no reason not to go with 3.4 or 3.5 at this point, unless y…

This is a superficially trivial bit of syntactic sugar, but an example of the way small tweaks can provide big impact. This: > do_something(*some_args, *some_more_args) is rocking my world right at the moment. That's a massive time saving feature I've been waiting for and worth the price of a 3.5 upgrade.

Oh cool, I didn't know about this!

Re: Making 1M requests with Python-aiohttp

#44

Post by haimez »

sandGorgon wrote: post #26
riyadparvez wrote: post #21

Earlier quoted context omitted.

Is there anything inherent to Python3 that is slower than Python2? Or is it just some of the performant packages still have not been ported to Python3?

i keep looking for a reason to switch to python 3 and cant find one. Plus if I want to use the cool stuff in Pypy.. then I better not ! overall - very less reason to consider Py3 at all. Performance would have been one - if there were a comparison between gevent and asyncio.

It seems ridiculous (to me, someone that is superficially aware of the python ecosystem but not a real user) that they haven't tried to find a way to incentivize people to switch with a real carrot. For example: removing the GIL.

Running out the clock on waiting for library support to become mostly python 3 is inevitable, but so far has been slower than molasses.

Re: Making 1M requests with Python-aiohttp

#47

Post by sandGorgon »

rdtsc wrote: post #41
sandGorgon wrote: post #26

Earlier quoted context omitted.

i keep looking for a reason to switch to python 3 and cant find one. Plus if I want to use the cool stuff in Pypy.. then I better not ! overall - very less reason to consider Py3 at all. Performance would have been one - if there were a comparison between gevent and asyncio.

I see you've been downvoted. Interesting, you said that you haven't found a reason to switch and someone looked at it and thought "How dare you not find a reason to switch, here let's teach you a lesson". But in large I agree. 3 hasn't provided enought of a carrot and 2 hasn't been enough of a pain for many people to want to switch. Especially when it comes to existing stable code bases. For new development, yes, man…

but Python 3 is not really an upgrade is it ? it is a very different language and most people who are pushing (downvoting?) for Python 3 dont seem to understand that.

I have zero problems with Python 3 per se - but when the vast majority of the ecosystem is on Py2 and there is no difference in performance... then I see no reason to consider any breakages.

Re: Making 1M requests with Python-aiohttp

#48

Post by jorge_leria »

imaginenore wrote: post #13
aaossa wrote: post #10

Earlier quoted context omitted.

Why you say is not amazing? Honestly curious here :)

Because it's trivial. I would be interested in anything doing 10,000+ req/sec on a cheap VPS. 320 is nothing. People achieve 2 million requests/second with C++ on EC2: https://medium.com/swlh/starting-a-tech-startup-with-c-6b5d5...

The article is not about serving, but about consuming. Not the same beast.

Re: Making 1M requests with Python-aiohttp

#49

Post by terom »

Re the EADDRNOTAVAIL from socket.connect(),

If you're connecting to 127.0.0.1:8080, then each connection from 127.0.0.1 is going to be assigned an ephemeral TCP source port. There are only a finite number of such ports available, on the order of ~30-50k, which limits the number of connections from a single address to a specific endpoint.

If you're doing 100k TCP connections with 1k concurrent conections, it's feasible that you'll run into those limits, with TCP connections hanging around in some TIME_WAIT state after close().

Not that this would be a documented errno for connect(), but it's the interpretation that makes sense..

http://www.toptip.ca/2010/02/linux-eaddrnotavail-address-not... http://lxr.free-electrons.com/source/net/ipv4/inet_hashtable...

Re: Making 1M requests with Python-aiohttp

#50

Post by jc4p »

jorge_leria wrote: post #46
velox_io wrote: post #45

The 1 million in the title is misleading (1M per hour is nothing to write home about, only 278/sec). There are frameworks that are able hit 1M per minute plus (16,666/sec).

1M per minute it is something. Could you name those frameworks?

Elixir is the name I see thrown around the most when it comes to stuff like this: http://www.phoenixframework.org/blog/the-road-to-2-million-w...
Post reply on HN