Live data from Hacker News

Making 1M requests with Python-aiohttp

pawelmhm.github.io

71–80 of 84 posts

Re: Making 1M requests with Python-aiohttp

#71
post #11

First off, awesome to see more benchmarks (even if it's just personal experimentation) for synchronous vs asyncio performance. I think the real argument for asyncio right now is that it makes it very easy for you to write extremely efficient code, even for hobbyist projects. Even though your experiment is only handling 320 req/s, that you were able to do that so quickly and with very, very little optimization is, I t…

Thanks for feedback. > You would probably handle more requests if you changed that -- I would do the file access in a run_in_executor with a max executor workers of 1000. This is really good point. I'm going to check this and edit post adding this information there. > Also, the placement of your semaphore acquisition doesn't make any sense to me. I would create a dedicated coroutine like this: looking into my semapho…

As per my comment further up, it might be interesting to spin up a handful of listening processes (eg: 127.0.0.1 through 127.0.0.10), and a handful of clients; and have the clients pick one at random, or something like that. Not so much for "real world testing", but just as an exercise to see if one can press the system to other limits than open connections/address pairs?

Re: Making 1M requests with Python-aiohttp

#72
post #64

Earlier quoted context omitted.

A few reasons which made me switch: - Python 2 will be EOL in 2020, that's four years - vastly improved Unicode support - a number new libraries are Python 3 only - asyncio and the new async syntax - exception chaining (!) - type annotations - lots of improvements all over the place

Python 2 will not be EOL in 4 years. Not with the billions of lines of code out there. If the Python foundation dares to do this, it will create a fork. Probably even funded by Dropbox, Google and the like. I wont dispute you on any other aspects - except two. have you tried using gevent versus asyncio ? gevent is running in production at several of the largest API services in the world. Asyncio is not yet deployed a…

The Python foundation won't budge on the 2020 EOL date, for sure. RedHat will support it until RHEL 7 is EOL (~2027), but that's security/critical fixes only - while Python 3 gets all the new features and development efforts.

At some point, the opportunity cost of staying with Python 2.7 is higher than the one-time effort of porting everything to Python 3, so companies will move. Especially the likes of Google and Dropbox.

gevent vs. asyncio - sure, gevent is more mature. But asyncio is undoubtedly the better/nicer API. asyncio's explicit await syntax is much nicer than gevent's implicit monkey-patching, which makes it harder to reason about the code.

Tensorflow isn't really a new library, it was only recently open sourced.

As for Python 3-only libraries:

- https://pypi.python.org/pypi?:action=browse&show=all&c=595

- https://github.com/aio-libs

Also note that Django will drop Python 2 support in time with the Python 2 EOL: https://www.djangoproject.com/weblog/2015/jun/25/roadmap/

Also, many new Python-based open source projects are Python 3 only.

Re: Making 1M requests with Python-aiohttp

#73
post #49

Re the EADDRNOTAVAIL from socket.connect(), If you're connecting to 127.0.0.1:8080, then each connection from 127.0.0.1 is going to be assigned an ephemeral TCP source port. There are only a finite number of such ports available, on the order of ~30-50k, which limits the number of connections from a single address to a specific endpoint. If you're doing 100k TCP connections with 1k concurrent conections, it's feasibl…

Localhost goes from 127.0.0.1 through 127.255.255.254. By binding each connection to a random IP in that range [1], one could get a better mileage.

[1]: https://idea.popcount.org/2014-04-03-bind-before-connect/

Re: Making 1M requests with Python-aiohttp

#74

The 1 million in the title is misleading (1M per hour is nothing to write home about, only 278/sec). There are frameworks that are able hit 1M per minute plus (16,666/sec).

Even 1M per minute is rather pathetic. The game is around multiple millions per second:

https://www.techempower.com/benchmarks/#section=data-r12&hw=...

Re: Making 1M requests with Python-aiohttp

#75
post #62
post #15

Earlier quoted context omitted.

>>JavaScript, unlike a lot of other languages, never blocks I think that's a little strong. It's more like "The group controlling Javascript has mostly tried to discourage introduction of things that block". You can, for example, do a blocking XMLHttpRequest. It's deprecated, but possible. https://jsfiddle.net/923d5sda/

You can do blocking everything. A 10.000 repetitions for look while block the whole interpreter for its duration. Any JSON parsing does the same. Processing strings. Doing math work. ...

Guess I should have said "blocking I/O"? I thought it was a given that a single threaded language wouldn't magically inject some kind of concurrency around tight loops or CPU intensive tasks. Node.js people don't really think that sort of thing is "non blocking", do they?

Re: Making 1M requests with Python-aiohttp

#76
post #54
post #49

Re the EADDRNOTAVAIL from socket.connect(), If you're connecting to 127.0.0.1:8080, then each connection from 127.0.0.1 is going to be assigned an ephemeral TCP source port. There are only a finite number of such ports available, on the order of ~30-50k, which limits the number of connections from a single address to a specific endpoint. If you're doing 100k TCP connections with 1k concurrent conections, it's feasibl…

Generally its the upper 32k ports that are ephemeral, and if your churn more than that per minute in connections, you'll run into that TIME_WAIT issue. Hacky way to get around that is to enable tcp_tw_reuse which will let you reuse ports, but it can be risky if you get a SYN from the previous connection that happens to lineup with segment number of the current connection (which will close your connection). Shouldn't…

> Hacky way to get around that is to enable tcp_tw_reuse which will let you reuse ports, but it can be risky if you get a SYN from the previous connection that happens to lineup with segment number of the current connection (which will close your connection)

Actually Linux will fall back to using TCP timestamps to distinguish between different connections. Ironically people will disable timestamps too to "fix" other issues[1] which also break PAWS[2] and may cause the issue you describing.

[1] It can break with some NAT and some load balancers. Actually the way I learned about tcp_tw_reuse was when we plugged in a new load balancer. We tested everything worked fine, but as soon as we sent production traffic many connections took few seconds to complete. Took 2 weeks to find the cause and looking at packet dumps. Turns out that the issue was that the load balancer was set up in active-active configuration, so different connections had different timestamps. This caused Linux to get confused and ignore some packets. Turned out one of managers wanted to make everything performant and copied some sysctls (that included tcp_tw_reuse and tcp_tw_recycle) from Internet without much though. After restoring the setting everything worked flawlessly.

[2] https://en.wikipedia.org/wiki/Transmission_Control_Protocol#...

Re: Making 1M requests with Python-aiohttp

#77
IMO you should place all requests within a single ClientSession().

This will provide two benefits:

1. You won't need to use a semaphore. To limit connections you will need to create a TCPConnection() object with limit set to the limit you used in the semaphore and pass it to the ClientSession() and aiohttp will not make more connections than the limit set (default behavior is to have unlimited number of connections).

2. With single ClientSession(), aiohttp will make use of keep-alive (i.e. it will reuse same connections for next requests, but it will keep at most the limit of connections you set in TCPConnection() object).

This should improve performance further, and (given sane limit) it'll also solve issue with "Cannot assign requested address" error.

BTW: Even without limit set aiohttp will try to reduce number of connections open so it might still fix the connection error issue as long as individual requests don't take long. It's still good idea to set limit, just to be nice to the remote server.

Re: Making 1M requests with Python-aiohttp

#78
post #11

First off, awesome to see more benchmarks (even if it's just personal experimentation) for synchronous vs asyncio performance. I think the real argument for asyncio right now is that it makes it very easy for you to write extremely efficient code, even for hobbyist projects. Even though your experiment is only handling 320 req/s, that you were able to do that so quickly and with very, very little optimization is, I t…

Thanks for feedback. > You would probably handle more requests if you changed that -- I would do the file access in a run_in_executor with a max executor workers of 1000. This is really good point. I'm going to check this and edit post adding this information there. > Also, the placement of your semaphore acquisition doesn't make any sense to me. I would create a dedicated coroutine like this: looking into my semapho…

No problem, it's especially hard to find external feedback for side projects and experiments so I try to give it when I can.

> I assumed it works correctly because it fixed my "too many open files" exception

It works, so at the end of the day that's what matters. The client vs server question, from my perspective, ultimately comes down to a question of test realism; in a real-world deployment you couldn't limit connections with client-side code because there are multiple clients. That's what I mean by "it doesn't make sense given that the error is server-side".

> Can you clarify why you think my use of semaphore does not make sense and why your suggestion is better? What is the benefit of dedicated coroutine?

I'm saying that mostly, but not exclusively, from a division of concerns standpoint. You're acquiring the semaphore in a completely different context than you're releasing it. On the one hand, that's partly a programming style issue. On the other hand, it can also have some really important consequences: for example, it's actually the event loop itself that is releasing the semaphore for you when the task is done. Because of the way the event loop works, it's hard to say exactly when the semaphore will be released. You want to hold it for the absolute minimum time possible, since it's holding up execution of other connections in the loop. Putting it into a dedicated coroutine makes it clearer what's going on, makes it such that the acquirer and releaser of the semaphore are the same, and means you are definitely holding the semaphore for the minimum amount of time possible (since, again, execution flow will not leave any particular coroutine until you yield/await another). In general I would say that releasing the semaphore in a callback is significantly more fragile, and mildly to moderately less performant, than creating a dedicated coroutine to hold the semaphore and handle the request.

Does that all make sense?

> Either it does not fail at all, which seems unlikely, or it fails silently, which is more likely and is bad.

That's a fair statement, I think. As an aside, the print statement is slow, so keep that in mind. It might actually be faster to have a single memory-mapped file for the whole thing, and then just append the error and traceback to the file. The built-in traceback library can be very useful for that. That's also a bit more realistic, since obviously IRL you wouldn't be using a print statement to keep track of errors. On a similar note, because file access is so slow, you'd be best off figuring out some way to remove the part where the server accesses the disk once per connection entirely. On a real-world system you'd possibly use some kind of memory caching system to do that, especially if you're just reading files and not writing them. That allows you to use a little more memory (potentially as little as enough to have a single copy of the file in memory) to drastically improve performance.

Re: Making 1M requests with Python-aiohttp

#79
post #65

Earlier quoted context omitted.

I think over time watching this behavior like the downvotes you received, I've figured it out. The idea is that newer folks come into Python, many don't want to learn the dominant version in effort to focus on the future as they understand it. So 2 continuing to live is viewed as a threat to that investment. Even though the two aren't that different and shouldn't matter which one you use, that isn't a popular point t…

Nah. I'm a long-time Python developer (10 years+) and I moved everything over to Python 3 because there are so many advantages. This includes a number of massive internal code bases that I maintain at my day job. Porting is surprisingly easy nowadays, it used to hurt a lot more. Management is fine with development time spent on migrating to Python 3, since it's an investment in the future (Python 2 will be EOL in 202…

Sounds like you're a lone wolf at a smaller company. Different ballgame from the "longtime Python devs" that I'm talking about. I had an employer with a 500KLOC Python2 codebase, a billion dollar business on the line, and new features to deliver. That 2020 date is just a big political stunt. As was 2015 or it wouldn't have been a snap of the fingers to extend it. Code won't stop working in 2020 and security is largely handled by a webserver.

I use Python3 sometimes as well, but doesn't mean what I'm saying isn't true.

Re: Making 1M requests with Python-aiohttp

#80
post #75
post #62

Earlier quoted context omitted.

You can do blocking everything. A 10.000 repetitions for look while block the whole interpreter for its duration. Any JSON parsing does the same. Processing strings. Doing math work. ...

Guess I should have said "blocking I/O"? I thought it was a given that a single threaded language wouldn't magically inject some kind of concurrency around tight loops or CPU intensive tasks. Node.js people don't really think that sort of thing is "non blocking", do they?

>Guess I should have said "blocking I/O"? I thought it was a given that a single threaded language wouldn't magically inject some kind of concurrency around tight loops or CPU intensive tasks.

Well, Erlang (and Elixir) does just that -- it's preemptive, and implicitly yields under the covers even in loops.

>Node.js people don't really think that sort of thing is "non blocking", do they?

Judging from forum threads and blog posts, a lot of them do, especially web programmers not familiar with blocking and non-blocking that only know that "Node is webscale".

Post reply on HN