31–40 of 94 posts
Earlier quoted context omitted.
How can I use it with SQLAlchemy instead of psycopg2?
SQLAlchemy ORM is not async, so there is no way currently to use any async dialect for it. However, it is possible to write an SQLAlchemy Core adapter for asyncpg.
Earlier quoted context omitted.
SQLAlchemy ORM is not async, so there is no way currently to use any async dialect for it. However, it is possible to write an SQLAlchemy Core adapter for asyncpg.
It might even be easier than writing an adapter if you just use SQLAlchemy Core's DSL as the generative mechanism for a SQL statement that is sent to asyncpg.
asyncpg was developed by @redcrowbar and me. Ask us anything ;)
Wonder how this (Erlang driver) would stack up in the benchmarks (devel branch): https://github.com/epgsql/epgsql
http://techspot.zzzeek.org/2015/02/15/asynchronous-python-an...
Earlier quoted context omitted.
I'd love to read the docs for the protocol itself, is it planned? Do you have a link explaining it otherwise? Great work!
The protocol is documented in the official PostgreSQL documentation: https://www.postgresql.org/docs/current/static/protocol.html
Thanks for the link though, I'll have a look
Does this library address any of the issues/concerns that Mike Bayer discussed in this earlier blog post? http://techspot.zzzeek.org/2015/02/15/asynchronous-python-an...
> asyncpg extensively uses PostgreSQL prepared statements. This is an important optimization feature, as it allows to avoid repeated parsing, analysis, and planning of queries. Additionally, asyncpg caches the data I/O pipeline for each prepared statement.
it would be great if psycopg2 did this, at least within the context of executemany(). You don't need async to use prepared statements. But even if they did this, it wouldn't be this amazing speed-everywhere kind of thing, just for when you do a bulk INSERT/UPDATE/DELETE. The prepared statement is linked to a cursor and a statement handle so it's not like you can have hundreds of these just queued up in a cache without a great deal of complexity.
The TL;DR; for my blog post you mention is that async itself usually has little to do with speed when talking to a database, because the database is typically on the same network as your application, you're using typically no more than a few dozen connections per process (postgresql uses a process per connection, so using thousands of connections isn't feasible anyway), so the overhead of async and especially Python 3's async IO slows things down considerably without offering any opportunity to regain that speed. The GIL doesn't block on IO so threads work just fine, and most applications are CRUD / transactional applications which means you can't spawn off several dozen queries within a transaction at the same time anyway, you need to wait for each one to complete serially regardless.
I'd be very wary of a new PG driver especially one that implements the wire protocol itself. pg8000 does this, and also does prepared statements, and while it doesn't have the speed approach here, it also has serious problems supporting all the features and datatypes that psycopg2 does. It's been around for years and reliably breaks on many Postgresql features (and Postgresql introduces new features like a firehose). Writing and maintaining a driver for Postgresql is not an easy task.