Twisted is an asynchronous networking framework, but most
database API implementations unfortunately have blocking
interfaces -- for this reason, twisted.enterprise.adbapi
was created. It is
a non-blocking interface to the standardized DB-API 2.0 API,
which allows you to access a number of different RDBMSes.
Twisted is an asynchronous framework. This means standard database modules cannot be used directly, as they typically work something like:
# Create connection... db = dbmodule.connect('mydb', 'andrew', 'password') # ...which blocks for an unknown amount of time # Create a cursor cursor = db.cursor() # Do a query... resultset = cursor.query('SELECT * FROM table WHERE ...') # ...which could take a long time, perhaps even minutes.
Those delays are unacceptable when using an asynchronous
framework such as Twisted. For this reason, twisted provides
twisted.enterprise.adbapi
, an
asynchronous wrapper for any
DB-API 2.0-compliant module. It is currently best tested
with the pyPgSQL
module for PostgreSQL.
enterprise.adbapi
will do
blocking
database operations in seperate threads, which trigger
callbacks in the originating thread when they complete. In the
meantime, the original thread can continue doing normal work,
like servicing other requests.
Rather than creating a database connection directly, use the
adbapi.ConnectionPool
class to manage
a connections for you. This allows enterprise.adbapi
to use multiple
connections, one per thread. This is easy:
# Using the "dbmodule" from the previous example, create a ConnectionPool from twisted.enterprise import adbapi dbpool = adbapi.ConnectionPool("dbmodule", 'mydb', 'andrew', 'password')
Things to note about doing this:
adbapi.ConnectionPool
's constructor.adbapi.ConnectionPool
's constructor.
Keyword parameters work as well.cp_min
and
cp_max
. The default minimum and maximum values
are 3 and 5.So, now you need to be able to dispatch queries to your
ConnectionPool. We do this by subclassing adbapi.Augmentation
. Here's an example:
class AgeDatabase(adbapi.Augmentation): """A simple example that can retrieve an age from the database""" def getAge(self, name): # Define the query sql = """SELECT Age FROM People WHERE name = ?""" # Run the query, and return a Deferred to the caller to add # callbacks to. return self.runQuery(sql, name) def gotAge(resultlist, name): """Callback for handling the result of the query""" age = resultlist[0][0] # First field of first record print "%s is %d years old" % (name, age) db = AgeDatabase(dbpool) # These will *not* block. Hooray! db.getAge("Andrew").addCallbacks(gotAge, db.operationError, callbackArgs=("Andrew",)) db.getAge("Glyph").addCallbacks(gotAge, db.operationError, callbackArgs=("Glyph",)) # Of course, nothing will happen until the reactor is started from twisted.internet import reactor reactor.run()
This is straightforward, except perhaps for the return value
of getAge
. It returns a twisted.internet.defer.Deferred
, which allows
arbitrary callbacks to be called upon completion (or upon
failure). More documentation on Deferred is available here.
Also worth noting is that this example assumes that dbmodule
uses the qmarks
paramstyle (see the DB-API specification). If
your dbmodule uses a different paramstyle (e.g. pyformat) then
use that. Twisted doesn't attempt to offer any sort of magic
paramater munging -- runQuery(query,
params, ...)
maps directly onto cursor.execute(query, params, ...)
.
That's all you need to know to use a database from within Twisted. You probably should read the adbapi module's documentation to get an idea of the other functions it has, but hopefully this document presents the core ideas.