Authentication with Perspective Broker

Motivation

In the examples shown in Using Perspective Broker there were some problems. You had to trust the user when they said their name was bob: no passwords or anything. If you wanted a direct-send one-to-one message feature, you might have implemented it by handing a User reference directly off to another User. (so they could invoke .remote_sendMessage() on the receiving User): but that lets them do anything else to that user too, things that should probably be restricted to the owner user, like .remote_joinGroup() or .remote_quit().

And there were probably places where the easiest implementation was to have the client send a message that included their own name as an argument. Sending a message to the group could just be:

class Group(pb.Referenceable):

# ...

  def remote_sendMessage(self, from_user, message):
    for user in self.users:
      user.callRemote("sendMessage", "[%s]: %s" % (from_user, message))

But obviously this lets users spoof each other: there's no reason that Alice couldn't do:

remotegroup.callRemote("sendMessage", "bob", "i like pork")

much to the horror of Bob's vegetarian friends.

(In general, learn to get suspicious if you see groupName or userName in the argument list of a remotely-invokable method).

You could fix this by adding more classes (with fewer remotely-invokable methods), and making sure that the reference you give to Alice won't let her pretend to be anybody else. You'd probably give Alice her own object, with her name buried inside:

class User(pb.Referenceable):
  def __init__(self, name):
    self.name = name
  def remote_sendMessage(self, group, message):
    g = findgroup(group)
    for user in g.users:
      user.callRemote("sendMessage", "[%s]: %s" % (self.name, message))

This improves matters because, as long as Alice only has a reference to this object and nobody else's, she can't cause a different self.name to get used. Of course, you have to make sure that you don't give her a reference to the wrong object.

Third party references (there aren't any)

Note that the reference that the server gives to a client is only useable by that one client: if they try to hand it off to a third party, they'll get an exception (XXX: which? looks like an assert in pb.py:290 RemoteReference.jellyFor). This helps somewhat: only the client you gave the reference to can cause any damage with it. Of course, the client might be a brainless zombie, simply doing anything some third party wants. When it's not proxying callRemote invocations, it's probably terrorizing the living and searching out human brains for sustenance. In short, if you don't trust them, don't give them that reference.

Also note that the design of the serialization mechanism (implemented in twisted.spread.jelly: pb, jelly, spread.. get it? Also look for banana and marmalade. What other networking framework can claim API names based on sandwich ingredients?) makes it impossible for the client to obtain a reference that they weren't explicitly given. References passed over the wire are given id numbers and recorded in a per-connection dictionary. If you didn't give them the reference, the id number won't be in the dict, and no amount of id guessing by a malicious client will give them anything else. The dict goes away when the connection is dropped, limiting further the scope of those references.

Of course, everything you've ever given them over that connection can come back to you. If expect the client to invoke your method with some object A that you sent to them earlier, and instead they send you object B (that you also sent to them earlier), and you don't check it somehow, then you've just opened up a security hole. A better design is to keep such objects in a dictionary on the server side, and have the client send you an index string instead. Doing it that way makes it obvious that they can send you anything they want, and improves the chances that you'll remember to implement the right checks.

But now she could sneak into another group. So you might have to have an object per-group-per-user:

class UserGroup(pb.Referenceable):
  def __init__(self, group, user):
    self.group = group
    self.user = user
  def remote_sendMessage(self, message):
    name = self.user.name
    for user in self.group.users:
      user.callRemote("sendMessage", "[%s]: %s" % (name, message))

But that means more code, and more code is bad, especially when it's a common problem (everybody designs with security in mind, right? Right??).

So we have a security problem. We need a way to ask for and verify a password, so we know that Bob is really Bob and not Alice wearing her Hi, my name is Bob t-shirt. And it would make the code cleaner (i.e.: fewer classes) if some methods could know reliably who is calling them.

A sample application

As a framework for this chapter, we'll be referring to a hypothetical game implemented by several programs using the Twisted framework. This game will have multiple players, where users log in using their client programs, and there is a server, and users can do some things but not othersThere actually exists such a thing. It's called twisted.reality, and was the whole reason Twisted was created. I haven't played it yet: I'm too afraid..

The players make moves in this game by invoking remote methods on objects that live in the server. The clients can't really be relied upon to tell the server who they are with each move they make: they might get it wrong, or (horrors!) lie to mess up the other player.

Let's simplify it to a server-based game of Go (if that can be considered simple). Go has two players, white and black, who take turns placing stones of their own color at the intersections of a 19x19 grid. If we represent the game and board as an object in the server called Game, then the players might interact with it using something like this:

class Game(pb.Referenceable):
  def remote_getBoard(self):
    return self.board # a dict, with the state of the board
  def remote_move(self, playerName, x, y):
    self.board[x,y] = playerName

But Wait, you say, yes that method takes a playerName, which means they could cheat and move for the other player. So instead, do this:

class Game(pb.Referenceable):
  def remote_getBoard(self):
    return self.board # a dict, with the state of the board
  def move(self, playerName, x, y):
    self.board[x,y] = playerName

and move the responsibility (and capability) for calling Game.move() out to a different class. That class is a pb.Perspective.

Perspectives

pb.Perspective (and some related classes: Identity, Authorizer, and Service) is a layer on top of the basic PB system that handles username/password checking. The basic idea is that there is a separate Perspective object (probably a subclass you've created) for each userActually there is a perspective per user*service, but we'll get into that later, and only the authorized user gets a remote reference to that Perspective object. You can store whatever permissions or capabilities the user possesses in that object, and then use them when the user invokes a remote method. You give the user access to the Perspective object instead of the objects that do the real work.

Your code can then look like this:

class Game:
  def getBoard(self):
    return self.board # a dict, with the state of the board
  def move(self, playerName, x, y):
    self.board[x,y] = playerName

class PlayerPerspective(pb.Perspective):
  def __init__(self, playerName, game):
    self.playerName = playerName
    self.game = game
  def perspective_move(self, x, y):
    self.game.move(self.playerName, x, y)
  def perspective_getBoard(self):
    return self.game.getBoard()

The code on the server side creates the PlayerPerspective object, giving it the right playerName and a reference to the Game object. The remote player doesn't get a reference to the Game object, only their own PlayerPerspective, so they don't have an opportunity to lie about their name: it comes from the .playerName attribute, not an argument of their remote method call.

Here is a brief example of using a Perspective. Most of the support code is magic for now: we'll explain it later.

This example has more support code than you'd actually need. If you only have one Service, then there's probably a one-to-one relationship between your Identities and your Perspectives. If that's the case, you can use a utility method called Perspective.makeIdentity() instead of creating the perspectives and identities in separate steps. This is shorter, but hides some of the details that are useful here to explain what's going on. Again, this will make more sense later.

pb5server.py pb5client.py

Note that once this example has done the method call, you'll have to terminate both ends yourself. Also note that the Perspective's .attached() and .detached() methods are run when the client connects and disconnects. The base class implementations of these methods just prints a message.

Ok, so that wasn't really very exciting. It doesn't accomplish much more than the first PB example, and used a lot more code to do it. Let's try it again with two users this time, each with their own Perspective. We also override .attached() and .detached(), just to see how they are called.

The Perspective object is usually expected to outlast the user's connection to it: it is nominally created some time before the user connects, and survives after they disconnect. .attached() and .detached() are invoked to let the Perspective know when the user has connected and disconnected.

When the client runs pb.connect to establish the connection, they can provide it with an optional client argument (which must be a pb.Referenceable object). If they do, then a reference to that object will be handed to the server-side Perspective's .attached method, in the clientref argument.

The server-side Perspective can use it to invoke remote methods on something in the client, so that the client doesn't always have to drive the interaction. In a chat server, the client object would be the one to which display text messages were sent. In a game, this would provide a way to tell the clients that someone has made a move, so they can update their game boards. To actually use it, you'd probably want to subclass Perspective and change the .attached method to stash the clientref somewhere, because the default implementation just drops it.

.attached() also receives a reference to the Identity object that represents the user. (The user has proved, by using a password of some sort, that they are that Identity, and then they can access any service/perspective on the Identity's keyring). The method can use that reference to extract more information about the user.

In addition, .attached() has the opportunity to return a different Perspective, if it so chooses. You could have all users initially access the same Perspective, but then as they connect (and .attached() gets called), give them unique Perspectives based upon their individual Identities. The client will get a reference to whatever .attached() returns, so the default case is to 'return self'.

Finally, when the client goes away (i.e., the network connection has been closed), .detached() will be called. The Perspective can use this to mark the user as having gone away: this may mean that outgoing messages should be queued in the Perspective until they reconnect, or callers should be given an error message because they messages cannot be delivered, etc. It can also be used to terminate or suspend any sessions the user was participating in. detached is called with the same 'clientref' and Identity objects that were given to the original 'attached' call. It will be invoked on the Perspective object that was returned by .attached().

pb6server.py pb6client1.py pb6client2.py

While pb6server.py is running, try starting pb6client1, then pb6client2. Compare the argument passed by the .callRemote() in each client. You can see how each client logs into a different Perspective.

Class Overview

Now that we've seen some of the motivation behind the Perspective class, let's start to de-mystify some of the parts labeled magic in pb6server.py. Here are the major classes involved:

You've already seen Application. It holds the program-wide settings, like which uid/gid it should run under, and contains a list of ports that it should listen on (with a Factory for each one to create Protocol objects). When used for PB, we put a pb.BrokerFactory on the port. The Application also holds a list of Services.

A Service is, well, a service. A web server would be a Service, as would a chat server, or any other kind of server you might choose to run. What's the difference between a Service and an Application? You can have multiple Services in a single Application: perhaps both a web-based chat service and an IM server in the same program, that let you exchange messages between the two. Or your program might provide different kinds of interfaces to different classes of users: administrators could get one Service, while mere end-users get a less-powerful Service.

Note that the Service is a server of some sort, but that doesn't mean there's a one-to-one relationship between the Service and the TCP port that's being listened to. In theory, several different Services can hang off the same TCP port. Look at the MultiService class for details.

The Service is reponsible for providing Perspective objects. More on that later.

The Authorizer is a class that provides Identity objects. The abstract base class is twisted.cred.authorizer.Authorizer, and for simple purposes you can just use DefaultAuthorizer, which is a subclass that stores pre-generated Identities in a simple dict (indexed by username). The Authorizer's purpose in life is to implement the .getIdentityRequest() method, which takes a user name and (eventually) returns the corresponding Identity object.

Each Identity object represents a single user, with a username and a password of some sort. Its job is to talk to the as-yet-anonymous remote user and verify that they really are who they claim to be. The default twisted.cred.authorizer.Identity class implements MD5-hashed challenge-response password authorization, much like the HTTP MD5-Authentication method: the server sends a random challenge string, the client concatenates a hash of their password with the challenge string, and sends back a hash of the result. At this point the client is said to be authorized for access to that Identity, and they are given a remote reference to the Identity (actually a wrapper around it), giving them all the privileges of that Identity.

Those privileges are limited to requesting Perspectives. The Identity object also has a keyring, which is a list of (serviceName, perspectiveName) pairs that the corresponding authorized user is allowed to access. Once the user has been authenticated, the Identity's job is to implement .requestPerspectiveForKey(), which it does by verifying the key exists on the keyring, then asking the matching Service to do .getPerspectiveForIdentity().

Finally, the Perspective is the subclass of pb.Perspective that implements whatever perspective_* methods you wish to expose to an authenticated remote user. It also implements .attached() and .detached(), which are run when the user connects (actually when they finish the authentication sequence) or disconnects. Each Perspective has a name, which is scoped to the Service which owns the Perspective.

Class Responsibilities

Now that we've gone over the classes and objects involved, let's look at the specific responsibilities of each. Most of these classes are on the hook to implement just one or two particular methods, and the rest of the class is just support code (or the main method has been broken up for ease of subclassing). This section indicates what those main methods are and when they get called.

Authorizer

The Authorizer has to provide Identity objects (requested by name) by implementing .getIdentityRequest(). The DefaultAuthorizer class just looks up the name in a dict called self.identities, so when you use it, you have to make the Identities ahead of time (using i = auth.createIdentity()) and store them in that dict (by handing them to auth.addIdentity(i)).

However, you can make a subclass of Authorizer with a .getIdentityRequest method that behaves differently: your version could look in /etc/passwd, or do an SQL database lookupSee twisted.enterprise.dbcred for a module that does exactly that., or create new Identities for anyone that asks (with a really secret password like '1234' that the user will probably never change, even if you ask them to). The Identities could be created by your server at startup time and stored in a dict, or they could be pickled and stored in a file until needed (in which case .getIdentityRequest() would use the username to find a file, unpickle the contents, and return the resulting Identity object), or created brand-new based upon whatever data you want. Any function that returns a Deferred (that will eventually get called back with the Identity object) can be used here.

For static Identities that are available right away, the Deferred's callback() method is called right away. This is why the interface of .getIdentityRequest() specifies that its Deferred is returned unarmed, so that the caller has a chance to actually add a callback to it before the callback gets run. (XXX: check, I think armed/unarmed is an outdated concept)

Identity

The Identity object thus returned has two responsibilities. The first is to authenticate the user, because so far they are unverified: they have claimed to be somebody (by giving a username to the Authorizer), but have not yet proved that claim. It does this by implementing .verifyPassword, which is called by IdentityWrapper (described later) as part of the challenge-response sequence. If the password is valid, .verifyPassword should return a Deferred and run its callback. If the password is wrong, the Deferred should have the error-back run instead.

The second responsibility is to provide Perspective objects to users who are allowed to access them. The authenticated user gives a service name and a perspective name, and .requestPerspectiveForKey() is invoked to retrieve the given Perspective. The Identity is the one who decides which services/perspectives the user is allowed to access. Unless you override it in a subclass, the default implementation uses a simple dict called .keyring, which has keys that are (servicename, perspectivename) pairs. If the requested name pair is in the keyring, access is allowed, and the Identity will proceed to ask the Service to give back the specified Perspective to the user. .requestPerspectiveForKey() is required to return a Deferred, which will eventually be called back with a Perspective object, or error-backed with a Failure object if they were not allowed access.

XXX: explain perspective names being scoped to services better

You could subclass Identity to change the behavior of either of these, but chances are you won't bother. The only reason to change .verifyPassword() would be to replace it with some kind of public-key verification scheme, but that would require changes to pb.IdentityWrapper too, as well as significant changes on the client side. Any changes you might want to make to .requestPerspectiveForKey() are probably more appropriate to put in the Service's .getPerspectiveForIdentity method instead. The Identity simply passes all requests for Perspectives off to the Service.

The default Identity objects are created with a username and password, and a keyring of valid service/perspective name pairs. They are children of an Authorizer object. The best way to create them is to have the Authorizer do it for you, then fill in the details, by doing the following:

i = auth.createIdentity("username")
i.setPassword("password")
i.addKeyByString("service", "perspective")
auth.addIdentity(i)

Service

The Service object's job is to provide Perspective instances, by implementing .getPerspectiveForIdentity(). This function takes a Perspective name, and is expected to return a Deferred which will (eventually) be called back with an instance of Perspective (or a subclass).

The default implementation (in twisted.spread.pb.Service) retrieves static pre-generated Perspectives from a dict (indexed by perspective name), much like DefaultAuthorizer does with Identities. And like Authorizer, it is very useful to subclass pb.Service to change the way .getPerspectiveForIdentity() works: to create Perspectives out of persistent data or database lookups, to set extra attributes in the Perspective, etc.

When using the default implementation, you have to create the Perspectives at startup time. Each Service object has an attribute named .perspectiveClass, which helps it to create the Perspective objects for you. You do this by running p = svc.createPerspective("perspective_name").

You should use .createPerspective() rather than running the constructor of your Perspective-subclass by hand, because the Perspective object needs a pointer to its parent Service object, and the Service needs to have a list of all the Perspectives that it contains.

How that example worked

Ok, so that's what everything is supposed to do. Now you can walk through the previous example and see what was going on: we created a subclass called MyPerspective, made a DefaultAuthorizer and added it to the Application, created a Service and told it to make MyPerspectives, used .createPerspective() to build a few, for each one we made an Identity (with a username and password), and allowed that Identity to access a single MyPerspective by adding it to the keyring. We added the Identity objects to the Authorizer, and then glued the authorizer to the pb.BrokerFactory.

How did that last bit of magic glue work? I won't tell you here, because it isn't very useful to override it, but you effectively hang an Authorizer off of a TCP port. The combination of the object and methods exported by the pb.AuthRoot object works together with the code inside the pb.connect() function to implement both sides of the challenge-response sequence. When you (as the client) use pb.connect() to get to a given host/port, you end up talking to a single Authorizer. The username/password you give get matched against the Identities provided by that authorizer, and then the servicename/perspectivename you give are matched against the ones authorized by the Identity (in its .keyring attribute). You eventually get back a remote reference to a Perspective provided by the Service that you named.

Here is how the magic glue code works:

app.listenTCP(8800, pb.BrokerFactory(pb.AuthRoot(auth)))

pb.AuthRoot() provides objects that are subclassed from pb.Root, so as we saw in the first example, they can be served up by pb.BrokerFactory(). AuthRoot happens to use the .rootObject hook described earlier to serve up an AuthServ object, which wraps the Authorizer and offers a method called .remote_username, which is called by the client to declare which Identity it claims to be. That method starts the challenge-response sequence.

Code Walkthrough: pb.connect()

So, now that you've seen the complete sequence, it's time for a code walkthrough. This will give you a chance to see the places where you might write subclasses to implement different behaviors. We will look at what happens when pb6client1.py meets pb6server.py. We tune in just as the client has run the pb.connect() call.

The client-side code can be summarized by the following sequence of function calls, all implemented in twisted/spread/pb.py . pb.connect() calls getObjectAt() directly, after that each step is executed as a callback when the previous step completes.

 getObjectAt(host,port,timeout)
 logIn(): authServRef.callRemote('username', username)
 _cbLogInRespond(): challenger.callRemote('respond', f[challenge,password])
 _cbLogInResponded(): identity.callRemote('attach', servicename,
                                          perspectivename, client)
 usercallback(perspective)

The client does getObjectAt() to connect to the given host and port, and retrieve the object named root. On the server side, the BrokerFactory accepts the connection, asks the pb.AuthRoot object for its .rootObject(), getting an AuthServ object (containing both the authorizer and the Broker protocol object). It gives a remote reference to that AuthServ out to the client.

Now the client invokes the '.remote_username' method on that AuthServ. The AuthServ asks the Authorizer to .getIdentityRequest(): this retrieves (or creates) the Identity. When that finishes, it asks the Identity to create a random challenge (usually just a random string). The client is given back both the challenge and a reference to a new AuthChallenger object which will only accept a response that matches that exact challenge.

The client does its part of the MD5 challenge-response protocol and sends the response to the AuthChallenger's .remote_response() method. The AuthChallenger verifies the response: if it is valid then it gives back a reference to an IdentityWrapper, which contains an internal reference to the Identity that we now know matches the user at the other end of the connection.

The client then invokes the .remote_attach method on that IdentityWrapper, passing in a serviceName, perspectiveName, and remoteRef. The wrapper asks the Identity to get a perspective using identity.requestPerspectiveForKey, which does the is this user allowed to get this service/perspective check by looking at the tuples on its .keyring, and if that is allowed then it gets the Service (by giving serviceName to the authorizer), then asks the Service to provide the perspective (with svc.getPerspectiveForIdentity). The default Service will ignore the identity object and just look for Perspectives by perspectiveName. The Service looks up or creates the Perspective and returns it. The .remote_attach method runs the Perspective's .attached method (although there are some intermediate steps, in IdentityWrapper._attached, to make sure .detached will eventually be run, and the Perspective's .brokerAttached method is executed to give it a chance to return some other Perspective instead). Finally a remote reference to the Perspective is returned to the client.

The client gives the Perspective reference to the callback that was attached to the Deferred that pb.connect() returned, which brings us back up to the code visible in pb6client1.py.

A Larger Example

Now it's time to look more closely at the Go server described before.

To simplify the example, we will build a server that handles just a single game. There are a variety of players who can participate in the game, named Alice, Bob, etc (the usual suspects). Two of them log in, choose sides, and begin to make moves.

We assume that the rules of the game are encapsulated into a GoGame object, so we can focus on the code that handles the remote players.

XXX: finish this section

That's the end of the tour. If you have any questions, the folks at the welcome office will be more than happy to help. Don't forget to stop at the gift store on your way out, and have a really nice day. Buh-bye now!