I had a chance to review the passbook programming guide from Apple and I will mention the salient features of the protocol here. The APIs provide allow for registering a device to receive push notifications for a pass, getting serial numbers for passes associated with a device, getting the latest version of a pass, unregistering a device and for logging errors. These are used by Passbook to communicate with the web server of the company that generates the passes. The application that creates the passes may also be made available for download by the company

In today's blog post, I want to continue the discussion on the developer portal for API programming. I mentioned that the REST based services could have code corresponding to the objects, in the parameters, requests and responses of the API that could be made available to download. When the data is xml or json, this was easily deserializable but things get quickly out of date adding to the developer frustrations especially when there is no other way to interpret why you are getting the 400 or 404s. Case sensitivity in the parameters or fields, This code that we enable to download can be easily detected as having changed and then updated unless the documentation is kept upto date. It's also common for model to be refactored into a separate assembly. So this may be already available.

There's actually some support out of box in the REST WebAPI framework to be able to populate this assembly of class definitions. Take for instance the Help Page sample generator in the Web API framework:
            Type type = ResolveType(api, controllerName, actionName, parameterNames, sampleDirection, out formatters);
            var samples = new Dictionary<MediaTypeHeaderValue, object>();

            // Use the samples provided directly for actions
            var actionSamples = GetAllActionSamples(controllerName, actionName, parameterNames, sampleDirection);
            foreach (var actionSample in actionSamples)
            {
                samples.Add(actionSample.Key.MediaType, WrapSampleIfString(actionSample.Value));
            }
Last but not the least, the error messages should go over and beyond to make it easy to use the APIs. This is because the APIs are not expected to be consumed by the non-programmer users so this is already a select audience. And the easier it is for the developer to debug and make progress on the application, the wider and easier the adoption of the APIs. This goes a long way to enable rapid development.
I also want to mention data quality as important in this post. Often a single field or type in the response is important for subsequent API calls. Therefore the data associated with it must be correct to pull up the responses in the subsequent calls. For an application development, its often a call sequence that enables a workflow. Hence this consideration will also help the developer. 

I want to compare developer or client registration page on several developer portals with the aim to show what makes it easier for developers. Let us say that there are companies that provide APIs for integration with their service such as Apple, UrbanAirShip etc. On the developer portal, there are usually steps marked 1,2 and 3 to get the developer from start to finish with the client. Now different companies may choose different groupings of their documentation but the workflow a developer expects is how to build the app, how to connect to the various services, and how to deploy or submit for public use. In all these there are usually read only documentation. And they need not require sign-in.  The signin is required only for the registration page where we issue apikeys or submit the application for approval. The registration to get the api keys and the submission are nearly the start and finish of the developer engagement. The documentation itself could be made public but this is a choice for the company. The main thing is that the developer may exchange information on the registration page such as redirect URI and apikeys that will matter to the application being developed and the easier we make it for the application to be developed with less time spent diagnosing the issues the better. I want to call out the issues we face when writing applications. First, there has to be a mention of a basic call sequence or sample code for the login page. This lets the developer try out the code the same way as it was used internally. Besides, it gives something to the developer to work with.  Second, there has to be categorization of API resources for the developer to pick and choose from. This categorization should reflect the capabilities of the API rather than the variations found in the API. Next in each such category, there has to be some API picked out with proper requests and responses. The parameters and responses often help expedite development because it shows how the calls are made. They also give an indication of the objects to be mapped on the client side. What would be ideal is if the classes or models corresponding to each API request and response were made available to developers to download. They need not be available as nuget feeds though that would be great but make it available  so that the developer doesn't have to guess the classes from the request or response. These assemblies  in the nuget packages don't have any logic just some classes with their object model for developers to serialized or seserialize their requests and responses. Next, the apikeys issued should be immediately usable instead if waiting for approval. If the apikeys do not immediately yield the responses as mentioned in the API documentation, this lets down the development enthusiasm. Lastly, there should be proper error messages in addition to error codes from each API so that the caller knows the corrective action to take. The error messages go a long way to facilitate rapid development 

In the previous post I summarized an implementation that almost exclusively relies on encrypting and decrypting tokens without having the need to keep tokens in a token table. That is great but it will require versioning and backward compatibility whenever the token definition changes. We can attempt to capture all required fields for encryption to build a token but its the protected resources that can require different privileges overtime. Therefore there may be variations to the scope parameter that we may not be able to anticipate. Besides, there might be revisions to the spec that we can't anticipate. If we have to do backward compatibility with the API, there's more than one artefact to take care of. Instead, we could keep this flexible with a hash for a token that we maintain attributes on.
 There's very few things we need to do in addition to what we discussed to have a token table. In this post, let us summarize the points for consideration with an implementation that does not encrypt/decrypt tokens but relies exclusively on looking up the hash of a token in a token table.
Here we will consider all improvements to the database and treat the security mechanism as equivalent to an API implementation that uses a database to store and lookup tokens.
First, we will need to populate tokens from all workflows. This means we don't leave out tokens from any token granting endpoint
Second, we look up the token as an action filter so that API calls will not need to take the call where validation fails.
Third, the token tables should not keep tokens that have expired.  A background periodic purge is good enough.
Fourth, the token tables need to have user and client ids as sequential integers in addition to the guids, if any. The guids may help correlate the token table with the user profiles elsewhere.
Fifth, RBAC access and row level security can be enhanced with the token table and scope parameter mapping. The scope can very simply translate to row level labels applied to different resources.
Sixth, some columns may need to be added to existing schema in other databases for this improved and enhanced security mechanism.
Seventh, the client and user id columns should be indexed for lookup of token based on client and user.
Eighth, special case client and user may need to be supported for scenarios such as where resources are accessed with guest access.
Ninth, Existing table upgrade should only involve addition of columns for security mapping
Tenth, the token hash may need to be random, hex or base64 encoded and should be indexed as well.
Lastly, client registrations should flow into corresponding client table.

To summarize, we have decided so far that
1) we can have a token table to store and lookup tokens that we issue OR we can generate tokens where the tokens bear user, client, time of issue and bearer status.
2) we can use encryption and decryption effectively to generate tokens. We could keep the encryption or decryption logic in the database.
3) Tokens that we generate will be tamperproof, cannot be faked and will have all the security information required to handle the requests.
4) Tokens can have fixed length strings for user, client, time of issue and bearer status, preferably integers
5) On decryption, the client Id in the token will be validated against the apikey of the caller to ensure that the caller is the authorized one.
6) The protected resources of the client should match the user information from the token.
7) These security validations happen as filters to the API calls so that the calls themselves don't have to implement any validation logic.
8) Audit trail of all token grants and calls can be made by intercepting traffic on the server side. URI logging may not work since bearer tokens may be passed in the request body.
9) Tokens are expected to be short lived such as for an hour so clients are forced to request again.
10) Token granting and validations should be performance. Choice of technology should not degrade performance. In the bigger picture,  the overall cost of token validation should not exceed one tenth the time taken in network delays and hops to reach the authorization server.
11) User and client information from token is independent from the userId and clientId in the API resource qualifier. The APIs could choose to shorten it
12) token expiration could be validated as well
13) token with bearer flag will skip client validation
14) token should include some kind of magic number from authorization server
15) Client registration could be handled as part of webUI.
16) Client registration information could keep track of redirect uri
17) Authorization code could be distinguished from the token by a flag.
18) Code to token translation can be permitted if code is validated just the same way as tokens were decrypted and validated
19) Token granting can be based on any workflow but the token itself does not need to have any of the information about the grant type. Special  userId and clientId will be required for cases when userId is missing or the token is permitted by different client types.
20) Tokens once issued is generally not expected to be issued again.
21) By tokens we are referring to the access_token parameter in the above and its length should preferably be within 50.
22) New workflows could be created or existing workflows can be used for generating or using the same token.

Integer vs GUID columns for identifying users and clients in OAuth
When considering whether to have Integer Identity columns or GUID identity columns or both, I wanted to reflect on what others have been discussing in general about integer versus guid columns.
First integer columns have an upper limit. If you are likely to exceed, pick GUID
Second integer columns are sequential and indexed. If all your data is internal and no other system is reading or writing to it based on the Id, then integer is a good choice. By
sequential, you mean that inserts are typically happening at the end whereas GUID inserts can happen. Sequential also means indexed and therefore improved performance.
If data is to be merged or replicated or synchronized with other systems, consider GUID for randomness and uniqueness.
Third if you are giving an ID out, give out GUIDs. If you are keeping it internal use integer
Fourth, GUID makes ID based security attacks more difficult
Fifth, if all things are equal, consider Id for readability.

Next I want to discuss obfuscation of userId and clientId. Both integers for userId and clientId could yield the same encrypted string over and over again If that becomes the substrings in each token such that they repeat between tokens, it will become easier to crack the userId and clientId. There are many obfuscation techniques but the ability to decrypt back the original string should not be lost otherwise the tokens will get compromised. Given these the hhmmss is a good addition to the original string so that the encrypted string varies each time. Just that the granularity of the time for tokens to be issued is now 1 second. Given that there are 3600 seconds where each second a million users might use the same client to login, the performance required is almost a single login per micro second. This requires performance comparision between decryption or database calls. Besides the hhmmss could be guessed unless a company specific magic number is also added to the remaining portion of the integer in which the hhmmss is stored.
Lastly I want to discuss the tradeoff between using a token table versus token encryption or decryption from a performance standpoint. The database calls can scale. Encryption and decryption will require space and time but can be done in memory. A token table is also not expected to grow to billions of records. Tokens will be inserted sequentially and in a forward only manner while cleanup happens on the other end where tokens expire. The cost to lookup can be minimized with the help of indexes based on clientId and userId. Many attributes can be added to the token in a token table versus the candidate for encryption where space is a premium. So a token table certainly has a lot of advantages and follows well known model for usage.

I mentioned in my previous post, the token should capture additional information for the token such as whether it's a bearer token or not. Today I want to discuss the other attributes of the token such as scope and state that may also need to be applied during policy resolution and a way to look them up. We mentioned that the scope and the state are not used for validating the client so they need not be part of what makes the token. However, applying scope to users protected resources is also relevant to security policy implementation. By default the scope may not be specified or set and the presence of a valid token could imply access to all of user's protected resources. In such cases, there is not scope resolution required. In the cases where scopes are specified and enforced, we need a mechanism that facilitates APIs to be called due to the presence of a token but fails when the required resources are outside of the scope of the token. we will discuss the scope storage/lookup and scope enforcement separately.
The security mechanism we discussed earlier is a layer above the API because it filters the calls to the API without the API knowing about it. During the calls, the user's resources are protected with row level security. This is done with labels on users protected resources. Labels correspond directly to the scope.
Scope storage and lookup is facilitated by token table. This doesn't mean scope labels cannot be included in the token itself.