OAuth testing discussion continued

The previous post on OAuth testing discussed the mapping of user and client for grant of token. Therefore testing discussed mostly on the setup of a user and client for the purpose of grant token. The token has a certain expiry time. There was some mention about scope and state. In this post, we discuss the revokes
When the authorization website grants access to a client from a user, the client can get a "bearer" token to access the protected resources. This token can be used by anyone for any of the APIs.  The authorization server must implement TLS so that the passwords and bearer tokens can flow.
OAuth only mandates that the token are issued for less than an hour. However clients, user and the authorization website could establish trust relationship the first time the authorization occurs. This is outside the scope of OAuth and may be provided as a mere convenience to the user from having to sign on each time for the client.
If this relationship were to be persisted it would merely be a table mapping the client with the user on this server. The records could be added when the authorization completes and revoked when the user explictly wants to clean up the clients she has authorized. The authorization website identifies the user by the user id and the client by the client id. When the client requests the authorization again subsequently, the website may have the user signed on from a previous session and hence uses the same to generate a token for the client. Neither the user specifies password again, nor does the client change request anything different other than accepting the redirect uri response with the token it is expecting.
Care should be taken to ensure that retries by the user do not create duplicate rows in the table maintained by the authorization website. In addition to user explicit revokes, the website may grant additional session time than the usual token expiry time. This can be facilitated with the use of a timestamp.
Lastly the website may do periodic cleanups and archival of the trust relationships. The default case could be to not store the trust relationships and have the token expirty time force re-authentication.

In the previous post, there was a sample code for objective C. The XCode project comes with a test  project that can be used for testing the code in the project. Here is an example for the code in the test:

//

//  Starbucks_API_ExplorerTests.m

//  Starbucks API ExplorerTests

#import "Starbucks_API_ExplorerTests.h"

@implementation Starbucks_API_ExplorerTests

- (void)setUp

{

    [super setUp];

}

- (void)tearDown

{

    // Tear-down code here.

    

    [super tearDown];

}

- (void)testExample

{

    STFail(@"Unit tests are not implemented yet in Starbucks API ExplorerTests");

}

@end

sample objective c code

//

//  ViewController.m

#import "ViewController.h"

@interface ViewController ()

@end

@implementation ViewController

@synthesize userInput;

@synthesize userParams;

@synthesize userOutput;

@synthesize username;

@synthesize password;

- (void)viewDidLoad

{

    [super viewDidLoad];

// Do any additional setup after loading the view, typically from a nib.

}

- (void)didReceiveMemoryWarning

{

    [super didReceiveMemoryWarning];

    // Dispose of any resources that can be recreated.

}

- (IBAction)Submit:(id)sender {

    ResponseViewer.text =  [self makeRequest];

}

- (NSString*) getAccessToken {

    NSString *urlString = @"https://test.openapi.starbucks.com/v1/oauth/token";

    NSMutableURLRequest *request = [[NSMutableURLRequest alloc] init];

    [request setURL:[NSURL URLWithString:urlString]];

    [request setHTTPMethod:@"POST"];

    

    NSMutableData *body = [NSMutableData data];

    NSString *requestBody = [NSString stringWithFormat:@"grant_type=password&username=%@&password=%@client_id=my_client_id&client_secret=my_client_secret&scope=test_scope&api_key=my_api_key", username.text, password.text];

    [body appendData:[[NSString stringWithFormat:@"%@\r\n", requestBody] dataUsingEncoding:NSUTF8StringEncoding]];

    

    // set request body

    [request setHTTPBody:body];

    [request setValue:@"application/json" forHTTPHeaderField:@"Accept"];

    

    //return and test

    NSData *returnData = [NSURLConnection sendSynchronousRequest:request returningResponse:nil error:nil];

    NSString *returnString = [[NSString alloc] initWithData:returnData encoding:NSUTF8StringEncoding];

    

    NSLog(@"%@", returnString);

    

    NSData *JSONData = [returnString dataUsingEncoding:NSUTF8StringEncoding];

    

    userOutput.text = returnString;

    

    NSDictionary* json = [NSJSONSerialization JSONObjectWithData:JSONData options:kNilOptions error:nil];

    

    return [json objectForKey:@"access_token"];

    

}

- (NSString*) makeRequest {

    NSString *urlString = [NSString stringWithFormat:@"https://test.openapi.starbucks.com/%@", userInput.text];

    NSMutableURLRequest *request = [[NSMutableURLRequest alloc] init];

    [request setURL:[NSURL URLWithString:urlString]];

    [request setHTTPMethod:[NSString stringWithFormat:@"%@", _method.accessibilityValue]];

    

    if ([_outputType.accessibilityValue compare:@"Json" options:NSCaseInsensitiveSearch] == TRUE)[request setValue:@"application/json" forHTTPHeaderField:@"Accept"];

    

    if ([_outputType.accessibilityValue compare:@"XML" options:NSCaseInsensitiveSearch] == TRUE)[request setValue:@"application/xml" forHTTPHeaderField:@"Accept"];

    

    

    if ([_method.accessibilityValue compare:@"POST" options:NSCaseInsensitiveSearch] == TRUE)

    {

        NSMutableData *body = [NSMutableData data];

        NSString *requestBody = userParams.text;

    

        [body appendData:[[NSString stringWithFormat:@"%@\r\n", requestBody] dataUsingEncoding:NSUTF8StringEncoding]];

    

        // set request body

        [request setHTTPBody:body];

    }

    

    //return and test

    NSData *returnData = [NSURLConnection sendSynchronousRequest:request returningResponse:nil error:nil];

    NSString *returnString = [[NSString alloc] initWithData:returnData encoding:NSUTF8StringEncoding];

    

    NSLog(@"%@", returnString);

    userOutput.text = returnString;

    return returnString;

    

}

    

@end

In the previous post we talked about database transforms of test results and mail. This was easy to implement when you apply the xslt transform. The resulting procedure with cursors looks something like this:

DECLARE @results table(
TestRunId uniqueidentifier NOT NULL,
SubmittedBy varchar(30) NOT NULL,
TestOwner varchar(30) NOT NULL,
TestName varchar(255) NOT NULL,
Environment varchar(10) NOT NULL,
TestCategory varchar(255) NOT NULL,
Submitted varchar(255) NOT NULL,
Status varchar(10) NOT NULL,
Elapsed varchar(255) NOT NULL,
Total int NOT NULL,
Passed int NOT NULL,
Fail int NOT NULL,
Inconclusive int NOT NULL,
Error int NOT NULL,
PercentPass varchar(10) NOT NULL,
ResultLabelWidth varchar(10) NOT NULL);

DECLARE @TestRunId uniqueidentifier ;
DECLARE @SubmittedBy varchar(30) ;
DECLARE @TestOwner varchar(30) ;
DECLARE @TestName varchar(255) ;
DECLARE @Environment varchar(10) ;
DECLARE @TestCategory varchar(255) ;
DECLARE @Submitted varchar(255) ;
DECLARE @Status varchar(10) ;
DECLARE @Elapsed varchar(255) ;
DECLARE @Total int ;
DECLARE @Passed int ;
DECLARE @Fail int ;
DECLARE @Inconclusive int ;
DECLARE @Error int ;
DECLARE @PercentPass varchar(10) ;
DECLARE @ResultLabelWidth varchar(10);


INSERT into @results SELECT * FROM dbo.Results;  -- or exec stored_proc
DECLARE @msg nvarchar(MAX);
SET @msg = '<head></head><body>
<div>Test Run Report</div><table id="ResultContainer" border="1">
<tr>
<th>Service Name</th>
<th>Environment</th>
<th>Test Category</th>
<th>Submitted By</th>
<th>Date</th>
<th>Status</th>
<th>Elapsed</th>
<th>Total</th>
<th>Pass</th>
<th>Fail</th>
<th>Incon</th>
<th>Error</th>
<th>Result</th>
<th>Test Owner</th>
</tr>'
DECLARE Results_Cursor CURSOR FOR  (SELECT * From @results); -- exec [ms01806.sbweb.prod].TestRunner.dbo.usp_GetSubmittedTestStatus @SubmittedBy='Ravi Rajamani (CW)',@TodayOnly=1,@TestType=N'Coded';
OPEN Results_Cursor;
FETCH NEXT FROM Results_Cursor INTO @TestRunId, @SubmittedBy , @TestOwner , @TestName, @Environment , @TestCategory , @Submitted , @Status , @Elapsed , @Total , @Passed , @Fail , @Inconclusive , @Error , @PercentPass , @ResultLabelWidth ;
WHILE @@FETCH_STATUS = 0
BEGIN
SET @msg = @msg + '<tr>
                            <td>' +  @TestName + '
                            </td>
                            <td> ' + @Environment +  '
                            </td>
                            <td>' + @TestCategory + '
                            </td>
                            <td>'  + @SubmittedBy + '
                            </td>
                            <td>' + @Submitted + '
                            </td>
                            <td> '  +  @Status + '
                            </td>
                            <td>' + @Elapsed + '</td>
                            <td>' + Convert(nvarchar, @Total) + '
                            </td>
                            <td>' + Convert(nvarchar, @Passed) + '
                            </td>
                            <td>'  + Convert(nvarchar, @Fail) + '
                            </td>
                            <td> ' + Convert(nvarchar, @Inconclusive) + '
                            </td>
                            <td>' + Convert(nvarchar, @Error) + '
                            </td>
                            <td>' + @PercentPass + '
                            </td>
                            <td> ' + @TestOwner + '
                            </td>
                        </tr>';
FETCH NEXT FROM Results_Cursor INTO @TestRunId, @SubmittedBy , @TestOwner , @TestName, @Environment , @TestCategory , @Submitted , @Status , @Elapsed , @Total , @Passed , @Fail , @Inconclusive , @Error , @PercentPass , @ResultLabelWidth ;
END
SET @msg = @msg + '</table></body>';
CLOSE Results_Cursor;
DEALLOCATE Results_Cursor;

EXEC msdb.dbo.sp_send_dbmail
    @profile_name = 'Mail Profile',
    @recipients = 'recipients@where.com',
@body_format = 'HTML',
    @body = @msg,
    @subject = 'Test Results Reports' ;

automating test results publishing from the database.

Test runs generate results that are often important for keeping a daily pulse on the software quality. Typically such data is published in a database and have details such as test run id, submitted by, test owner, test name, environment, test category, submitted, status, Elapsed time, total tests passed, total tests failed, inconclusive tests, Tests that have errors, percentage pass etc. These records give sufficient detail about the tests that were run and they just need to be transformed into xslt to make pretty reports. Both the xslt transform and associated css if any could be the text template with which the data is presented. The individual spans for each column may need to have distinguished identifiers so that the transform can be rendered as an html. These identifiers could all have row number as suffix to distinguish them.  A header row followed by one or more rows for the data is sufficient to display the test run results. The table itself could use a css class to make it prettier by differentiating alternate rows. This is done with a class id and a style borrowed from the css. The transform itself could be wrapped in a stored procedure which retrieves the results from the database and enumerates them. This stored procedure has several advantages. First this logic is as close to the data as possible. Second it can be invoked by many different clients on scheduled or on demand basis. Third, it can be used to register a trigger for the insert of results from well marked runs. Runs can be marked based on some attributes such as a login for automated test execution from reporting. This allows regular test invocation and publish by all users without any reporting. Only the test execution by a separate login is candidate for reporting. The stored procedure may need to enumerate through the records. This can be done with a cursor. A cursor is also useful in working with one record at a time. The entire stored procedure will have plans cached for improved performance over adhoc TSQL. Therefore this will have performance improvements as well. Finally the database itself can send out mails with this html text. This can be done with a database mail  profile set up with an smtp server. Setting up automated e-mails requires the use of sysadmin or appropriate server scoped privileges.
If transforms are not needed, the results from the queries or stored procedures can be directly sent out. This may be a quick and easy thing to do since the results can be output to text format and imported via excel and other BI applications.
Also, test results are often associated with team foundation server work items or queries. Such articles might involve items such as active and resolved bug counts. While it is easier to use the Excel application to open and format such TFS queries, it is also possible to directly issue http requests from the database. This consolidates the results and the logic for results reporting in one place. 

REST API documentation

There are several tools to manage REST API documentation. Apiary.IO lets you create documentation from a terse blueprint written in their syntax. Input and output can be listed in an excel sheet and the scripts can be used to generate html pages.
HOST: https://testhost.openapi.starbucks.com

--- Sample API v2 ---
---
Welcome to the our sample API documentation. All comments can be written in (support [Markdown](http://daringfireball.net/projects/markdown/syntax) syntax)
---

--
OAuth workflow
The following is a section of resources related to the OAuth access grant methods
--
List products added into your shopping-cart. (comment block again in Markdown)
POST /v1/token
{ "grant_type":"password", "client_id":"abcdefg8hijklmno16", "client_secret=some_secret", "username":"test", "password":"password", scope:"test_scope"}
< 200
< Content-Type: application/json
{"return_type":"json","access_token":"nmhs7y7qmmh8mngyq2ncn3gp","token_type":"bearer","expires_in":3600,"refresh_token":"5skg9u4qdybksete8gaauvk3","scope":"test_scope","state":null,"uri":null,"extended":null}

OAuth workflow

The following is a section of resources related to the OAuth access grant methods

POST

/v1/token

Add Comment

List products added into your shopping-cart. (comment block again in Markdown)

• Example
 
• Debugger
 
• Comments
 
• Try It

Show code sample

                                            none (raw data)                                           curl                                           C#                                           javascript                                           node.js                                           python                                           php                                           ruby                                       

Request

{ 

"grant_type":"password", 

"client_id":"abcdefg8hijklmno16", 

"client_secret=some_secret", 

"username":"test", 

"password":"password", 

scope:"test_scope"

}

Response

200 (OK)
    Content-Type: application/json
    

{"return_type":"json",

"access_token":"nmhs7y7qmmh8mngyq2ncn3gp",

"token_type":"bearer",

"expires_in":3600,

"refresh_token

Test automation framework for API testing :
The following post attempts to enumerate some of the salient features that make an automation easier to use for API testing. APIs such as those that are available over the web as RESTful APIs are great candidates for automations because there is a lot of repetitions and overlaps in the test matrix. The test matrix consists of variations in  protocol such as http or https and their versions, verbs such as GET, PUT, POST, DELETE, etc, endpoints and their ports for test, stage, production environments, endpoint versions, resource qualifiers and their query strings, their response types such xml or json and the request body in the case of post, the content-headers and their various options. A request and response sequence for each API alone constitutes a bulk of the test matrix.
Note that the testing largely relies on default values so that the test scripts are easier to write. Python requests library really help with making the scripts easier to write. One of the primary goals of automation framework is to make it easy to add tests. The lesser the code required to write the tests the lesser the chances of making mistakes, the lesser the frustrations, the easier it is to maintain and hence more popular. Python library enables succinct and terse test scripts. It's cool to write Python scripts.
Besides, UI based tools such as SOAPUI or TestRunner are nice to haves but consider that the primary input and output for REST API testing is all text. Variations in the text, their comparision, serialization and deserialization into objects for comparision with Data access layer are tedious and may need to be specified in the tests. While load testing, performance and call inspectors, capture and replay are undeniably UI features, the automation framework can separate out the UI tests from the scripts and in my case argue for script based framework.
The object model library should be comprehensive because such would be available in the corresponding development code. In fact test should refrain from from rewriting or redefining any or all objects that the requests and responses use. These objects should all be instanted via reflection and can live in properly namespaced development assemblies. Unifying the object model for development and test is not just for convenience, it enforces single maintenance policy and reuse. Code may be tweaked for different purposes in either test side or development side but object model has no reason to be different between test and dev code. To illustrate this point, in a well written development code, if you were to add strongly typed views in a model view controller framework, when you expand the drop down to choose the model for the view, you should be able to see all ( read dozens ) of objects to choose from. The same or similar should be available in test. The data and the object model mapping is also common. When the object model changes and the development code changes, the corresponding tests will break. The more they are tied, the more visible the regressions.
Testing often relies on interpreting data from the layers below the API implementation to compare the expected and actual results from the API. With default values for test scripts and a unified object model collection of assemblies, the test automation will be easier to use, more consistent and organized.