Test automation framework for API testing :
The following post attempts to enumerate some of the salient features that make an automation easier to use for API testing. APIs such as those that are available over the web as RESTful APIs are great candidates for automations because there is a lot of repetitions and overlaps in the test matrix. The test matrix consists of variations in protocol such as http or https and their versions, verbs such as GET, PUT, POST, DELETE, etc, endpoints and their ports for test, stage, production environments, endpoint versions, resource qualifiers and their query strings, their response types such xml or json and the request body in the case of post, the content-headers and their various options. A request and response sequence for each API alone constitutes a bulk of the test matrix.
Note that the testing largely relies on default values so that the test scripts are easier to write. Python requests library really help with making the scripts easier to write. One of the primary goals of automation framework is to make it easy to add tests. The lesser the code required to write the tests the lesser the chances of making mistakes, the lesser the frustrations, the easier it is to maintain and hence more popular. Python library enables succinct and terse test scripts. It's cool to write Python scripts.
Besides, UI based tools such as SOAPUI or TestRunner are nice to haves but consider that the primary input and output for REST API testing is all text. Variations in the text, their comparision, serialization and deserialization into objects for comparision with Data access layer are tedious and may need to be specified in the tests. While load testing, performance and call inspectors, capture and replay are undeniably UI features, the automation framework can separate out the UI tests from the scripts and in my case argue for script based framework.
The object model library should be comprehensive because such would be available in the corresponding development code. In fact test should refrain from from rewriting or redefining any or all objects that the requests and responses use. These objects should all be instanted via reflection and can live in properly namespaced development assemblies. Unifying the object model for development and test is not just for convenience, it enforces single maintenance policy and reuse. Code may be tweaked for different purposes in either test side or development side but object model has no reason to be different between test and dev code. To illustrate this point, in a well written development code, if you were to add strongly typed views in a model view controller framework, when you expand the drop down to choose the model for the view, you should be able to see all ( read dozens ) of objects to choose from. The same or similar should be available in test. The data and the object model mapping is also common. When the object model changes and the development code changes, the corresponding tests will break. The more they are tied, the more visible the regressions.
Testing often relies on interpreting data from the layers below the API implementation to compare the expected and actual results from the API. With default values for test scripts and a unified object model collection of assemblies, the test automation will be easier to use, more consistent and organized.
The following post attempts to enumerate some of the salient features that make an automation easier to use for API testing. APIs such as those that are available over the web as RESTful APIs are great candidates for automations because there is a lot of repetitions and overlaps in the test matrix. The test matrix consists of variations in protocol such as http or https and their versions, verbs such as GET, PUT, POST, DELETE, etc, endpoints and their ports for test, stage, production environments, endpoint versions, resource qualifiers and their query strings, their response types such xml or json and the request body in the case of post, the content-headers and their various options. A request and response sequence for each API alone constitutes a bulk of the test matrix.
Note that the testing largely relies on default values so that the test scripts are easier to write. Python requests library really help with making the scripts easier to write. One of the primary goals of automation framework is to make it easy to add tests. The lesser the code required to write the tests the lesser the chances of making mistakes, the lesser the frustrations, the easier it is to maintain and hence more popular. Python library enables succinct and terse test scripts. It's cool to write Python scripts.
Besides, UI based tools such as SOAPUI or TestRunner are nice to haves but consider that the primary input and output for REST API testing is all text. Variations in the text, their comparision, serialization and deserialization into objects for comparision with Data access layer are tedious and may need to be specified in the tests. While load testing, performance and call inspectors, capture and replay are undeniably UI features, the automation framework can separate out the UI tests from the scripts and in my case argue for script based framework.
The object model library should be comprehensive because such would be available in the corresponding development code. In fact test should refrain from from rewriting or redefining any or all objects that the requests and responses use. These objects should all be instanted via reflection and can live in properly namespaced development assemblies. Unifying the object model for development and test is not just for convenience, it enforces single maintenance policy and reuse. Code may be tweaked for different purposes in either test side or development side but object model has no reason to be different between test and dev code. To illustrate this point, in a well written development code, if you were to add strongly typed views in a model view controller framework, when you expand the drop down to choose the model for the view, you should be able to see all ( read dozens ) of objects to choose from. The same or similar should be available in test. The data and the object model mapping is also common. When the object model changes and the development code changes, the corresponding tests will break. The more they are tied, the more visible the regressions.
Testing often relies on interpreting data from the layers below the API implementation to compare the expected and actual results from the API. With default values for test scripts and a unified object model collection of assemblies, the test automation will be easier to use, more consistent and organized.
No comments:
Post a Comment