Generate REST API Test Code

No waiting for test automation

Overview

What took us so long?
Client applications
Specification databases
- RAML
- WADL Example
Test code
- JavaScript LoadRunner Example
- Java Selenium Example
Generator Control File
CI/CD Toolchain
Test code generation programs
More on API Microservices

I have a dream …

where developers get their test scripts not by waiting for manual coding, but by

fig test code gen v01 998x568.pptx

test code generation programs referencing
a specification database
also used to create client applications making calls to servers.
All this done by a toolchain of Continuous Integration/Continous Deployment.
The specifications can be specified manually or generated from programs which scan programming code to extract specifications.
Variations in the generated code are defined in a control file.
This control file can be edited manually using a text editor and manually referencing the specifications,
adjusting for issues observed by the generator.
Results from test generator run include timings of transactions and size of resources processed on the client and
metrics from monitoring of servers.
Analysis of these results are made in light of the original specifications.
Insights from human or artificial intelligence doing the analysis would influence what testing code to generate in subsequent rounds.

What took us so long?

I think that when software vendors focus on tools for testers, they may miss or dismiss the idea of eliminating them.

The automation discusseed here has the objective of “testers” being developers themselves – developers of test code generation programs.

The audience for this page are developers who want to move faster.

Client applications

Let’s for now focus on examples of clients making REST API calls.

Yahoo Weather
Google Weather
Twitter http://www.ibm.com/developerworks/library/x-twitterREST/index.html
https://developer.salesforce.com/page/REST_API

Tutorials on APIs include:

http://www.restapitutorial.com/

Specification databases

Several databases for APIs have grown up over the years:

Specifications for REST API (such as Swagger) are more amenable as the basis for generating code if they are complete specifications of how client computers interact with servers.

Specifications about screen elements may be too complex or don’t provide enough information for use in code generation at this point in time.

	Swagger	RAML	API-Blueprint
Format:	JSON	YAML	Markdown
Sponsor:	Reverb	Mulesoft	Apiary
Initial:	July, 2011	Sep, 2013	April, 2013
Approach:	Bottom-up	Top-down	Top-down

Other alternatives include:

IO-Docs from Mashery (licensed).
JASONDoc has a Editor that reads a JSON Schema and generates an HTML form to manipulate it.
Apiary.io (licensed)
WADL
WSDL SOAP
WADL (Web Application Description Language) generated to describe SOAP are not viable for REST API because they do not include enough information.
DOORS requirements

RAML

RAML (at RAML.org) can reuse WADL (pronounced “waddle”) introspection logic of Restlet Framework resources developed by Java (SE/EE, Google AppEngine, OSGi, GWT, Android) REST API developers using the Restlet Studio and the APISpark cloud managed by Reslet based in France with an office in Palo Alto.

Reslet (in Oct. 2015) bought Czech Filip Kolařík’s DHC (Dev HTTP Client) Chrome Add-in and https://www.sprintapi.com/dhcs.html to test and debug web APIs

Swagger2RAML converts Swagger JSON to RAML YAML.

API Description Languages: Which One Is Right For Me? Aug. 15, 2014 by Laura dot Heritage at soa.com (Akana, formerly SOA Software).

WADL Example

Let’s use the sample yahoo-news-call.wadl.xml file from the Wikipedia page defining one. Open the file using an XML reader app.

The service endpoint is specified by:


<resources base="http://api.search.yahoo.com/NewsSearchService/V1/">

Test code

There are several languages used by test script processors:

XML for JMeter
Java for Selenium
C for LoadRunner
JavaScript for LoadRunner
VB for HP QTP
SOAPUI
many others

JavaScript LoadRunner Example

An example of the JavaScript code generated to emulate a client retrieving an index.html file:

  web.url(
    {
      name : 'index.html',
      url : '{pEndPoint}',
      resource : 0,
      recContentType : 'text/html',
      referer : '',
      snapshot : 't1.inf',
      mode : 'HTML'
    }
  );

“{pEndPoint}” is the run parameter containing an End Point such as:

http://api.search.yahoo.com/NewsSearchService/V1/

The “mode: HTML” specifies processing by LoadRunner to scan the html returned and issue requests for links to CSS, JavaScript, images, etc.

(Images specified within CSS are not retrieved this way)

Java Selenium Example

TODO:

Generator Control File

We use the yml format that can be read by humans and computers. Ruby programmers are used to such files.

Since Swagger users may prefer JSON formatting:

Input Specification database
Output Test code language
Authentication (OAuth2, JWT, BasicAuth)
Call Method (GET, PUT, POST, UPDATE, etc.).
Authorization (such as tokens in the HTTP header).
Response code (if not standard 200).
Repetitions.
Negative tests (to force 400 to return).
Data errors (bad attribute names, bad attribute value).

CI/CD Toolchain

Test code generation programs

I have examples of code generation programs written in Python, PHP, Java, Scala, and even VBScript in Excel files.

More on API Microservices

This is one of a series:

Wilson Mar