Wilson Mar bio photo

Wilson Mar

Hello!

Calendar YouTube Github

LinkedIn

Pact for Integration Testing of REST APIs

Simplify integration testing by verifying equivalance of twins vs. real services

US (English)   Norsk (Norwegian)   Español (Spanish)   Français (French)   Deutsch (German)   Italiano   Português   Estonian   اَلْعَرَبِيَّةُ (Egypt Arabic)   Napali   中文 (简体) Chinese (Simplified)   日本語 Japanese   한국어 Korean

Overview

Pact is a word (not an acronym) for a family of software testing frameworks (at https://docs.pact.io) to take the place of integration testing.

The traditional approach of integration testing is to test the interaction between each service against all the other providers of services. This can be a massive effort.

But instead of testing all the combinations, the approach of Pact is to use mocks to independently test consumers and providers. First, during unit test runs, Pact captures web service calls consumers make with providers of web services in order to derive “contract files” that define their interaction.

Thus, this process is called “Contract Testing”.

The technical name for the contract files are “Consumer Driven Contract” (CDC) definitions. to ensure that independent of the are actually satisfied by the collaborating services built.

Since each microservice has its own database, data duplication is problematic.

All the steps below is summarized and animated by a PowerPoint file :

https://youtu.be/V2xMkAXlH60

pact-flow-650x372-59032

  1. NOTE: The genius behind Pact is it tests the integration of a Consumer and Provider by separating tests into two unit testing phases. Because only one component is being tested at a time, failures are easier to identify.

  2. Pact enables “Consumer Driven” testing in that it begins with testing the codebase from the Consumers’ point of view. Martin Fowler’s colleague Ian Robinson of Thoughtworks describes Consumer Driven Contracts: A Service Evolution Pattern. Since that 2006 article, the industry has advanced beyond Schemas and XML to JSON format. But the process is still useful.

  3. A reference to import or include the Pact framework are added to the top of unit test code. Pact functions and methods have been implemented in multiple programming languages (through several organizations):

    JVM Pact: https://github.com/DiUS/pact-jvm

    • https://github.com/DiUS/pact-jvm/tree/master/pact-jvm-consumer-java8 makes use of Lambda DSL.

    Pact.js: https://github.com/pact-foundation/pact-js which uses Mocha:

    const chai = require('chai')
const pact = require('pact')

    Pact Go: https://github.com/pact-foundation/pact-go

    Pact Python: https://github.com/pact-foundation/pact-python

    Ruby Pact: https://github.com/pact-foundation/pact-ruby which began at https://github.com/realestate-com-au/

    let Pact = require('pact')

    Pact Swift: https://github.com/DiUS/pact-consumer-swift

    .Net C# Pact: https://github.com/SEEK-Jobs/pact-net

    This means a consumer service can be written in JavaScript while the Provider service is written in Java.

    These frameworks implement the Pact specification (in JSON) defined by the Pact Foundation at https://github.com/pact-foundation/pact-specification.

    pact-mock-diagram-650x234-40214

  4. After code for a service is written and compiled, test code is run.

  5. The pact.setup() command creates a mock service Provider with which the Consumer code communicates. This means the Provider service does not need to be built in order for Consumer work to begin.

  6. pact.addInteraction() objects register an expectation on the Mock Server called by test case(s). This causes HTTP requests to be issued.

  7. ??? includes a state the provider should be in:

    given: "happy 123"

  8. If successful, the Pact framework automatically records interactions between a Consumer the mock Provider into JSON-formatted files called conract or pact files such as this.

  9. The framework then uses the contract file in a replay to the Provider in “Pact Provider tests”.

  10. The pact.verify() command then verifies that all and only the interactions specified occurred and that they match. This is called after any other assertions and once per test case. The pact.verify() command implicitly calls the removeInteractions to clear out expectations of the Mock Service for the next test run.

  11. pact.finalize() records the interactions registered to the Mock Server into the pact file and shuts it down.

  12. The Provider runs verification tests against a real (live) version of its service, using the shared pact file. This testing determines whether Test Doubles (mocks) of microservices are a valid stand-in for the real service.

  13. The contracts are stored either a) committed to a shared Git repo; b) uploaded to a shared file storage; or c) stored by the Pact Broker application.

  14. Pact Broker software can format reports about pacts in its store.

  15. [15:30] Because the Pact Broker handles the traffic, it can automatically generate a Network Diagram of relationships between services (who calls whom).

    The Contract Testing above ensures that pact file representing Consumer requests are compatible with how the Provider codebase handles those requests.

  16. Pact commands can be get pacts to run against either the Consumer or Provider.

### Versioning Backward Compatibility

When contracts are committed (or otherwise saved) into a source repository, the event is recognized by an event hook which kicks off a Jenkins job invoking the Pact server.

When known-compatible versions of Consumer and Provider goes into Production together at the same time, the Provider codebase (how it actually works) has already been tested as being compatible with pact files representing requests made by Consumers in Production.

But because Consumer and Provider are not always updated together at the same time, a legacy version in the Provider may not be compatible with a Consumer pact’s most recent version, which Git automatically labels as the latest “HEAD”.

New changes in the Provider codebase may break compatibility with Consumers in production.

These situations are illustrated using this “Pact Matrix”:

Versions: Consumer HEAD pact Consumer Prod pact
Provider HEAD codebase a) Contract tests first green-field release c) Contract tests to ensure provider is backwards compatible
Provider
Prod
codebase 		d) Contract tests to ensure consumer is backwards compatible b) Already tested when released together

So, to detect compatibility issues from disconnected versions:

  1. Recent changes in Consumer pacts should be tested against production Provider code.

  2. Recent changes to the Provider codebase should be tested against pacts for the Consumer in production.

### Postel’s Law

Postel’s Law, also called Robustness Principle, says be strict in requests sent out, but interpretation of responses should be loose (liberal or be accomodating) such as by allowing (not rejecting) extra keys unused by some consumers. This and this video uses the example of internet browsers being forgiving of HTML errors.

Network Diagram

pact-network-graph-654x513.jpg

Competitors

Pact eliminates the need for integration tests and thus a separate set of servers for the integration environment.

  • Pact takes the place of structured serialization formats such as protobuf, thrift, or messagepack.

  • Spring Cloud Contract is a competitor to Pact. See this video by Adib Saikali. It is only Java.

Tests generated from Swagger (now OpenAPI) specifications are complimentary to Pact because they focus on whether servers can handle various HTTP response codes and data types defined in the spec. This is done by Atlassian’s swagger-mock-validator command line tool which confirms whether mock request and responses conform to the schema specified in a swagger specification (rules defined). Its command format is:

swagger-mock-validator /path/to/swagger.json \
   https://pact-broker \
   --provider my-provider-name

Sample Contract

A Consumer Driven Contract (aka pact) is a JSON-formatted file that contains a collection of agreements between a client (Consumer) and an API (Provider) that describes the interactions that can take place between them.

{
  "consumer": {
    "name": "billy"
  },
  "provider": {
    "name": "bobby"
  },
  "interactions": [
    {
      "description": "My test",
      "providerState": "User billy exists",
      "request": {
        "method": "POST",
        "path": "/users/login",
        "headers": {
          "Content-Type": "application/json",
        },
        "body": {
          "username":"billy",
          "password":"issilly"
        }
      },
      "response": {
        "status": 200,
      }
    },
  ],
  "metadata": {
    "pactSpecification": {
      "version": "2.0.0"
    }
  }
}

The above is from https://codefresh.io/blog/how-to-test-microservice-integration-with-pact/

There have been several versions of the Pact format (1.0, 1.1, 2.0).

If a pact such as the above is stored in a Pact Broker service, pasting the pact URL into a browser would return a formatted presentation of the pact generated.

Test separately

Beth Skurrie (@bethesque, of Melbourne, Victoria and https://github.com/realestate-com-au/pact ) explains in a [26:37] video on Sep 04, 2015 at [9:32] says “Pact mocks the provider when it the consumer makes calls and handles responses correctly. [9:45] While consumer tests are running, the interactions is recorded into a Pact file as a contract. That file is then used to interact with the real provider to ensure “test symmetry” that mock and real providers work the same way.

The process also supports the inverse scenario.

Combinatorial: 3 classses x 4 code paths each (4 * 4 * 4) = 64 tests

Pact Broker

Harry Winser (@hazz223, Pact Broker author), in VIDEO: Consumer Driven Contract Testing with Pact and Docker 18 Oct 2017 illustrates the flow of work in a Pact service [13:05] pact-test-flow-616x327-22025

[12:19] shows that Pact Provider contract tests can also be used in QA and Staging. pact-in-pipeline-567x250-18831 [19:09]

[15:20] Consumer tests pact-consumer-tests-650x362-27115

[20:03] Docker

Pact Broker Install

There are several ways

Dockerized

  1. Use the sample Docker Compose setup at:

    https://github.com/DiUS/pact_broker-docker/blob/master/docker-compose.yml

  2. Modify the docker-compose.yml file as required.

    For a quick start with the Pact Broker and Postgres, we have an

    Run docker-compose up to get a running Pact Broker and a clean Postgres database

    Now you can access your local broker:

  3. Get IP of your running Docker instance

    DOCKER_HOST=$(docker-machine ip $(docker-machine active))
curl -v http://$DOCKER_HOST # you can visit in your browser too!

NOTE: this image should be modified before using in Production, in particular, the use of hard-coded credentials

A) Use Terraform on AWS

Alternately, instantiate instances on AWS using https://github.com/nadnerb/terraform-pact-broker

C) Create a Google Insta

Custom Docker image

Alternately, one sample Dockerfile to bring up Pact-Js within Docker,

FROM ubuntu:16.04
RUN apt-get update
RUN apt-get install -y   default-jre   default-jdk   git curl
RUN apt-get update
RUN curl -sL https://deb.nodesource.com/setup_7.x | bash -
RUN apt-get install -y nodejs
RUN npm -v
RUN node -v
RUN apt-get update
RUN curl -sL  https://deb.nodesource.com/setup_8.x | bash -
RUN apt-get install -y nodejs
RUN npm -v && node -v
RUN git clone https://github.com/pact-foundation/pact-js.git
RUN cd pact-js && npm install ; exit 0
RUN cd pact-js/examples/e2e && npm install
RUN cd pact-js/examples/e2e && npm run test:consumer
RUN cd pact-js/examples/e2e && npm run test:publish
RUN cd pact-js/examples/e2e && npm run test:provider

If you want to test your own app, substitue pact-js/examples/e2e with your own app.

  1. Install Docker
  2. Instantiate a Postgress database.
  3. Look at https://github.com/DiUS/pact_broker-docker

  4. Instantiate a container using the Docker image at

    https://hub.docker.com/r/dius/pact-broker/tags/

    
 docker pull dius/pact-broker

  5. Start Docker image:

    
docker run -d -p 80:80 pact-broker
  6. Re-run

Jenkins

pipeline {
    agent none 
    stages {
        stage('Example Build') {
            agent { docker "rest-test" } 
            steps {
                sh 'ls -ltr'
            }
        }
       
    }
}

Sample Pact exchange

First of all, if the consumer assumes the API would be based on posting a JSON document (being invoked from a web browser), make sure that the provider is not implemented using an application/x-www-form-urlencoded POST API.

Development of the interaction starts from the consumer side, with a test that at first fails until code is written to make the test pass (TDD here).

This contract is then provided to the Provider team to implement the provider service to fulfill the contract.

PROTIP: Contracts are readable by both people and software.

[11:04] Arrange, Pact, and Assert

This DSL from Guidelines:

Pact.service_consumer "Zoo App" do
  has_pact_with "Animal Service" do
    mock_service :animal_service do
      port 1234
    end
  end
end

https://gist.github.com/bethesque/9d81f21d6f77650811f4 You will need ruby or the standalone mock service to run these examples.

gem install pact-mock_service

pact-mock-service help start

References

Ruby Pact wiki: github.com/realestate-com-au/pact/wiki

How to Test Microservice Integration with Pact 9 Oct 2017 by Anton Weiss

Pact 101 – Getting started with Pact and Consumer Driven Contract Testing blog 03/02/2016 By Ron Holshausen

https://www.youtube.com/watch?v=h-79QmIV824 Deploy with Confidence! - Ron Holshausen Video April 6, 2016.

https://www.youtube.com/watch?v=-6x6XBDf9sQ Verifying Microservice Integrations with Contract Testing - Atlassian Summit 2016 Atlassian

https://app.pluralsight.com/library/courses/code-contracts/table-of-contents Code Contracts</a> 1h 51m video course released 17 Jul 2012 by John Sonmez provides an introduction to Code Contracts in the Microsoft .NET Framework.

Martin Fowler on Microservice Testing ( http://martinfowler.com/articles/microservice-testing/#definition Article)

Introduction to consumer-driven contracts with Pact (http://dius.com.au/2016/02/03/microservices-pact/ Article)

Integrated tests are a scam - J.B. Rainsberger (https://vimeo.com/80533536 Video / http://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam Article)

Verifying Microservice Integrations with Contract Testing - Atlassian (https://www.youtube.com/watch?v=-6x6XBDf9sQ&feature=youtu.be Video)

Escape the integration syrup with contract tests by Stefan Smith ([45:11] Video at Agile on the Beach 2015

https://www.youtube.com/watch?v=MDydAqL4mYE Consumer Driven Contracts and Your Microservice Architecture</a> [51:01] at Devoxx by Marcin Grzejszczak (@MGrzejszczak) and Josh Long (at Pivotal).

https://www.youtube.com/watch?v=sAAklvxmPmk&t=540s by Marcin Grzejszczak

Social

Twitter: @pact_up #contract-tests

Gitter: Join the chat at https://gitter.im/realestate-com-au/pact where Beth talks with Kevin Meiresonne, dengayevskiy, etc.

Stackoverflow: https://stackoverflow.com/questions/tagged/pact ruby pact questions or general pact questions

https://groups.google.com/forum/#!forum/pact-dev Google users group: for ongoing discussions rather than questions

More

Pact for Integration Testing of REST APIs was published on .