Wilson Mar bio photo

Wilson Mar

Hello. Hire me!

Email me Calendar Skype call 310 320-7878

LinkedIn Twitter Gitter Google+ Youtube

Github Stackoverflow Pinterest

Verify equivalance of twins vs. real services

Pact is a word (not an acronym) for a family of software testing code frameworks (at https://docs.pact.io) to verify whether “Consumer Driven Contracts” (CDCs) definitions are actually satisfied by the collaborating services built.

It’s called “Consumer Driven” because Pact test framework and server software enables a development process that drives the development of service Providers from its consumers’ point of view. Martin Fowler’s colleague Ian Robinson of Thoughtworks describes Consumer Driven Contracts: A Service Evolution Pattern. Since that 2006 article, the industry has advanced beyond Schemas and XML to JSON format. But the process is still useful.

Versioning Backward Compatibility

Changing a published API over time is risky due to backward compatibility concerns. This is even more of an issue in a microservice architecture with 100’s of microservices that each publish an API.

This blog posted December 5, 2014 by Beth Skurrie describes this “Pact Matrix”:

- Consumer Head (pact) Consumer Prod (pact)
Provider Head (codebase) Contract tests Contract tests to ensure provider is backwards compatible
Provider Prod (codebase) Contract tests to ensure consumer is backwards compatible Already tested

Combinatorial: 3 classses x 4 code paths each (4 * 4 * 4) = 64 tests

Workflow

NOTE: The genius behind Pact is that it splits integration testing into separate unit tests. The advantage of this approach is that because only one component is being tested at a time, failures are easier to identify.

pact-diagram-1010x652-61431.png [.pptx]

  1. A reference to import or include the Pact framework are added to the top of test code. Pact functions and methods have been implemented in multiple programming languages (through several organizations):

    JVM Pact: https://github.com/DiUS/pact-jvm

    Pact.js: https://github.com/pact-foundation/pact-js

    const pact = require('pact')
    

    Pact Go: https://github.com/pact-foundation/pact-go

    Pact Python: https://github.com/pact-foundation/pact-python

    Ruby Pact: https://github.com/realestate-com-au/pact

    Pact Swift: https://github.com/DiUS/pact-consumer-swift

    .Net C# Pact: https://github.com/SEEK-Jobs/pact-net

    This means a consumer service can be written in JavaScript while the Provider service is written in Java.

    These frameworks implement the Pact specification (in JSON) defined by the Pact Foundation at https://github.com/pact-foundation/pact-specification.

  2. Pact creates and spins up a provider stub against which the test executes.

    The pact.setup() command starts a Mock server.

  3. Tests that interact with a Provider:

    pact.addInteraction() objects register an expectation on the Mock Server, which must be called by your test case(s). You can add multiple interactions per server. These will be validated and written to a pact if successful.

    This includes a state the provider should be in,

  4. pact.verify() verifies that all and only the interactions specified occurred and that they match. This is called after any other assertions and once per test case.

    The verify() command implicitly call the removeInteractions to clear out expectations of the Mock Service for the next test run.

  5. pact.finalize() records the interactions registered to the Mock Server into the pact file and shuts it down.

  6. The output of Pact Consumer runs is one or more json files such as this.

  7. These contracts (also called “pacts”) are either a) committed to a shared Git repo; b) uploaded to a shared file storage; or c) stored by the Pact Broker application.

  8. Pact files are then used to make requests by consumers to the Provider service.

    Pact testing is useful to determine whether Test Doubles (mocks) of microservices are a valid stand-in for the real service.

  9. The Provider runs its own verification tests against a real-life version of its service, using the shared pact file.

  10. When consumer software makes calls to live Provider services,

Competitors

  • Pact takes the place of structured serialization formats such as protobuf, thrift, or messagepack.

  • Spring Cloud Contract is a competitor to Pact. See this video by Adib Saikali. It is only Java.

Tests generated from Swagger (now OpenAPI) specifications are complimentary to Pact because they focus on whether servers can handle various HTTP response codes and data types defined in the spec. This is done by Atlassian’s swagger-mock-validator command line tool which confirms whether mock request and responses conform to the schema specified in a swagger specification (rules defined). Its command format is:

swagger-mock-validator /path/to/swagger.json https://pact-broker --provider my-provider-name

PROTIP:

Test Doubles / Mocks

Sample Contract

A Consumer Driven Contract is a JSON file that contains a collection of agreements between a client (Consumer) and an API (Provider) that describes the interactions that can take place between them.

{
  "consumer": {
    "name": "billy"
  },
  "provider": {
    "name": "bobby"
  },
  "interactions": [
    {
      "description": "My test",
      "providerState": "User billy exists",
      "request": {
        "method": "POST",
        "path": "/users/login",
        "headers": {
          "Content-Type": "application/json",
        },
        "body": {
          "username":"billy",
          "password":"issilly"
        }
      },
      "response": {
        "status": 200,
      }
    },
  ],
  "metadata": {
    "pactSpecification": {
      "version": "2.0.0"
    }
  }
}

The above is from https://codefresh.io/blog/how-to-test-microservice-integration-with-pact/

There have been several versions of the Pact format (1.0, 1.1, 2.0).

Pact provides a fluent API for service consumers to define the HTTP requests they will make to a service provider and the HTTP responses they expect back.

Test separately

Beth Skurrie (@bethesque, of Melbourne, Victoria and https://github.com/realestate-com-au/pact ) explains in a [26:37] video on Sep 04, 2015 at [9:32] says “Pact mocks the provider when it the consumer makes calls and handles responses correctly. [9:45] While consumer tests are running, the interactions is recorded into a Pact file as a contract. That file is then used to interact with the real provider to ensure “test symmetry” that mock and real providers work the same way.

The process also supports the inverse scenario.

Pact Broker

Harry Winser (@hazz223, Pact Broker author), in VIDEO: Consumer Driven Contract Testing with Pact and Docker 18 Oct 2017 illustrates the flow of work in a Pact service [13:05] pact-test-flow-616x327-22025

[12:19] shows that Pact Provider contract tests can also be used in QA and Staging. pact-in-pipeline-567x250-18831 [19:09]

[15:20] Consumer tests pact-consumer-tests-650x362-27115

[20:03] Docker

Network Graph from Broker

[15:30] Because the Pact Broker knows about traffic, like Google, the Pact Broker can mine the data, to automatically generate a graph of relationships between services (who calls whom).

pact-network-graph-654x513.jpg

Pact Broker Install

There are several ways.

Dockerized

Use the sample Docker Compose setup at:

https://github.com/DiUS/pact_broker-docker/blob/master/docker-compose.yml

  1. Modify the docker-compose.yml file as required.

For a quick start with the Pact Broker and Postgres, we have an

    Run docker-compose up to get a running Pact Broker and a clean Postgres database

Now you can access your local broker:

Get IP of your running Docker instance

DOCKER_HOST=$(docker-machine ip $(docker-machine active)) curl -v http://$DOCKER_HOST # you can visit in your browser too!

NOTE: this image should be modified before using in Production, in particular, the use of hard-coded credentials

  1. Install Docker
  2. Instantiate a Postgress database.
  3. Look at https://github.com/DiUS/pact_broker-docker
  4. Instantiate a container using the Docker image at https://hub.docker.com/r/dius/pact-broker/tags/

    
     docker pull dius/pact-broker
    
  5. Start Docker image:

    
    docker run -d -p 80:80 pact-broker
    
  6. Re-run

Sample Pact exchange

First of all, if the consumer assumes the API would be based on posting a JSON document (being invoked from a web browser), make sure that the provider is not implemented using an application/x-www-form-urlencoded POST API.

Development of the interaction starts from the consumer side, with a test that at first fails until code is written to make the test pass (TDD here).

This contract is then provided to the Provider team to implement the provider service to fulfill the contract.

PROTIP: Contracts are readable by both people and software.

[11:04] Arrange, Pact, and Assert

This DSL from Guidelines:

Pact.service_consumer "Zoo App" do
  has_pact_with "Animal Service" do
    mock_service :animal_service do
      port 1234
    end
  end
end
   

Contract types:

  • Preconditions (“You must be this tall to ride”)
  • Postconditions (e=mc^2)
  • Variants

https://gist.github.com/bethesque/9d81f21d6f77650811f4 You will need ruby or the standalone mock service to run these examples.

gem install pact-mock_service

pact-mock-service help start

Pastel’s Law

[15:09] “Pastel’s Law” – stict in what is sent out, but responses allow extra keys for different consumers.

References

Ruby Pact wiki: github.com/realestate-com-au/pact/wiki

How to Test Microservice Integration with Pact 9 Oct 2017 by Anton Weiss

Pact 101 – Getting started with Pact and Consumer Driven Contract Testing blog 03/02/2016 By Ron Holshausen

https://app.pluralsight.com/library/courses/code-contracts/table-of-contents Code Contracts</a> 1h 51m video course released 17 Jul 2012 by John Sonmez provides an introduction to Code Contracts in the Microsoft .NET Framework.

Martin Fowler on Microservice Testing ( http://martinfowler.com/articles/microservice-testing/#definition Article)

Introduction to consumer-driven contracts with Pact (http://dius.com.au/2016/02/03/microservices-pact/ Article)

Deploy with Confidence! - Ron Holshausen (https://www.youtube.com/watch?v=h-79QmIV824 Video April 6, 2016.

Integrated tests are a scam - J.B. Rainsberger (https://vimeo.com/80533536 Video / http://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam Article)

Verifying Microservice Integrations with Contract Testing - Atlassian (https://www.youtube.com/watch?v=-6x6XBDf9sQ&feature=youtu.be Video)

Escape the integration syrup with contract tests by Stefan Smith ([45:11] Video at Agile on the Beach 2015

https://www.youtube.com/watch?v=MDydAqL4mYE Consumer Driven Contracts and Your Microservice Architecture</a> [51:01] at Devoxx by Marcin Grzejszczak (@MGrzejszczak) and Josh Long (at Pivotal).

https://www.youtube.com/watch?v=sAAklvxmPmk&t=540s by Marcin Grzejszczak

Social

Twitter: @pact_up #contract-tests

Gitter: Join the chat at https://gitter.im/realestate-com-au/pact where Beth talks with Kevin Meiresonne, dengayevskiy, etc.

Stackoverflow: https://stackoverflow.com/questions/tagged/pact ruby pact questions or general pact questions

https://groups.google.com/forum/#!forum/pact-dev Google users group: for ongoing discussions rather than questions

More