Wilson Mar bio photo

Wilson Mar

Hello. Hire me!

Email me Calendar Skype call 310 320-7878

LinkedIn Twitter Gitter Instagram Youtube

Github Stackoverflow Pinterest

Achieve behavior-driven development (BDD) by automating Gherkin test specs using Pytest

US (English)   Español (Spanish)   Français (French)   Deutsch (German)   Italiano   Português   Cyrillic Russian   中文 (简体) Chinese (Simplified)   日本語 Japanese   한국어 Korean

Overview

After following this hands-on tutorial manually, you would be able to add to your resume:

Automated testing of BDD based on Gherkin using Python-based pytest-bdd installed using a Bash script. Integrated libraries for test coverage analysis, data-driven tests, and localization verification.

Video tutorials

  1. First, learn about the basic concepts about Pytest.

    Andrew Knight (@automationpanda, AutomationPanda.com) gradually presents, in a logic sequence and with quizzes 9 videos at Applitools’ Test Automation University (TAU).

    Behavior Driven Python with pytest-bdd

    Automation of install and run

    My value-add here is writing a Bash script (below) that automatically installs what is needed and runs the test on a public website under test:

    https://github.com/wilsonmar/tau-pytest-bdd

    This is a fork of Andrew’s repo in order to avoid future possible breaking changes in Andrew’s upstream repo. I occassionally sync with it and reconcile changes in my forked version.

    https://github.com/AndyLPK247/tau-pytest-bdd

Bash Script

  1. Highlight and copy this command: TODO:

    ... bash https://github.com/wilsonmar/tau-pytest-bdd.sh

    Note the parameters at the end of the command above:

    -v -V

  2. Open a Terminal.
  3. Navigate to a folder and paste the command to execute it.

    Note the script uses Bash coding techniques described at: https://wilsonmar.github.io/bash-scripts

The text below describes what the script above does, if you do it manually instead,

  1. Clone the repo from Andrew:

    git clone https://github.com/AndyLPK247/tau-pytest-bdd.git

  2. Fork it using hub command

    git remote add upstream https://github.com/AndyLPK247/tau-pytest-bdd.git

  3. Be inside virtualenv

  4. Install pre-requsities referencing file pipfile:

    pipenv install

    The response:

    Installing dependencies from Pipfile.lock (f55e24)…
      🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 19/19 — 00:00:07
    To activate this project's virtualenv, run pipenv shell.
    Alternatively, run a command inside the virtualenv with pipenv run.
    

    NOTE: This is instead of running individual “pip install” commands:

    pip install pyenv
    pip install -U pytest
    pip install -U pytest-bdd
    pip install -U pytest-cov    # to generate code coverage reports
    pip install -U pytest-xdist  # to run tests in parallel
    

    Alternately, after git cloning, install: https://github.com/hchasestevens/fault-localization

  5. Activate

    pipenv shell

    The response should be a change to the prompt, such as:
    (tau-pytest-bdd) bash-5.0$

    Launching subshell in virtual environment…
    bash-5.0$  . /Users/wilson_mar/.local/share/virtualenvs/tau-pytest-bdd-YNf2NFbA/bin/activate
    (tau-pytest-bdd) bash-5.0$ 
    

    Run individual test

  6. The script runs a specific test while at the repo’s root folder:

    pipenv run python -m pytest

    The response begins with:

    Creating a virtualenv for this project…
    Pipfile: /Users/wilson_mar/gits/wilsonmar/tau-pytest-bdd/Pipfile
    Using /usr/local/bin/python3 (3.7.6) to create virtualenv…
    ⠏ Creating virtual environment...Already using interpreter /usr/local/opt/python/bin/python3.7
    Using base prefix '/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7'
    New python executable in /Users/wilson_mar/.local/share/virtualenvs/tau-pytest-bdd-YNf2NFbA/bin/python3.7
    Also creating executable in /Users/wilson_mar/.local/share/virtualenvs/tau-pytest-bdd-YNf2NFbA/bin/python
    Installing setuptools, pip, wheel...
    done.
    Running virtualenv with interpreter /usr/local/bin/python3
    ✔ Successfully created virtual environment! 
    Virtualenv location: /Users/wilson_mar/.local/share/virtualenvs/tau-pytest-bdd-YNf2NFbA
    /Users/wilson_mar/.local/share/virtualenvs/tau-pytest-bdd-YNf2NFbA/bin/python: No module named pytest
    
  7. The script runs a specific test while at the repo’s root folder:

    pipenv run python -m pytest test_web_steps.py
  8. A new window pops up and disappears.
  9. The response on the Terminal (with … elipses replacing long strings):

    ======...====== test session starts ======...======
    platform darwin -- Python 3.7.6, pytest-4.4.1, py-1.8.0, pluggy-0.9.0
    rootdir: /Users/wilson_mar/gits/wilsonmar/tau-pytest-bdd/tests/step_defs
    plugins: bdd-3.1.0
    collected 2 items                                 
     
    test_web_steps.py .                             ...[100%]
    
    

======…====== 2 passed in 23.64 seconds ======… (tau-pytest-bdd) bash-5.0$ pipenv run python -m pytest test_web_steps.py ======…====== test session starts ===========…====== platform darwin – Python 3.7.6, pytest-4.4.1, py-1.8.0, pluggy-0.9.0 rootdir: /Users/wilson_mar/gits/wilsonmar/tau-pytest-bdd/tests/step_defs plugins: bdd-3.1.0 collected 2 items   test_web_steps.py . …[100%]   ======…====== 2 passed in 19.80 seconds ======…====== (tau-pytest-bdd) bash-5.0$ </pre>

### Test coverage

  1. After the test finishes, look at the test log/report and test coverage report.

    View test code

  2. Additionally, some want to install files to enable reference from within IDEs such as PyCharm, Eclipse, VSCode, etc.

    Alternative runs

  3. To run all tests defined:

    pipenv run python -m pytest
  4. To run all “web” tests (not “api” tests):

    pipenv run python -m pytest -k "web"

“test” folder contents

In the root of the repo is file cucumbers.py file, which defines the subject being tested (a basket of cucumbers).

In your mind, substitute the word “cucumber” with a “batch of invoices”, or whatever else your own application manages.

In the repo is a “test” folder, which in an integrated Agile team would be inside the source code repo of the app being tested.

### Folder structure for Pytest-bdd

Under the sample test folder, folders “features” and “step_defs” is for use by the pytest-bdd framework.

“pytest-bdd” is used here because it is not a standalone framework like alternative Behave. Pytest-bdd is a plugin for pytest (https://docs.pytest.org/en/latest) and all of its features and other plugins.

Like other BDD frameworks, pytest-bdd test scenarios are written within “.feature” files using the Gherkin language which uses specific vocabulary. Keywords in Gherkin can be in several spoken languages.

Since Gherkin can be userstood by non-technical people, it is a common way to communicate specifications among “Three Amigos” (business analysts, testers, developers).

A frameworks for BDD (“black box testing”) is very different from traditional testing frameworks like unittest and pytest which specify specific CSS markers coded by developers.

However, the combination provides a separation of concerns between test cases and test code. Gherkin steps may also be reused by multiple scenarios.

BTW, Within step_defs, file init.py (with no content) is for Python 3.3 and earlier * to look for submodules inside that directory.

  1. PROTIP: In your editor, open a feature file in one pane and its associated py file in another pane. Better yet, if you have two monitors, have “features” folder in one and “step_defs” files in another.

Pytest

Pytest introduces fixtures that sets up objects needed for testing, such as a SMTP port for sending email.

Fixtures can have scope to run once or multiple times.

Pytest can be augmented with plug-ins for code coverage, Flask integration, etc.

https://docs.pytest.org/en/latest/

https://automationpanda.com/python/

https://automationpanda.com/2018/09/27/book-review-pytest-quick-start-guide/

https://pragprog.com/book/bopytest/python-testing-with-pytest

VIDEO: Automated testing with pytest and fixtures</a> at PyGotham 2017

https://pytest-bdd.readthedocs.io/en/latest/index.html?#hooks Pytest-BDD – Hooks

https://github.com/AndyLPK247/tau-pytest-bdd/tree/chapter-9 GitHub Repo – Chapter 9

https://pytest-bdd.readthedocs.io/ Pytest-BDD Documentation

https://automationpanda.com/2017/03/14/python-testing-101-pytest/ Automation Panda - Python Testing 101: pytest

https://automationpanda.com/2018/10/22/python-testing-101-pytest-bdd/ Automation Panda - Python Testing 101: pytest-bdd

https://github.com/AndyLPK247/behavior-driven-python GitHub Repo – Python BDD Test Framework Examples

pytest-bdd

Pytest-bdd is a plug-in to Pytest.

https://github.com/pytest-dev/pytest-bdd

https://pytest-bdd.readthedocs.io/en/latest/

https://github.com/AndyLPK247/tau-pytest-bdd/tree/chapter-2

Packt Book: “Pytest quick start guide” by Bruno Olivera

Book: “Python Testing with pytest” by Brian Okken

### Hooks in conftest.py

To share common steps, fixtures, and BDD hooks between test modules.

per-directory hooks

Shared steps video

See https://docs.pytest.org/en/2.7.3/plugins.html

### Gherkin feature files

  1. Name each of the various features to be tested as a “feature” file.

    For pytest-bdd, a “features” folder is added to contain files with names ending with “.feature”. Such files are in the Gherkin language.

    @web duckduckgo at the top of the file enable features defined to be referenced by all scenarios by a filter. See this video.

    Numbers are in quotes so that the parser can recognize where parameter substitution can occur.

    This video</strong> shows the use of scenario outlines that references example tables.

    Feature files are not runnable as a program. So each step definition in Gherkin (Given, When, and Then) is “glued” to a Python function.

    Python step_defs

  2. For each step definition, specify a fixture decorated by a matching string in a Pytest step definition module to associate with Python code.

    A “step_defs” folder contains files containing Python code. The top line of each file starts with:

    See https://testautomationu.applitools.com/behavior-driven-python-with-pytest-bdd/chapter4.html

    from pytest_bdd import scenario, given, when, then
    from cucumbers import CucumberBasket
    from pytest bdd import parsers
    from functools import partial

    A fixture statement (with @) decorates functions.

    The @scenarios fixture creates functions for all … within the feature.

    @scenarios('../features/cucumbers.feature')

    That takes the place of specific code:

    @scenario("...")
    def test_add('../features/cucumbers.feature','Add ... to a basket'):
       pass
    @scenario("...")
    def test_remove('../features/cucumbers.feature','Remove ... from a basket'):
       pass
    @given("The basket has 2 tasks")

    @when(“…”)

    @then(“…”)

    The above steps can be re-used, which enables more rapid test development.

    Parameters

    Instead of static numbers and text strings, values can be replaced with parameters such as:

    @then(parsers.cfparse('the basket contains "{total:Number}" cucumbers', extra_types=dict(Number=int)))

    This means that references within features are within single quotes.

Alternatives

pytest-bdd test scenarios are written in Gherkin “.feature” files using plain language. Thus, it is a BDD test framework that is similar to Behave, Cucumber, and SpecFlow. For Python there is also radish, which extends Cucumber with constants and scenario loops.

There is also lettuce, which is not used much anymore.

Implement BDD with TDD: Using Python, Behave, and Mocking Mar 16, 2019

Book “BDD in Action” by John Ferguson Smart.

“The Cucumber Book” by Matt Wynne and Asiak Hellesoy.

Localization (L10N)

There are two ways to select another language (such as “es” for Spanish, “ko” for Korean, “de” for German, etc.):

A. English was selected in browser’s Preferences, but another the app displays another language.

B. Another language was selected in browser’s Perferences, and the app displays that language.

To simulate selecting another language in the browser’s Preferences in Firefox:

FirefoxOptions options = new FirefoxOptions();
options.addPreference("intl.accept_languages", language);
driver = new FirefoxDriver(options);

… and in Chrome:

HashMap<String, Object> chromePrefs = new HashMap<String, Object>();
chromePrefs.put("intl.accept_languages", language);
ChromeOptions options = new ChromeOptions();
options.setExperimentalOption("prefs", chromePrefs);
driver = new ChromeDriver(options);

More about Python

This is one of a series about Python:

  1. Python install on MacOS using Pyenv
  2. Python install on MacOS
  3. Python install on Raspberry Pi for IoT

  4. Test Python using Pytest BDD Selenium framework
  5. Test Python using Robot testing framework

  6. Python certifications
  7. Python tutorials
  8. Python coding notes
  9. Jupyter Notebooks provide commentary to Python

  10. Pulumi controls cloud using Python, etc.
  11. Microsoft Azure Machine Learning makes use of Python
  12. Testing AI uses Python code

  13. Python REST API programming using the Flask library
  14. Python coding for AWS Lambda Serverless programming
  15. Streamlit visualization framework powered by Python
  16. Web scraping using Scrapy, powered by Python
  17. Neo4j graph databases accessed from Python