Protractor (automated functional testing)

Using Jasmine to verify AngularJs web app UI based on Selenium (behavior) driven by Gherkin specs run by Cucumber

NOTE: Content here are my personal opinions, and not intended to represent any employer (past or present). “PROTIP:” here highlight information I haven’t seen elsewhere on the internet because it is hard-won, little-know but significant facts based on my personal research and experience.

TL;DR Summary

Protractor was written by the team within Google who created the Angular testing framework in order to automate its testing. Unlike traditional static HTML, client browsers running Angular apps dynamically generates HTML and JavaScript on the fly. Protractor adds “accessors” to Angular models, bindings, ng-options, and finding elements inside ng-repeat. Since static or generated on the client, HTML is stored in a DOM (Document Object Model) web browsers maintain for each session. And Protractor grabs specific HTML elements from within each DOM.

The first version of Protractor was released in July, 2013. Google continues to support the work.

Even if a website is not using Angular, many still prefer coding Protractor rather than Selenium alone. One of the major advantages pointed out on https://protractor.angular.io (formerly protractortest.org) is that Protractor runs tests quicker than Selenium because it optimizes the need for manually adding (usually arbitrary) “sleep” and “wait” commands in test scripts. JavaScript is asynchronous (not sychronous like Java). Protractor can advance to the next step in the script even if a promise is pending.

Overall, Protractor requires less coding than Selenium and provides additional functions to make querying of elements easier.

Daniel Amorim, in his 17 Apr 2014 “Testing AngularJS apps with Protractor”, showed this diagram:

Protractor is a NodeJs application written in JavaScript or TypeScript (invented by Microsoft). It makes calls to a WebDriver which controls a web browser such as Microsoft’s Edge, Apple’s Safari, Google’s Chrome, Mozilla’s Firefox, etc.

Other components are described as we install each:

Install prerequisites

1. Install package manager for your operating system so that in the future it recognizes the need for updates and does it with one command.

   On Macs, install Homebrew using the default Ruby instance:

   /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

   On Windows, install Chocolatey from chocolatey.org. See this StackOverflow thread.

2. Install browser UI apps for cross-browser testing. On Macs:

   brew install --cask google-chrome
   brew install --cask firefox
   

   Alternately, on Windows:

   choco install googlechrome -y
   choco install firefox -y
   

3. Open a Terminal session on Mac or cmd session on Windows.

4. Install Node and NPM (Node Package Manager) globally:

   On Windows, see this blog. The use of choco should add to the PATH variable:

   C:\Program Files\nodejs

5. On Windows, go to Computer Properties to add a System Variable NODE_HOME to point to the same path above.

6. Use NPM to install Protractor from https://github.com/angular/protractor

   npm install -g protractor --save-dev
   protractor --version
   

   -g installs globally for accessibility from all folders rather than just the node_module folder of the current folder.

   –save-dev adds an entry within the ““devDependencies” section of package.json instead of “dependencies”. This is because the test suite is not needed to run the app in production state.

   At the end of sample output should be:

   Version 5.4.2

   NOTE: There is no need to install Selenium and WebDrivers as it comes within the Protractor install.

   BTW, the same command is used to update to the latest version of Protractor.

   The above only needs to be done once, and occassionally to update.

7. To update WebDriver “plug-in” for each internet browser needed to take over the keyboard and mouse.

   webdriver-manager update
   webdriver-manager version
   

   Sample output:

   [08:06:39] I/version - webdriver-manager 12.1.1
   

   NOTE: There is a WebDriver that is “headless” and does not involve iteraction on a browser UI: PhantomJS uses GhostDriver to run tests in Headless mode.

   • https://code.google.com/p/chromedriver/
     
     

8. Install Git using a package manager. On Mac, use Homebrew:

   brew install git
   

   Alternately, On Windows:

   choco install git.install -y
   

   -y enables you to skip confirmation prompts.

9. Install Java8 using a package manager. On Mac, use Homebrew:

   brew tap caskroom/versions
   brew install --cask java8
   

   You’ll need to input your password.

   Alternately, on Windows:

   choco install jdk8 -y
   

   WARNING: You must specify version 8 because the default version 11 onward have a different licensing by Oracle.

10. Verify Java install:

    java -version
    

    Sample response:

    java version "1.8.0_162"
    Java(TM) SE Runtime Environment (build 1.8.0_162-b12)
    Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)
    

1, Exit your command instance and enter it again so installations take.

1. If you don’t have a projects folder, create one under your user home folder. On Bash mine is:

   cd ~
   mkdir gits
   cd gits
   

   Others use “project”.

Sample app and tests

1. Identify a repository of sample Protractor code you want to use. It’s best to use a sample repo in GitHub that works on a sample app running in the cloud. There are several:

   Nate Taylor’s sample test assets at https://github.com/taylonr/intro-to-protractor runs against the sample app at https://meanjs.herokuapp.com as described in his “Protractor Introduction” video class released March 10 2015.

   The sample app repo includes the MEAN.JS stack which consists of MongoDB, Node.js, Express API framework, and AngularJS.

   But the site is no longer active and there are errors when building the app.

   Another one is “http://www.thetestroom.com/”. But the full URL specified in the script associated with it is no longer available.

   So let’s not use them.

2. Alternately, in a browser view the sample app running.

   

   Write down steps to run through the app, such as:

   1. Load http://juliemr.github.io/protractor-demo
   2. Click on the left box and enter 2.
   3. Ignore the operator as “+”.
   4. Click on right box and enter 2.
   5. Click on “Go!” to perform the calculation.
   6. Verify the negative test case of 5, which makes the test fail.
   7. Change the test result to 4.
   8. Run again to verify that the result is 4, which makes the test pass.
      
      

   Fork test assets repository to your account

3. View the repository of a sample app, containing test assets targeting its sample site:

   https://github.com/juliemr/protractor-demo

4. Notice the repo has NOT been updated since 2015. So it will likely encounter deprecation and security errors if built locally.

   There are also Pull Requests and Issues pending for a long time.

5. Fork the repository online in GitHub so that you can make changes.

6. Because git clone does not establish a folder for the account (just the repo), I manually create then cd into a folder for each GitHub account before I clone the repo. Several repos can have the same repo name.

   Since you forked the repo you would specify your own GitHub account name when cloning. For example, if your GitHub account is “wilsonmar”, then create a folder:

   cd ~/gits
   mkdir wilsonmar
   cd wilsonmar
   pwd
   

   Alternately, if you didn’t fork the repo, create folder for account “juliemr”, the account holding our sample repo.

   Clone locally

7. Clone the sample repository using your own account name:

   git clone https://github.com/your account name/protractor-demo/
   cd protractor-demo
   

   Sample response:

   Cloning into 'protractor-demo'...
   remote: Enumerating objects: 150, done.
   remote: Total 150 (delta 0), reused 0 (delta 0), pack-reused 150
   Receiving objects: 100% (150/150), 112.96 KiB | 3.32 MiB/s, done.
   Resolving deltas: 100% (56/56), done.
   

8. List folders and files at the top of the repo:

   ls
   

   The response:

   LICENSE      README.md    app          howtos       package.json test
   

   File package.json specifies dependencies that Node will download before beginning runs.

   PROTIP: The “app” folder contains the app’s code. The “tests” folder contains tests associated with the app’s source code in the same repository. This is getting more common than separate teams which maintain separate repositories for app and test code.

9. Get into the test folder:

   cd test
   

   File server.js is the file specified to Node.js to begin processing.

   PROTIP: Create a test-suites folder to hold …spec.ts (TypeScript) files.

   PROTIP: Have a page-objects folder to define a folder for each page (login, etc.).

10. To download libraries to implement the package.json file:

    npm install
    

    Older libraries may encounter deprecated dependencies.

IDE to edit

If you prefer using an IDE, see my tutorial on text editors:

• Eclipse
• WebStorm - https://www.jetbrains.com/webstorm/
• Visual Studio Code - https://code.visualstudio.com/
• others
  
  

Then add code completions and other helpers.

To install plugins for autocomplete in Eclipse*

1. Pull down menu Help.
2. Find “Angularjs”.
3. Scroll down to select “AngularJs Eclipse 1.2.0” to click “Install”.
4. To verify, pull down Preferences to see “AngularJs” on the left pane.
5. Right-click on your project to select Configure, “Convert to Tern Project…”.
6. Under Tern, Modules, select “Protractor”. Apply and Close.
7. When you type “browser.” and press contrl+space to see autocomplete suggestions.

To install plugins for autocomplete in Microsoft Visual Studio Code*

To run Jasmine tests through WebStorm, follow WebStorm’s simple 13-step process to get it running. But note Jasmine isn’t a “first class citizen”. Jasmine can run only through the karma test driver. Follow more steps to be able to use WebStorm’s debugger.

Protractor script edits

1. If you need to navigate to a page which does not use Angular, you can* turn off waiting for Angular by setting before the browser.get:

   browser.waitForAngularEnabled(false);

   PROTIP: Remember the semi-colon to end each sentence.

Run Test

These are steps every time you run:

Protractor Config for Jasmine

Protractor is controlled by a config.js file such as this simple example:

// conf.js
exports.config = {
  framework: 'jasmine',
  seleniumAddress: 'http://localhost:4444/wd/hub',
  specs: ['local.chrome.spec.js']
}
   

The above specifies the use of the Jasmine JavaScript framework to run specs (short for specifications, or test case files). Jasmine (https://jasmine.github.io from Pivotal Labs) allows you to write both unit and functional tests. Alternately, there is a “cucumber” framework or Mocha

NOTE: The seleniumAddress URL shown here does not display on a browser because it is an “end point” that listens for API requests. But it does have a UI to display its sessions.

The seleniumAddress URL is to one of the options made available by Protractor:*:

1. seleniumServerJar - to start a standalone Selenium Server locally.
2. seleniumAddress - to connect to a Selenium Server which is already running.
3. sauceUser/sauceKey - to use remote Selenium Servers via Sauce Labs.
4. browserstackUser/browserstackKey - to use remote Selenium Servers via BrowserStack.
5. directConnect - to connect directly to the browser Drivers for Firefox and Chrome browsers.
   
   

PROTIP: In the file name specify where the run occurs and what browser. For example: “local.chrome.conf.js” for chrome run locally. Or “sauce.firefox.conf.js” for running Firefox on the remote “SauceLabs cloud.

The square “[ ]” brackets for specs:</tt> (notice the plural) means that a list of several spec.js files can be specified, separated by commas.

config.js for cloud runs

Examples of other configuration js files include:

• SauceLabs.com
• CrossBrowserTesting.com
• PerfectoMobile.com
  
  

config.js for reporting

1. Generate HTML report by installing Abhishek Yadav’s https://www.npmjs.com/package/protractor-html-reporter-2

   npm install -g protractor-html-reporter-2
   npm install -g jasmine-reporters
   

2. Add to the conf.js file:

   //HTMLReport called once tests are finished:
   // https://www.npmjs.com/package/protractor-html-reporter-2
    
   onComplete: function() {
     var browserName, browserVersion;
     var capsPromise = browser.getCapabilities();
    
     capsPromise.then(function (caps) {
        browserName = caps.get('browserName');
        browserVersion = caps.get('version');
        platform = caps.get('platform');
    
        var HTMLReport = require('protractor-html-reporter-2');
    
        testConfig = {
            reportTitle: 'Protractor Test Execution Report',
            outputPath: './',
            outputFilename: 'ProtractorTestReport',
            screenshotPath: './screenshots',
            testBrowser: browserName,
            browserVersion: browserVersion,
            modifiedSuiteName: false,
            screenshotsOnlyOnFailure: true,
            testPlatform: platform
        };
        new HTMLReport().from('xmlresults.xml', testConfig);
    });
    }
    

TODO: https://www.npmjs.com/package/jasmine-reporters

Start WebDriver

1. Start WebDriver to a port. It was installe with Protractor:

   webdriver-manager start 
   

   This response expected:

   INFO [SeleniumServer.boot] - Selenium Server is up and running on port 4444
   

   This runs on your Terminal session, so on Macs, press control+C to stop the session.

2. Use your cursor to open another Terminal/Command session.

3. View processes in the new Terminal session:

   ps -a
   

   Among the response:

   12260 ttys000    0:00.60 node /usr/local/bin/webdriver-manager start
   12261 ttys000    0:01.31 /usr/bin/java -Dwebdriver.chrome.driver=/usr/local/lib
   

   Alternately, start in background by adding a “&” to the end of the command:

   webdriver-manager start &
   

   Now you can continue on the same Terminal session while webdriver runs in the background.

   SeleniumAddress WebDriver UI

4. View WebDriver web page: On Mac Terminal, use the open command to open a browser at the URL specified:

   open http://localhost:4444/wd/hub/static/resource/hub.html
   

5. On Windows, click the URL link above to open in your default browser.

   

6. Click “Create Session” and select the browser:

   

Specs.js file

View a sample spec.js file that was specified in a config.js file, which gets (opens) a URL to expect the title to be as stated in “.toContain” method:

// spec.js
describe('Angular.io landing', function() {
  it('should have a title', function() {
    browser.get('https://angular.io');
    expect(browser.getTitle()).toContain('Angular');
  });
});
   

The above is the angular.io marketing page .(formerly AngularJs.org).

Another example is:

// spec.js
describe('Protractor Demo App', function() {
  it('should have a title', function() {
    browser.get('http://juliemr.github.io/protractor-demo/').then(function(){
       console.log("Executed.");
    });
    expect(browser.getTitle()).toContain('Super Calculator');
  });
});
   

// (double forward slashes) in front of a line comments it out from being read.

describe describes a test suite group (category) of tests.

it specifies a spec (specification) or test case.

The two are referenced in run reports

browser.get specifies the URL handled by the browser.

.then forces conditional execution of the console.log() so that it’s not random due to JavaScript exeuction being asynchronous.

expect statements specify verifications.

PROTIP: We recommend an incremental approach. First, run for just the URL appearing. Then add more test steps. Once you obtain a successful test, commit the changes to the team repo (off your laptop). This would make it easier to debug.

So first let’s run the file as-is without changes.

Run from inside Eclipse

1. When using Eclipse on Windows*, navigate to copy the spec1.js file from where Protractor was installed. For example, if my user name is wilsonmar, then it would be
   

   C:\Users\wilsonmar\AppData\Roaming\npm\node_modules

2. Eclipse needs to know the path to a “Main” file. That’s the cli.js file. So copy the whole protractor folder from under node_modules into the root of your test assets repository project’s folder.

   Then specify the Main file as: $(workspace_loc:/Js1/protractor/built/cli.js)

Alternately, if on Eclipse configured with the Protractor plugin:

1. Right-click on the Protractor project. Run As, Run Configurations…
2. Scroll to “Node.js Application” and click on it.
3. Click the “New Launch Configuration” icon at the top left of the dialog.
4. Click “Browse” and navigate to select your Protractor test asset repository as the Project.
5. Click “Main File” and navigate to where Protractor is installed, then the “cli.js” file.

Run from CLI

1. To invoke from a command-line Terminal:

   protractor local.chrome.conf.js
   

   A sample test output:

   1 tests, 1 assertion, 0 failures.
   

Test Runner Karma

Jasmine does not have a command line utility to run tests like Mocha:

mocha tests --recursive --watch
   

“tests” are where tests are located.

The recursive flag finds all files in subdirectories.

The watch flag reruns tests automatically when a change is detected in source or test files.

Mocha’s approach enable tests to be initited by a right-click on the spec folder within WebStorm.*

Jasmine users can use Karma, also written by the Angular team, at http://karma-runner.github.io.

Karma supports Mocha too.

This article reports the various ways to run various test runners. While mocha-parallel-tests are fast. But even though Jest (the testing platform developed by Facebook) and AVA are slower, their additional features may be worth the cost. Those features include snapshot testing and test coverage.

See https://raygun.com/blog/mocha-vs-jasmine-chai-sinon-cucumber/

Browser appearance

During test runs, browser windows are opened and closed by Protractor.

Code Coverage Reporter

The istanbul coverage reporter instruments ES5 and ES2015+ JavaScript code with line counters, to enable tracking of how well unit-tests exercise the codebase.

Identifying elements

Use Chrome Developer Tools to see HTML id and names.

Editing scripts

1. To use Protractor on an non-Angular.js website, access the webdriver instance directly with browser.driver. For example:

   browser.driver.findElement(by.id('username')).sendKeys('Jane');
   browser.driver.findElement(by.id('password')).sendKeys('1234');
   browser.driver.findElement(by.id('clickme')).click();
    
    browser.driver.wait(function() {
      return browser.driver.getCurrentUrl().then(function(url) {
        return /index/.test(url);
      });
    });
   

2. PROTIP: Instead of hard-coding username and password in the code, read a file from your user home folder to populate values in variables, such as:

   https://stackoverflow.com/questions/22312671/setting-environment-variables-for-node-to-retrieve/28821696#28821696

   browser.driver.findElement(by.id('password')).sendKeys(process.env.APP_PASSWORD);
   

   Alternately, read from a CSV file.

   See https://medium.com/@tacomanator/environments-with-create-react-app-7b645312c09d

Timings

• Add helper function to pause for 9 seconds (9000 milliseconds):

  browser.sleep(9000);

• Add wait helper function to pause until an event is detected.

• Capture the amount of time taken to do each step or series of steps.

Verifications

• Use driver.manage() to manage timeouts: implicitlyWait, pageLoadTimeOut, Manage Current Window: maximize, getPosition, Manage Cookies: addCookie, deleteCookie

• There is a findElement and plural findElements, isElementPresent

• Browser management functions: get, quit, close, executeScript, getTitle, getCurrentUrl

• UI actions: click, sendKeys, isDisplayed, isSelected, getAttribute, getText, clear

• Error messages

• To take a screen shot on error, use the takeScreenshot helper function.

• Same user & same page vs. other fields (title of page, field labels, field values)

• Same user but different pages

• Test different users (persona) to ensure those with different permissions can still do their job.

See https://github.com/abhishekkyd/WebDriverJS-examples

Sample app for testing

You need a sample app to test against, and sample scripts that test that site.

Test Gmail as sample app

The provided repository contains an automated test for email sending functionality via Gmail as outlined below:

1. Login to gmail
2. Compose and Send an email with an alert message for not having body
3. No verification is done in the code for any action like Login page displayed, compose email box etc
   
   

Your task is to implement below test case:

1. Login to account1 on Gmail
2. Compose an email with unique subject, body and attachment
3. Send it to account2
4. Login to account2
5. Open the email
6. Verify the subject, body and attachment name
   
   

Notes:

1. Account details should be configurable in params.
2. Improve the existing code base to the best of your knowledge and expertise. 3. Don’t include packages and additional auto generated files like node_modules. 4. The tests should pass

The tests above are called “e2e” or end-to-end from login.

1. To run the tests:

   npm run e2e

Object recognition

https://github.com/agilethought/inside-protractor-locators

Every protractor locator is formatted as (by.*), where * is the locator you have chosen to locate the element. Here is a list of the most common locators:

Page Objects

https://www.protractortest.org/#/page-objects

Additional Functions

Protractor adds a convenient “waitfor” functions and accessors (locators) by button text, partial button text.

Protractor enables “find” by a combination of CSS and text (get me all the divs with class ‘pet’ and text ‘dog’).

Protractor adds the “addLocator” function to add custom locators. For example, get elements by handlebars properties.

By.id By.css By.className By.linkText By.js By.name By.xpath By.tagName

   element(by.model(‘locator’));
   element(by.binding(‘locator’));
   element(by.repeater(‘locator’));
   

Cucumber

CucumberJS supports async programming concepts in scripts with TypeScript.

1. for Cucumber, we use Typescript, which adds additional (“object oriented programming” featuers) to JavaScript.

   A strongly typed superset of plain Javascript.

Mocha

The Mocha library (https://mochajs.org) appeared in 2011 with a different approach than Jasmine.

While Jasmine describes itself as having “batteries included,” meaning that it attempts to provide everything a developer needs in a test framework, Mocha instead aims to cover the basics and allow other developers to extend it with other frameworks, such as Chai and Sinon, which provides more sophisticated capabilities than Jasmine alone.

The syntax between Jasmine and Mocha/Chai are not that different. Where Jasmine is:

expect(group.validFrom).toEqual('2016-01-22T19:00:00+00:00');

Chai’s syntax:

expect(group.validFrom).to.equal('2016-01-22T19:00:00+00:00');

See https://medium.com/@praveenjanakarajan/jasmine-or-mocha-66942388b196

Chai

Mocha does not have a built in assertion library. So alternatives are Chai, should.js, expect.js, and better-assert.

Chai is often chosen as the assertion library with Mocha. Chai comes with three different assertion flavors:

• The expect style is similar to what Jasmine provides – a style from Behavior-Driven Development.
• “Should” uses a similar chained format, and is different only in style.
• “Assert” is a more “classical” format rooted in traditional TDD (Test-Driven Development)
  
  

Developers tend to choose the style with which they are most familiar.

Chai uses a “fluent” syntax where comparison operators can be chained together:

expect(foo).to.equal('foo')       // equality
   expect(foo).to.not.equal('foo')   // inequality
   expect(foo).to.be.a('string')   	 // type assertion
   

For example, if you want to write an expectation that verifies that calculator.add(1, 4) returns 5, see https://www.codementor.io/codementorteam/javascript-testing-framework-comparison-jasmine-vs-mocha-8s9k27za3

Test doubles

A “test double” library is used to replace one object with another for testing purposes, like actors being replaced with stunt doubles for dangerous action scenes during moviemaking. Or like a clone of an object.

In Jasmine, test doubles come in the form of “spies”. Each spy function replaces a function whose behavior you want to manipulate in a test while recording the results.

• Tell a spy to call the original function (the function it is spying on). By default, a spy will not call the original function.
• See how many times each spy was called
• See what arguments a spy was called with
• Specify a return value to force the code to go down a certain path
• Force a spy to throw an error
  
  

For sample coding, see https://raygun.com/blog/mocha-vs-jasmine-chai-sinon-cucumber/

Mocha does not come with a “test double” library. So Sinon is added.

Sinon breaks up test doubles into three different categories, each with subtle differences: spies, stubs, and mocks.

Fake server

One feature that Sinon has that Jasmine does not is a fake server.

Fakes are used to simulate external behaviors without actually making any external calls. It’s needed because unit tests should not make calls outside their scope to networks or databases. So test “fakes” are used to isolate a test from external dependencies.

More precisely, a fake server provides fake responses to AJAX requests made to specified URLs.

In summary, SinonJS is a more complete framework test double framework than Jasmine, including not only spies but also stubs and fakes.

Learning Resources

a. https://www.protractortest.org/#/tutorial
b. https://chercher.tech/protractor/
c. https://jasmine.github.io/2.0/introduction

To learn Angular, consider Thinkster Popular Guide, and Egghead Videos.

https://developers.perfectomobile.com/display/PD/Writing+Protractor+First+Test+Script

https://medium.com/@igniteram/e2e-testing-with-protractor-cucumber-using-typescript-564575814e4a Sep 14, 2016

https://spin.atomicobject.com/2014/12/17/asynchronous-testing-protractor-angular/

https://bridge360blog.com/2015/05/05/improving-protractor-tests-using-shared-functions-and-promises/

Project ideas

Add OpenCV and Tesseract