Wilson Mar bio photo

Wilson Mar

Hello. Hire me!

Email me Calendar Skype call 310 320-7878

LinkedIn Twitter Gitter Instagram Youtube

Github Stackoverflow Pinterest

Maximize user productivity


Overview

This article is a hands-on, step-by-step tutorial about load testing an SAP Fiori 3 sample app using Flood.io Elements. We setup the scripts to run automatically when changes to a sample app is pushed into a GitHub repo. Covered are setup and interpretation of APM (Application Performance Monitoring) metrics about application health. Results of light load tests trigger an email alert if anomalies are identified.

An introduction to SAP Fiori is at https://www.tutorialspoint.com/sap_fiori/sap_fiori_introduction.htm

The SAP Fiori UX is tuned to specified SAP Screen Personas.

Under the covers, Fiori apps make use of RESTful APIs connecting with SAP HANA in-memory databases.

A Google Chrome browser is usually used to run SAP Fiori v3 apps created using the Quartz theme running on the SAP EPM cloud.

In this diagram from the OpenSAP SAPUI5 class (which ran from May 8, 2019): sap-fiori-flow-900x428-21988.jpg

SAP Fiori elements such as the Overview Page or Analytical List allow ABAP teams to accelerate app development cycles while also maintaining those apps more effectively.

SAP’s APF (Analysis Path Framework) is a framework for configuring interactive, chart-oriented analytical drilldown apps to view and analyze key performance indicators (KPIs) from different data sources. It can create different analysis steps and filters for users to interactively navigate a flexible analysis path.

Demo apps

SAP has created Fiori v3 demo apps at:

https://sapui5.hana.ondemand.com/#/demoapps

  1. Employee Directory
  2. Shopping cart showcase app
  3. Quickstart
  4. Worklist demo app
  5. Browse a master-detail app.
  6. Basic Template template app
  7. Bulletin Board
  8. Manage Products
  9. Walkthrough
  10. Browse Orders showcase app
  11. Icon Explorer
  12. Theme Parameter Toolbox
  13. Ice Cream Machine is created with the ‘Ice Cream Machine’ tutorial contains various controls, including Generic Tiles, Micro Charts, and a Process Flow. It uses control Library:sap.suite.ui.commons
  14. Manage Sales Orders (SAP Fiori Elements) showcase app
  15. Shop Administration Tool showcase app Backoffice tool, built with the sap.tnt.ToolPage
  16. UI Adaptation at Runtime SAPUI5 Flexibility Demo App
  17. UI Adaptation at Runtime for SAP Fiori Elements SAPUI5 Flexibility Demo App
  18. Worklist (FLP) template app
  19. Master-Detail (FLP) template app Best-practice starting point for a master-detail app, using SAP Fiori launchpad features using the Control Library:sap.m
  20. Chart Demo App - Miscellaneous Demo App to visualize data easily using charts controls of the sap.viz library

To view the code or run locally, download from the sample apps page.

BDD integration tests

Some demo apps are listed with a link to “Integration tests” for BDD (Behavior Driven Development) tests written in the Gherkin language.

Component Tests

Some of the demo apps are presented with a link to OPA5 (One Page Acceptance Tests) for that app.

OPA5 is a JavaScript-based API for controlling SAPUI5 controls used by “component-level tests” as illustrated by Mike Cohn’s book “Succeeding with Agile” and described by this blog.

sap-fiori-pyramid-812x441

See Simulating User Interactions on Controls

So it can be used with JavaScript unit test frameworks such as QUnit, Sinon, or Jasmine.

OPA5 handles asynchronous exchanges using a polling mechanism to test user interactions, inner-app navigation, integration with SAPUI5, and data binding as well as navigation.

However, OPA5’s limitations include:

  • Screen captures to image files
  • Testing across more than one page
  • Remote test execution
  • End-to-end tests (due to authentication issues and fragility of test data)

However, Flood.io element is able provide the above features.

Analyze demo code

Most of the packages contain a package.jason which Node uses to download dependencies. The ui5.yaml defines metadata for UI5 engine. For example:

specVersion: '1.0'
metadata:
  name: BasicTemplate
type: application

The WebApp folder holds the app’s assets.

  • file manifest.json
  • index.html invokes the app
  • file Component.js is a JavaScript program file that runs this.getRouter().initialize();
  • test.html
  • folder test contains both unit and integration folders qunit
  • folder controller contains the Home.controller.js file which invokes …
  • folder model containing file formatter.js and models.js
  • folder css contains the style.css file
  • folder view contains App.view.xml and Home.view.xml which references {i18n>title within …
  • folder i18n containing the i18n.properties that are usually defined for a specific company

Python

The Salesforce chapter of Pluralsight video course Business Automation with Python [1h 1m] by Eduardo Feitas shows automation of Salesforce Opportunity webform data entry by Python coding using Visual Studio Code.

SAPUI5 internals

The demo apps are built using libraries from OpenUI5.org bootstrapped into HTML like this:

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8">
    <title>Quickstart</title>
    <script id="sap-ui-bootstrap"
        src="https://openui5.hana.ondemand.com/resources/sap-ui-core.js"
        data-sap-ui-theme="sap_belize"
        data-sap-ui-libs="sap.m, sap.ui.layout, sap.tnt"
        data-sap-ui-resourceroots='{"Quickstart": "./"}'
        data-sap-ui-async="true">
    </script>
    <script>
        sap.ui.getCore().attachInit(function () {
            sap.ui.require([
                "sap/ui/core/mvc/XMLView"
            ], function (XMLView) {
                XMLView.create({viewName: "Quickstart.App"}).then(function (oView) {
                    oView.placeAt("content");
                });
            });
        });
    </script>
</head>
<body class="sapUiBody" id="content"></body>
</html>

The above code is also incorporated into apps created using the SAP BUILD visual prototyping tool.

Documentation on SAPUI5 for Fiori 2.0.

https://developers.sap.com/tutorials/xsa-e2e-sapphire-cloud-native.html

Why perftest a cloud app?

Speed performance (productivity of users) is claimed as a key advantage for moving to SAP Fiori.

sap-evolution_demokit-835x400-25263.png

Traditional JMeter Java or LoadRunner C scripts were designed to mimic the exchange of static HTML between user client and web server. Such client programs pretend to be hundreds of actual users on a single server because it generates requests but does not do much with responses from web servers.

Web apps are built for users who access internet browsers such a Google Chrome.

But “modern” apps are built using UI framework libraries such as Angular, React, Vue. Such frameworks have JavaScript perform actions that operate within user client browser programs that dynamically alter the HTML and CSS processed by the browser.

Traditional scripts require custom coding to process ASP.NET apps which exchanged an “ACTIVE STATE” containing instructions on how forms are processed. Such complications can be avoided when crafting scripts that focus on browser JavaScript rather than interpreting HTML protocol.

Flood.io can run scripts written to run using JMeter and Gatling which read HTTP.

VIDEO: Continuous Load Testing for SAP Flori April 3, 2019 [9:50] by Kevin Dunne

Flood.io Element

Flood.io can also run its proprietary “Element” protocol TypeScript (JavaScript) to control UI components in the DOM of each virtual user.

Instead of WebDrive executables for each browser (used by Selenium), Flood Element launches and controls for each user a browser instance of Google Chrome using Google’s Puppeteer automation library used by Google Lighthouse within Developer Tools on Google Chrome browsers.

Flood Elements runs on both its cloud environment and (through a component) on-premises.

Either way,

  1. Obtain an account at:

    https://flood.io/load-performance-testing-tool/free-load-testing-trial

    Use your GitHub or Google account for authentication.

  2. Log in at:

    https://app.flood.io/login

  3. Define a project.

Flood Element scripts can be run online or using the Element CLI.

Run Element demo script online

Flood can run scripts stored in GitHub.

  1. At https://element.flood.io scroll down to the white-on-black are containing tabs “flood-challenge   magento   navigating   test-data   waiting   woocommerce   youtube”.

  2. Click on “youtube” tab.

    PROTIP: Examples are defined in https://github.com/flood-io/load-testing-playground/tree/master/element

  3. Click the blue “Run example on flood”.

Run SAP demo script online

To run a script stored in a custom GitHub repo:

  1. Select Test Type “Element”.
  2. Specify a Test Name.
  3. Upload script by dragging a file from a Finder.

    Jason Rizio of Flood.io has written a script to create a Sales Order on SAP Fiori at
    https://github.com/flood-io/element/blob/beta/examples/sap-fiori/SAP4Hana_createSalesOrder.ts. Viewing the file:

    • .ts is for TypeScript.
    • export const settings defines values for variabls controlling the run.
    • export default () => { contains the various steps:
    1. Visit S/4 Hana Trial Login
    2. Create Sales Order
    3. Sales Order Entry - Enter Details
    4. Sales Order Entry - Add Item
    5. Sales Order Entry - Save
    6. Sales Order Entry - Back to Dashboard
  4. Click CONTINUE.
  5. In “Configure Load”, drag the number of browsers to 1 browser per region.
  6. Select a region closest to you.
  7. Click on Timing: Test Duration for 4 minutes.
  8. Read the Summary to confirm.
  9. Click “LAUNCH TEST”.

Flood CLI

To install the Flood CLI “element”, there are several alternatives. yarn and npm can be used instead of brew installer described below.

  1. On Macs, open a Terminal, navigate
  2. Hightlight the line below to paste on your Terminal to both install and update your .bash_profile file:

    bash ???

    NOTE: Run it instead of manually typing Flood CLI using brew usin

    brew install flood-io/taps/element

    The response (as of this writing), “….” indicates substitution of irrelevant lines:

    Updating Homebrew...
    ==> Auto-updated Homebrew!
    Updated 4 taps (homebrew/core, homebrew/cask, homebrew/services and caskroom/versions).
    ==> New Formulae
    ....
    ==> Updated Formulae
    ....
    ==> Deleted Formulae
    ....
    ==> Tapping flood-io/taps
    Cloning into '/usr/local/Homebrew/Library/Taps/flood-io/homebrew-taps'...
    remote: Enumerating objects: 35, done.
    remote: Counting objects: 100% (35/35), done.
    remote: Compressing objects: 100% (32/32), done.
    remote: Total 35 (delta 29), reused 4 (delta 2), pack-reused 0
    Unpacking objects: 100% (35/35), done.
    Tapped 32 formulae (90 files, 63KB).
    ==> Installing element from flood-io/taps
    ==> Installing dependencies for flood-io/taps/element: icu4c and node
    ==> Installing flood-io/taps/element dependency: icu4c
    ==> Downloading https://homebrew.bintray.com/bottles/icu4c-64.2.mojave.bottle.tar.gz
    ==> Downloading from https://akamai.bintray.com/e8/e858556120acc0c2d52b8fb572b677856724cc28d24a7e1b2762d458a2977c8e?__gda__=exp=1558943434
    ######################################################################## 100.0%
    ==> Pouring icu4c-64.2.mojave.bottle.tar.gz
    ==> Caveats
    icu4c is keg-only, which means it was not symlinked into /usr/local,
    because macOS provides libicucore.dylib (but nothing else).
     
    If you need to have icu4c first in your PATH run:
      echo 'export PATH="/usr/local/opt/icu4c/bin:$PATH"' >> ~/.bash_profile
      echo 'export PATH="/usr/local/opt/icu4c/sbin:$PATH"' >> ~/.bash_profile
     
    For compilers to find icu4c you may need to set:
      export LDFLAGS="-L/usr/local/opt/icu4c/lib"
      export CPPFLAGS="-I/usr/local/opt/icu4c/include"
     
    For pkg-config to find icu4c you may need to set:
      export PKG_CONFIG_PATH="/usr/local/opt/icu4c/lib/pkgconfig"
     
    ==> Summary
    🍺  /usr/local/Cellar/icu4c/64.2: 257 files, 69.2MB
    ==> Installing flood-io/taps/element dependency: node
    ==> Downloading https://homebrew.bintray.com/bottles/node-12.3.1.mojave.bottle.tar.gz
    ==> Downloading from https://akamai.bintray.com/4d/4ded0c82859da5c88b37e4295e95dd3a30e88d3a399e02ffa3592e3d3a41d5c6?__gda__=exp=1558943468
    ######################################################################## 100.0%
    ==> Pouring node-12.3.1.mojave.bottle.tar.gz
    ==> Caveats
    Bash completion has been installed to:
      /usr/local/etc/bash_completion.d
    ==> Summary
    🍺  /usr/local/Cellar/node/12.3.1: 4,505 files, 53.8MB
    ==> Installing flood-io/taps/element
    ==> Downloading https://registry.npmjs.org/@flood/element-cli/-/element-cli-1.0.4.tgz
    ######################################################################## 100.0%
    ==> npm install -ddd --global --build-from-source --cache=/Users/wilsonmar/Library/Caches/Homebrew/npm_cache --prefix=/usr/local/Cellar/el
    Error: Failed changing dylib ID of /usr/local/Cellar/element/1.0.4/libexec/lib/node_modules/@flood/element-cli/node_modules/puppeteer/.local-chromium/mac-575458/chrome-mac/Chromium.app/Contents/Versions/69.0.3494.0/Chromium Framework.framework/Versions/A/Libraries/libswiftshader_libGLESv2.dylib
      from @rpath/libswiftshader_libGLESv2.dylib
     to /usr/local/opt/element/libexec/lib/node_modules/@flood/element-cli/node_modules/puppeteer/.local-chromium/mac-575458/chrome-mac/Chromium.app/Contents/Versions/69.0.3494.0/Chromium Framework.framework/Versions/A/Libraries/libswiftshader_libGLESv2.dylib
    Error: Failed to fix install linkage
    The formula built, but you may encounter issues using it or linking other
    formulae against it.
    ==> Summary
    🍺  /usr/local/Cellar/element/1.0.4: 11,993 files, 257.5MB, built in 3 minutes 6 seconds
    ==> `brew cleanup` has not been run in 30 days, running now...
    ....
    ==> Caveats
    ==> icu4c
    icu4c is keg-only, which means it was not symlinked into /usr/local,
    because macOS provides libicucore.dylib (but nothing else).
     
    If you need to have icu4c first in your PATH run:
      echo 'export PATH="/usr/local/opt/icu4c/bin:$PATH"' >> ~/.bash_profile
      echo 'export PATH="/usr/local/opt/icu4c/sbin:$PATH"' >> ~/.bash_profile
     
    For compilers to find icu4c you may need to set:
      export LDFLAGS="-L/usr/local/opt/icu4c/lib"
      export CPPFLAGS="-I/usr/local/opt/icu4c/include"
     
    For pkg-config to find icu4c you may need to set:
      export PKG_CONFIG_PATH="/usr/local/opt/icu4c/lib/pkgconfig"
     
    ==> node
    Bash completion has been installed to:
      /usr/local/etc/bash_completion.d
    

</pre>

In the above, note Brew installer makes use of npm.

  1. Verify install

    element --version

    The response (at time of writing):

    1.0.4
    If you need to have icu4c first in your PATH run:
      echo 'export PATH="/usr/local/opt/icu4c/bin:$PATH"' >> ~/.bash_profile
      echo 'export PATH="/usr/local/opt/icu4c/sbin:$PATH"' >> ~/.bash_profile
     
    For compilers to find icu4c you may need to set:
      export LDFLAGS="-L/usr/local/opt/icu4c/lib"
      export CPPFLAGS="-I/usr/local/opt/icu4c/include"
     
    For pkg-config to find icu4c you may need to set:
      export PKG_CONFIG_PATH="/usr/local/opt/icu4c/lib/pkgconfig"
    
  2. Clone the Element script repo to your local machine:

  3. Execute element CLI to run the example on your local machine:

    npx @flood/element-cli run ./element/fiori/example.ts --no-headless

    Before running this, you may want to install metrics collection

APM

This tutorial combines assets from this article about analyzing a sample app within a Docker image. The app is familiar to many functional testers because it is used in a popular testing book by Dave Hoffner becuase the app purposely introduces problematic UI features which perform badly.

SAP Fiori was first introduced in 2013 as a replacement for the venerable linear, menu-driven SAP GUI UX.

Rather than typing commands and scrolling through complex menus and opening multiple window, Fiori app users operate touch screens such that tasks started on a desktop can be completed on a mobile smartphone. This video shows converting a common accounts receivable task in SAP Simple Finance achieved a 64% reduction in duration and 71% fewer clicks.

Code Editor setup

Visual Studio Code

auto-complete

spelling

Show image capture

APM Setup

Add metrics to APM in order to show transactions that APM tools do not track on their own based on capture of HTTP traffic.

New Relic is free for Flood IO users and free for monitoring pre-production apps on Heroku. We’re about to demonstrate some of New Relic Professional’s features including detailed transaction traces and code level visibility. See https://flood.io/blog/performance-tuning-with-flood-io-and-new-relic-part-1/

We take advantage of the Dynatrace free license.

https://flood.io/blog/performance-tuning-with-flood-io-and-new-relic-part-1/

Scripting

Rather than maintaining several scripts, it is usually more productive on the long run to edit a single load testing script project. TODO: Feature flags are added in the script to control what each run exercises among different aspects of the infrastructure or app. Below is a typical sequence of concerns:

  1. “Website not available” page to stress the load balancer
  2. Just Landing page to stress the network infrastrure
  3. User registration and login to stress authentication infrastructure
  4. User menu navigation to stress web server infrastructure such as logging
  5. User browsing to stress random database access efficiency
  6. Report generation to stress sequential database access efficiency
  7. User data submission to stress queue overflow detection and state handling needed for scaling

A “workload model” defines the mix of the above to mimic load expected in actual production usage.

So that all requests can go to a different URL on a different environment with only a single change, scripts are created using TODO: parameters that can be specified externally when invoking a run.

To replicate the additional activity within a browser, Flood.io Elements browser-level scripts are written in modern JavaScript or TypeScript, which generates JavaScript. Scipts containing Typescript have a file extention of “.ts”, as in “ test.ts”.

TODO: capture image of the screen at specific points during users’ workflow.

To insert think times, Flood’s Test Builder tool uses by default a “Gaussian Random Timer” which mimics a random amount of time. See https://flood.io/blog/6-common-performance-testing-mistakes/

Verification of responses from the server, such as page titles, text expected. This would catch mistakes in either script coding or issues with the app.

Capture of response text to “correlate”

Variation of data to make requests unique to mimic database.

App Code Profiling

run to analyze code running on servers during load tests.

memory leaks

Test runs

Decide how long you’d like to run the script and at what scale (number of users).

Anomaly detection

To identify “anomalies” that require attention, statistics from “known good” runs are designated as “baseline” for use in comparisons against later runs.

too many images or images too large (not compressed)

look at the graph. At this point the graph contains a treasure trove of info that you’ll have to learn to interpret.

References

This article referenced these websites: