K6

Code JavaScript to do GUI & API performance tests locally and in their k6.io SaaS cloud

Overview

Install locally
Run Simplest Script Locally
extensions
Using Visual Studio Code
Run Script in k6 Cloud
API testing
Visualize Results
Customize k6 JavaScript
Config Prometheus to Grafana
Inject faults
Dashboard
References
Product comparisons
K6 Champions
More on Security

NOTE: Content here are my personal opinions, and not intended to represent any employer (past or present). “PROTIP:” here highlight information I haven’t seen elsewhere on the internet because it is hard-won, little-know but significant facts based on my personal research and experience.

K6 (at k6.io) has become a popular FOSS tool for performance testing using JavaScript coding.

k6 itself is a FOSS command-line tool to run performance (load) tests on local machines.

It was written in Go within loadimpact.com, which was acquired in May 2020 by Grafana Labs to compliment their Grafana dashboards and Prometheus data collection.

Thus, k6 documentation is at:

https://k6.io/docs/
https://grafana.com/docs/grafana-cloud/k6/

Like many other FOSS tools, Grafana makes money from its k6 cloud service in 21 “load zones” in AWS around the world.

The k6 cloud service is free for up to 50 VUs (virtual users) per test, with a limit of 50 tests per month.

Unlike JMeter and LoadRunner, the cloud edition provides a GUI that illustrates insights about each run.

Install locally

Install k6 locally:
```
brew install k6
```
Verify version installed:
```
k6 version
```
On an older Intel amd64 running Apple macOS:
```
k6 v0.48.0 (go1.21.5, darwin/amd64)
```

Run Simplest Script Locally

PROTIP: Ideally, an app development project’s github repo would contain a folder in the same repo as the app code (such as “k6”) so that changes in app code have corresponding test code.

In Terminal, cd to the folder, for example:
```
cd ~/Documents/GitHub/
```

Create a folder by cloning from my repo containing :

git clone git@github.com:wilsonmar/oss-perf.git --depth 1
cd oss-perf/k6

View the k6-test-ping.js which “pings” or lands on the test.k6.io web server:
```
import http from 'k6/http';
import { sleep } from 'k6';
export const options = {
  vus: 10,
  duration: '30s',
};
export default function () {
 http.get('http://test.k6.io');
 sleep(1);
}
```
The ‘k6/http’ specifies a built-in module imported to make requests using the HTTP protocol.

The function JavaScript object (called “vu code”) contains calls for getting the specified URL, then sleep for 1 second.

The options object (called “init code”) here specifies that 10 vus (virtual users) run for 30 seconds.

See https://k6.io/docs/using-k6/test-lifecycle/
To run using options in the init code:
```
time k6 run k6-test-ping.js
```
NOTE: The time command is an optional Unix command that measures the time it takes to run the k6 command.
PROTIP: To avoid text wrapping, expand the width of the Terminal window before running the script.

Analyze the response:

       /\      |‾‾| /‾‾/   /‾‾/
  /\  /  \     |  |/  /   /  /
 /  \/    \    |     (   /   ‾‾\
/          \   |  |\  \ |  (‾)  |
  / __________ \  |__| \__\ \_____/ .io
 
  execution: local
  script: k6-test-ping.js
  output: -
 
  scenarios: (100.00%) 1 scenario, 10 max VUs, 1m0s max duration (incl. graceful stop):
        * default: 10 looping VUs for 30s (gracefulStop: 30s)
 
 
  data_received..................: 2.8 MB 90 kB/s
  data_sent......................: 50 kB  1.6 kB/s
  http_req_blocked...............: avg=14.12ms  min=2µs      med=6µs      max=351.4ms  p(90)=9µs      p(95)=43.44µs
  http_req_connecting............: avg=6.1ms    min=0s       med=0s       max=146.84ms p(90)=0s       p(95)=0s
  http_req_duration..............: avg=147.64ms min=122.69ms med=141.52ms max=355.05ms p(90)=157.57ms p(95)=177.47ms
    { expected_response:true }...: avg=147.64ms min=122.69ms med=141.52ms max=355.05ms p(90)=157.57ms p(95)=177.47ms
  http_req_failed................: 0.00%  ✓ 0         ✗ 460
  http_req_receiving.............: avg=5.15ms   min=40µs     med=99.5µs   max=177.8ms  p(90)=172.3µs  p(95)=4.88ms
  http_req_sending...............: avg=55.06µs  min=8µs      med=25µs     max=7.7ms    p(90)=41µs     p(95)=56µs
  http_req_tls_handshaking.......: avg=4.39ms   min=0s       med=0s       max=212.04ms p(90)=0s       p(95)=0s
  http_req_waiting...............: avg=142.42ms min=122.55ms med=140.6ms  max=189.64ms p(90)=154.45ms p(95)=158.91ms
  http_reqs......................: 460    14.950299/s
  iteration_duration.............: avg=1.32s    min=1.26s    med=1.28s    max=1.97s    p(90)=1.4s     p(95)=1.42s
  iterations.....................: 230    7.47515/s
  vus............................: 10     min=10      max=10
  vus_max........................: 10     min=10      max=10
 
 
running (0m30.8s), 00/10 VUs, 230 complete and 0 interrupted iterations
default ✓ [======================================] 10 VUs  30s

“vu” is “virtual user”.

Alternately, override the init code options with parameters in the run command, such as Ramp VUs from 0 to 100 over 10s, stay there for 60s, then 10s down to 0:
```
time k6 run -u 0 -s 10s:100 -s 60s:100 -s 10s:0 k6-test-ping.js
```

Get to know the various parameters on the k6 run menu:

k6 run --help

The response:

Start a test.
 
This also exposes a REST API to interact with it. Various k6 subcommands offer
a commandline interface for interacting with it.
 
Usage:
  k6 run [flags]
 
Examples:
  # Run a single VU, once.
  k6 run script.js
 
  # Run a single VU, 10 times.
  k6 run -i 10 script.js
 
  # Run 5 VUs, splitting 10 iterations between them.
  k6 run -u 5 -i 10 script.js
 
  # Run 5 VUs for 10s.
  k6 run -u 5 -d 10s script.js
 
  # Ramp VUs from 0 to 100 over 10s, stay there for 60s, then 10s down to 0.
  k6 run -u 0 -s 10s:100 -s 60s:100 -s 10s:0
 
  # Send metrics to an influxdb server
  k6 run -o influxdb=http://1.2.3.4:8086/k6
 
Flags:
  -u, --vus int                             number of virtual users (default 1)
  -d, --duration duration                   test duration limit
  -i, --iterations int                      script total iteration limit (among all VUs)
  -s, --stage stage                         add a stage, as `[duration]:[target]`
   --execution-segment string            limit execution to the specified segment, e.g. 10%, 1/3, 0.2:2/3
   --execution-segment-sequence string   the execution segment sequence
  -p, --paused                              start the test in a paused state
   --no-setup                            don't run setup()
   --no-teardown                         don't run teardown()
   --max-redirects int                   follow at most n redirects (default 10)
   --batch int                           max parallel batch reqs (default 20)
   --batch-per-host int                  max parallel batch reqs per host (default 6)
   --rps int                             limit requests per second
   --user-agent string                   user agent for http requests (default "k6/0.48.0 (https://k6.io/)")
   --http-debug string[="headers"]       log all HTTP requests and responses. Excludes body by default. To include body use '--http-debug=full'
   --insecure-skip-tls-verify            skip verification of TLS certificates
   --no-connection-reuse                 disable keep-alive connections
   --no-vu-connection-reuse              don't reuse connections between iterations
   --min-iteration-duration duration     minimum amount of time k6 will take executing a single iteration
  -w, --throw                               throw warnings (like failed http requests) as errors
   --blacklist-ip ip range               blacklist an ip range from being called
   --block-hostnames pattern             block a case-insensitive hostname pattern, with optional leading wildcard, from being called
   --summary-trend-stats stats           define stats for trend metrics (response times), one or more as 'avg,p(95),...' (default 'avg,min,med,max,p(90),p(95)')
   --summary-time-unit string            define the time unit used to display the trend stats. Possible units are: 's', 'ms' and 'us'
   --system-tags strings                 only include these system tags in metrics (default "proto,subproto,status,method,url,name,group,check,error,error_code,tls_version,scenario,service,expected_response")
   --tag tag                             add a tag to be applied to all samples, as `[name]=[value]`
   --console-output string               redirects the console logging to the provided output file
   --discard-response-bodies             Read but don't process or save HTTP response bodies
   --local-ips string                    Client IP Ranges and/or CIDRs from which each VU will be making requests, e.g. '192.168.220.1,192.168.0.10-192.168.0.25', 'fd:1::0/120', etc.
   --dns string                          DNS resolver configuration. Possible ttl values are: 'inf' for a persistent cache, '0' to disable the cache,
                                         or a positive duration, e.g. '1s', '1m', etc. Milliseconds are assumed if no unit is provided.
                                         Possible select values to return a single IP are: 'first', 'random' or 'roundRobin'.
                                         Possible policy values are: 'preferIPv4', 'preferIPv6', 'onlyIPv4', 'onlyIPv6' or 'any'.
                                          (default "ttl=5m,select=random,policy=preferIPv4")
   --include-system-env-vars             pass the real system environment variables to the runtime (default true)
   --compatibility-mode string           JavaScript compiler compatibility mode, "extended" or "base"
                                         base: pure goja - Golang JS VM supporting ES5.1+
                                         extended: base + Babel with parts of ES2015 preset
                                                       slower to compile in case the script uses syntax unsupported by base
                                          (default "extended")
  -t, --type string                         override test type, "js" or "archive"
  -e, --env VAR=value                       add/override environment variable with VAR=value
   --no-thresholds                       don't run thresholds
   --no-summary                          don't show the summary at the end of the test
   --summary-export string               output the end-of-test summary report to JSON file
   --traces-output string                set the output for k6 traces, possible values are none,otel[=host:port] (default "none")
  -o, --out uri                             uri for an external metrics database
  -l, --linger                              keep the API server alive past test end
   --no-usage-report                     don't send anonymous stats to the developers
  -h, --help                                help for run
 
Global Flags:
  -a, --address string      address for the REST API server (default "localhost:6565")
  -c, --config string       JSON config file (default "/Users/wilsonmar/Library/Application Support/loadimpact/k6/config.json")
   --log-format string   log output format
   --log-output string   change the output for k6 logs, possible values are stderr,stdout,none,loki[=host:port],file[=./path.fileformat] (default "stderr")
   --no-color            disable colored output
   --profiling-enabled   enable profiling (pprof) endpoints, k6's REST API should be enabled as well
  -q, --quiet               disable progress updates
  -v, --verbose             enable verbose logging

Documentation:

https://grafana.com/docs/k6/latest/get-started/installation/

Know all the k6 commands:

k6

Alternately:

k6 --help

The response:

       /\      |‾‾| /‾‾/   /‾‾/   
  /\  /  \     |  |/  /   /  /    
 /  \/    \    |     (   /   ‾‾\  
/          \   |  |\  \ |  (‾)  | 
  / __________ \  |__| \__\ \_____/ .io
 
Usage:
  k6 [command]
 
Available Commands:
  archive     Create an archive
  cloud       Run a test on the cloud
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  inspect     Inspect a script or archive
  login       Authenticate with a service
  new         Create and initialize a new k6 script
  pause       Pause a running test
  resume      Resume a paused test
  run         Start a test
  scale       Scale a running test
  stats       Show test metrics
  status      Show test status
  version     Show application version
 
Flags:
  -a, --address string      address for the REST API server (default "localhost:6565")
  -c, --config string       JSON config file (default "/Users/wilsonmar/Library/Application Support/loadimpact/k6/config.json")
  -h, --help                help for k6
   --log-format string   log output format
   --log-output string   change the output for k6 logs, possible values are stderr,stdout,none,loki[=host:port],file[=./path.fileformat] (default "stderr")
   --no-color            disable colored output
   --profiling-enabled   enable profiling (pprof) endpoints, k6's REST API should be enabled as well
  -q, --quiet               disable progress updates
  -v, --verbose             enable verbose logging
   --version             version for k6
 
Use "k6 [command] --help" for more information about a command.

curl -L https://github.com/gatewayd-io/gatewayd/releases/download/v0.8.11/gatewayd-linux-amd64-v0.8.11.tar.gz | tar zxvf - brew create https://github.com/gatewayd-io/gatewayd/releases/download/v0.8.11/gatewayd-linux-amd64-v0.8.11.tar.gz –go

extensions

https://k6.io/docs/extensions/get-started/explore/

Using Visual Studio Code

Install the Visual Studio k6 extension to run tests from within Visual Studio Code:

Click on this: https://marketplace.visualstudio.com/items?itemName=k6.k6

Alternately, in Terminal, enter:
```
code --install-extension k6.k6
```
Run from within Visual Studio Code, press command+shift+p to get the Command Palette, then type k6 to get the menu:
```
>k6: Run current file
k6: Run current file in k6 cloud
k6: Open Settings
```

References to the above:

https://medium.com/swlh/beginners-guide-to-load-testing-with-k6-ff155885b6db

Run Script in k6 Cloud

View the script aws-k6-ramp.js within my GitHub repo.

Notice it references file users.json in the same folder.

{
 "users": [
     { "username": "admin", "password": "123" },
     { "username": "test", "password": "1234" },
     { "username": "invaliduser", "password": "password"}
 ]
}

Notice the script at https://test.k6.io/ which loads utility functions from https://jslib.k6.io/k6-utils/1.0.0/index.js

export function uuidv4() {
  return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
 let r = Math.random() * 16 | 0, v = c === 'x' ? r : (r & 0x3 | 0x8);
 return v.toString(16);
  });
}
 
export function randomIntBetween(min, max) { // min and max included
  return Math.floor(Math.random() * (max - min + 1) + min);
}
 
export function randomItem(arrayOfItems){
  return arrayOfItems[Math.floor(Math.random() * arrayOfItems.length)];
}
 
export function randomString(length) {
  const charset = 'abcdefghijklmnopqrstuvwxyz';
  let res = '';
  while (length--) res += charset[Math.random() * charset.length | 0];
  return res;
}

Consider K6 extensions:

VIDEO: Most useful K6 extensions:
- xk6-csv for working one of the most ubiquitous portable data formats, the CSV file. With this extension, reading and parsing CSV data from a specified file is vastly simplified. Functionally comparable to the CSV Data Set Config element of JMeter, a CSV file will be converted into a JavaScript array which allows direct interaction with the test script.
- xk6-file – Using this extension, your k6 test script can directly write to the local file system. The file object exposes various attributes and methods which makes appending test data to a file convenient and easy. When using RedLine13, you can configure these files to be written to the out directory so that they can be downloaded as part of Output Files for your test.
- xk6-dotenv – If your target test application is separated into environments (e.g., ‘dev’, ‘test’, and ‘prod’), then you may consider this extension useful. Often the differences between testing environments and production environments are minor and incremental. This extension will allow you to specify configuration information as environment variables enabling your test script to switch between testing and production deployments without changes to code.
- xk6-faker – A common scenario when performing load testing against a user-facing service pits a large CSV file containing user data against the target test application. This file may contain hundreds or even thousands of fictitious names, email addresses, phone numbers, etc. With k6, there is an easier way to accomplish this, and that is using the xk6-faker plugin. This library allows the generation of hundreds of everyday names and objects including people, dates, financials, words, and more. A full list of functions can be found here.
- xk6-mock – This extension allows you to simulate HTTP/S requests within k6 tests. To compare with JMeter, this would be the equivalent of the Dummy Sampler. You can define requests and simulated responses conveniently with this extension. It is extremely useful when debugging a test script, but want to avoid running actual requests to the target test server. When you are ready to run your test in production, simply change the import statement from k6/http to k6/x/mock.

Notice the script’s ramp-up pattern.

scenarios: (100.00%) 1 scenario, 200 max VUs, 5m30s max duration (incl. graceful stop):
        * default: Up to 200 looping VUs for 5m0s over 3 stages (gracefulRampDown: 30s, gracefulStop: 30s)

Notice the script changes for bytes received.

const checkRes = check(res, {
  'Homepage body size is 11026 bytes': (r) => r.body.length === 11026,
});

Notice the script’s login function.

https://k6.io/docs/using-k6/test-life-cycle

Run the script

time k6 run aws-k6-ramp.js

          /\      |‾‾| /‾‾/   /‾‾/
  /\  /  \     |  |/  /   /  /
 /  \/    \    |     (   /   ‾‾\
/          \   |  |\  \ |  (‾)  |
  / __________ \  |__| \__\ \_____/ .io
 
  execution: local
  script: aws-k6-ramp.js
  output: -
 
  scenarios: (100.00%) 1 scenario, 200 max VUs, 5m30s max duration (incl. graceful stop):
        * default: Up to 200 looping VUs for 5m0s over 3 stages (gracefulRampDown: 30s, gracefulStop: 30s)
 
 
  █ Front page
 
    ✗ Homepage body size is 11026 bytes
     ↳  0% — ✓ 0 / ✗ 2332
    ✓ Homepage welcome header present
 
    █ Static assets
 
      ✓ Is stylesheet 4859 bytes?
 
  █ Login
 
    ✓ Users should not be auth'd. Is unauthorized header present?
    ✗ is logged in welcome header present
     ↳  32% — ✓ 760 / ✗ 1572
 
✗ check_failure_rate.............: 41.85% ✓ 3904      ✗ 5424
  checks.........................: 66.51% ✓ 7756      ✗ 3904
  data_received..................: 78 MB  252 kB/s
  data_sent......................: 5.0 MB 16 kB/s
  group_duration.................: avg=3.94s    min=264.78ms med=699.46ms max=11.42s   p(90)=10.74s   p(95)=10.76s
  http_req_blocked...............: avg=10.55ms  min=1µs      med=4µs      max=902.66ms p(90)=12µs     p(95)=136.15ms
  http_req_connecting............: avg=2.53ms   min=0s       med=0s       max=707.95ms p(90)=0s       p(95)=0s
✓ http_req_duration..............: avg=147.84ms min=120.6ms  med=144.84ms max=805.51ms p(90)=162.27ms p(95)=168.76ms
    { expected_response:true }...: avg=147.84ms min=120.6ms  med=144.84ms max=805.51ms p(90)=162.27ms p(95)=168.76ms
  ✗ { staticAsset:yes }..........: avg=148.81ms min=120.6ms  med=146.02ms max=702.96ms p(90)=163.54ms p(95)=169.7ms
  http_req_failed................: 0.00%  ✓ 0         ✗ 25652
  http_req_receiving.............: avg=390.88µs min=24µs     med=84µs     max=142.65ms p(90)=176µs    p(95)=302µs
  http_req_sending...............: avg=29.21µs  min=7µs      med=24µs     max=6.37ms   p(90)=46µs     p(95)=62µs
  http_req_tls_handshaking.......: avg=1.19ms   min=0s       med=0s       max=422.18ms p(90)=0s       p(95)=0s
  http_req_waiting...............: avg=147.42ms min=120.48ms med=144.45ms max=805.45ms p(90)=161.85ms p(95)=168.19ms
  http_reqs......................: 25652  82.793655/s
  iteration_duration.............: avg=21.45s   min=21.23s   med=21.41s   max=22.59s   p(90)=21.68s   p(95)=21.89s
  iterations.....................: 2332   7.526696/s
  successful_logins..............: 760    2.452954/s
  time_to_first_byte.............: avg=149.11ms min=123.25ms med=146.12ms max=702.8ms  p(90)=163.35ms p(95)=169.29ms
  vus............................: 3      min=3       max=200
  vus_max........................: 200    min=200     max=200
 
 
running (5m09.8s), 000/200 VUs, 2332 complete and 0 interrupted iterations
default ✓ [======================================] 000/200 VUs  5m0s
ERRO[0311] thresholds on metrics 'check_failure_rate, http_req_duration{staticAsset:yes}' have been crossed
k6 run aws-k6-ramp.js  15.54s user 7.89s system 7% cpu 5:11.66 total

The last line is from the time command.

API testing

Based on https://medium.com/swlh/beginners-guide-to-load-testing-with-k6-ff155885b6db

Still in my k6 folder.
Run
```
time k6 run httpbin-apis.js
```

FIXME: The result I got:

WARN[0000] There were unknown fields in the options exported in the script  error="json: unknown field \"max_vus\""
ERRO[0000] invalid threshold defined on RTT; reason: no metric name "RTT" found
k6 run httpbin-apis.js  0.69s user 0.07s system 105% cpu 0.715 total

Is “max_vus” the “vus_max” mentioned in https://medium.com/swlh/beginners-guide-to-load-testing-with-k6-73d55ee23723

Visualize Results

https://k6.io/blog/ways-to-visualize-k6-results/

grafana-influxdb-dashboard.json

Customize k6 JavaScript

Edit file larger-k6.js

https://k6.io/docs/testing-guides/running-large-tests/
Consider: VIDEO: Learn k6 Series - E2 - Recording in k6 using browser extensions by QAInsights

Config Prometheus to Grafana

https://k6.io/blog/k6-loves-prometheus/

Inject faults

https://github.com/grafana/xk6-disruptor

Dashboard

Let’s use Grafana to create a dashboard to graphically visualize the results of k6 tests:

Mostafa Moradian https://medium.com/swlh/beginners-guide-to-load-testing-with-k6-85ec614d2f0d

https://grafana.com/grafana/dashboards/2587-k6-load-testing-results/

References

VIDEO: Running distributed k6 tests on Kubernetes by Simon Aronsson, Olha Yevtushenko

VIDEO: Basics of load testing with k6 and Grafana in 20 minutes k6

VIDEO: Intro to load testing with k6 and Grafana (k6 data source plugin and Prometheus Remote Write) k6

VIDEO: How to Use k6 to Run Load Testing for a Website (for free)

https://k6.io/blog/k6-loves-prometheus/

https://www.youtube.com/watch?v=5OgQuVAR14I&list=RDLV5OgQuVAR14I&start_radio=1&rv=5OgQuVAR14I The Best Performance And Load Testing Tool? k6 By Grafana Labs DevOps Toolkit

https://www.youtube.com/watch?v=Hu1K2ZGJ_K4 Performance Testing your web app with k6 Chris James

https://www.youtube.com/watch?v=ZAq87eZ1w2U What is K6 & How to get started with k6 Is it Observable

https://www.youtube.com/watch?v=AEk-wgkIo4s Performance Testing with K6 Day 1 On 10th August. Call or whatsapp us on +91-8019952427 to enroll Performance Testing basics and advanced

https://tsh.io/blog/how-to-do-performance-testing-using-k6/ by https://www.linkedin.com/in/marcin-basiakowski/ Marcin Basiakowski

https://devqa.io/k6-load-testing/

https://medium.com/swlh/tagged/k6 The Startup on Medium: k6

VIDEO: In Russian Intro to K6 Load Testing Tool

Product comparisons

https://www.youtube.com/watch?v=KECr2BujqtM 15 Top Load Testing Tools Open Source MUST KNOW in 2021 Automation Testing with Joe Colantonio

https://www.youtube.com/watch?v=noZppBruOSY JMeter vs k6: Comparing two popular open-source load testing tools k6

https://www.youtube.com/watch?v=Be66Db4wHLA Postman for load testing using k6, with Tim Haselaars (k6 Office Hours #43) k6

https://k6.io/docs/testing-guides/api-load-testing/

K6 Champions

https://k6.io/blog/topics/community

Champion Grzegorz Piechnik, Performance Engineer in Poland
Champion Marlo Henrique, QA in Brazil
Champion Donald Le in Vietnam
Champion Ziv Kalderon in Israel
Champion Sahani Perera in Sri Lanka
Champion Sarah Zipkin in Montana
Champion Sarah Zipkin in Montana
Champion Sarah Zipkin in Montana
Champion Sarah Zipkin in Montana
Champion Sarah Zipkin in Montana

More on Security

This is one of a series on Security and DevSecOps:

Wilson Mar

K6