Wilson Mar bio photo

Wilson Mar

Hello. Hire me!

Email me Calendar Skype call 310 320-7878

LinkedIn Twitter Gitter Instagram Youtube

Github Stackoverflow Pinterest

It’s less like Power Rangers clicking rings, more like living among expats in a foreign country

US (English)   Español (Spanish)   Français (French)   Deutsch (German)   Italiano   Português   Cyrillic Russian   中文 (简体) Chinese (Simplified)   日本語 Japanese   한국어 Korean

Overview

I began looking into Microsoft’s WSL (Windows Subsystem for Linux) at its first “beta” release on June 2017.

Around May 2019, Microsoft released WSL2.

WSL is so much simpler than Cygwin

VIDEO and BLOG: Scott Hanselman explains it all: VS Code with Remote Extension, Docker, and the Windows Terminal.

Architecture

Here is the diagram for WSL1:

bash-windows-wsl file-system-graphic-1024x547

Lxcore.sys is the driver that recognizes Bash commands and other Linux utilities such as chmod to change permissions.

VFS is the Virtual File System.

From inside Bash, DrvFS gets to Windows.

From inside Windows, VolFS gets to Linux symbolic files and its case sensitivity.

NTFS (New Technology File System) is the file system controlling hard drives.

WSL provides no GPU support, so it can’t run Linux GUI programs such as Gnome, KDE, etc.

Got 64-bit?

Verify your PC’s CPU architecture and Windows version/build number:

  1. Start: Click the Windows icon or keyboard key to open Settings, System, About.

    bash-windows-build-951x388

    If your System type is not “64-bit”, get a computer that is. The technical explanation is that WSL currently runs ELF64 Linux binaries.

    Windows 10 Build

    Alternately, if you have configured PowerShell to run commands, get your Windows Server build number within PowerShell:

    systeminfo | Select-String "^OS Name","^OS Version"

    If your OS Build is not 18917 or higher, update.

    Windows Insider Build

    As of this writing, WSL is in Preview, so Register to be in the Windows Insider. (Beta was in the Windows 10 “Anniversary Update Creators Update”, then “Fall Creators Update”)

  2. Go to Settings > Update & Security > Windows Insider Program and click Get Started to access the latest build.
  3. Enter the account you used to sign into the Windows Insider Program.
  4. Follow the prompts to install.

    As of this writing, Windows Insider version 19041.1 was available on the slow ring. You don’t need a fast ring build to get WSL 2.

  5. Go to Settings > Update & Security > Windows Update and click Check for updates to install.

    Docker Desktop Edge 2.1.6.0 + requires Windows 10 Pro or Enterprise Insider Preview build 19018 or higher to integrate with WSL Linux.

    https://www.microsoft.com/en-us/software-download/windowsinsiderpreviewSDK

    Virtualization

    Virtualization must be enabled in the computer bios, instructions for this vary between manufacturers but it’s usually a simple on/off listing in the BIOS.

    Enable WSL Windows Feature

  6. Press the Windows Start button and immediately type in the search box that appears:

    Turn Windows features on or off
    
  7. Type enough until the option appears in the menu, then click on the option.

    Wait a while for all items to appear.

  8. Scroll down to check “Windows Subsystem for Linux”, then click OK to exit dialog, then Restart your computer.

    (This option was added since the “Anniversary” and “Creators Update” of Windows 10.)

    Windows Terminal

  9. Click the Windows Start and get in the Microsoft Window Store to install the new Windows Terminal. It is like tmux - it makes it easy to open multiple panes with different prompts for bash, command prompt and PowerShell.

    Download installer

  10. Following instructions at https://docs.microsoft.com/en-us/windows/wsl/wsl2-install, in PowerShell, enable the ‘Virtual Machine Platform’ optional component (whatever that means):

    dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
    dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
    
  11. Confirm whether Windows Subsystem for Linux is enabled in PowerShell:

    Get-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux
    

    Set Linux distro to be backed by WSL 2

  12. PROTIP: WSL provides a choice of Linux distributions.

    wsl -l
    • Ubuntu-18.04 - CanonicalGroupLimited.Ubuntu18.04onWindows_1804.2018.817.0_x64__79rhkp1fndgsc
    • Ubuntu 18.04 ARM for running on Raspberry Pi?
    • Ubuntu-16.04 - Ubuntu_1604.2019.523.0_x64.appx
    • Debian GNU/Linux - DebianGNULinux_1-1-3-0_x64__76v4gfsz19hv4
    • Kali Linux
    • OpenSUSE Leap 42
    • SUSE Linux Enterprise Server 12
    • Fedora Remix for WSL (licensed)
    • Pengwin is paid/licensed. Based on Debian.
    • WLinux ???

    PROTIP: Ubuntu was the first distro tested with WSL, so it’s probably the most well tested. But it’s not supported by Windows 10 S. Both Ubuntu and Debian make use of the apt-get (Advanced Packaging Tool) package manager and dpkg command.

    BTW Ubuntu is a Linux distribution from Canonical, Inc. which also created Virtualenv for Python.

    Kali Linux is used by security researchers.

  13. From the Start menu, select the Microsoft Windows Store. Search for “Linux”. Click Ubuntu, then ‘Install’:

    Once installed, the ‘Install’ button will change to ‘Launch’, click the ‘Launch’ button.

    Alternately, download the “.appx” installer by constructing the curl command for the Ubuntu version listed above, such as:

    curl -L -o ubuntu-1604.appx https://aka.ms/wsl-ubuntu-1604

    This is instead of the Invoke-WebRequest PowerShell command or Microsoft Store for Business used within enterprises.

  14. In PowerShell, construct the command using the name of the .appx file downloaded:

    Add-AppxPackage .\Ubuntu_1604.2019.523.0_x64.appx
  15. Set the distribution code for the Linux distro downloaded (I don’t know why):

    wsl --set-version Ubuntu 2
  16. Verify what versions of WSL your distros are using:

    wsl -l -v
  17. Make WSL 2 your default architecture (as if you’ll ever want to go back):

    wsl --set-default-version 2
    
    ## Launch and configure
    
    
  18. Launch the new instance of WSL by from the Start menu section titled “WSL Linux Distros”.

  19. The first time a newly installed distro runs, a Console window opens, and you’ll see

    Installing, this may take a few minutes...

    BTW: In previous versions, you had to go to Open Settings -> Update and Security -> For developers to check the Developer Mode radio button to “install any signed app”.

    Then…

    Installation successful
    Please create a default UNIX user account. The username does not need to match your Windows username.
    For more information visit: https://aka.ms/wslusers
    Enter new UNIX username: _
  20. Enter a username your make up.

    PROTIP: The WSL user is not “root” with admin priviledges.

  21. PROTIP: Type the password in 1Password or other Password Manager, then copy it to paste in the screen.

    A shortcut is added to your start menu named:

    Bash on Ubuntu on Windows

  22. Click the shortcut so you don’t have to type “bash”.

    By default, the prompt is your Linux user name @ your machine name:/mnt/c/Users/%USERNAME%$

    Where is WSL?

    PROTIP: WSL runs within the Vemmem process.

  23. Open Task Manager to see it.

    FAQ

  24. Read the FAQ for known weirdness.

    Bugs with WSL are reported to developers at https://github.com/microsoft/WSL

    https://docs.microsoft.com/en-us/windows/wsl/troubleshooting

    Profile to define prompt

  25. Open the Bash command prompt</strong> (click Windows Start and type Bash until you can select it from the list that arises).

    NOTE: You can run Linux binaries such as ls from the Windows Command Prompt (CMD or PowerShell) by invoking wsl.exe there. These are called interop features.

  26. Open the Visual Studio Code text editor to the file in the $HOME folder, which is what ~ stands for, just like in macOS:

    code ~/.profile
    

    By default, WSL reads and executes commands from the file /etc/profile if that file exists. After reading that file, it looks for ~/.bash_profile, ~/.bash_login, and ~/.profile, in that order, and reads and executes commands from the first one that exists and is readable. It skips the other files if one is found.

  27. Define keyboard aliases in a .bash_aliases file in the .bashrc file *

    code ~/.bashrc
    

    My list is in https://github.com/wilsonmar/git-utilities/master/aliases.sh

    Zsh

    See https://medium.com/@edwardbaeg9/using-homebrew-on-windows-10-with-windows-subsystem-for-linux-wsl-c7f1792f88b3

    Linux commands

  28. Get to know the built-in Linux commands: cat, cd, chmod, chown, curl, df, diff, echo, exit, find, finger, grep, groups, gzip, head, history, kill, less, ls, man, mkdir, mv, passwd, ping, ps, pwd, shutdown, ssh, sudo, tail, tar, top, uname, w, whoami. There’s also cp.

    Rather than a translation layer built by the WSL 1 team, WSL 2 includes its own open-sourced Linux kernel with full system call compatibility with the LTS Linux kernel. WSL2 does not work under HyperV.

    Sean Dearnaley points out that since macOS currently doesn’t have good GPU support, if Windows introduces GPU support for WSL, Windows machines could become a very powerful machine learning development platform when running Nvidia CUDA based apps.

    pwd file storage mounts

  29. To list drives mounted:

    cd /mnt
    

    The response is the drives:

    c d f
  30. For the purpose of this tutorial, create folder dev/project for use by both Windows and Linux tools:

    cd /mnt/c/dev/
    mkdir project
    

    You can also make whatever directory name you want.

  31. PROTIP: Use $HOME or ~ to reference your home folder:

    cd ~
    pwd
    

    Unlike/users on macOS, the folder above user accounts is:

    /home/me
  32. Open Windows Explorer to view files from both Windows and from Bash:

    explorer.exe .
    
  33. Access files from both Windows and from Bash as:

    /mnt/c/path
    
  34. Run a Linux utility such as disk usage of the current folder (represented by a dot):

    du -sh
    

    The response is like 56K .

  35. View the manual on the du command:

    man du
    
  36. PROTIP: Remember this move whenever you see that “:” in the lower-left corner:

    type q to quit out.

    Folders and variables

  37. See where Ubuntu is installed using a Windows system variable referenced using a Windows % wrapper rather than Bash $ prefix:

    %localappdata%\lxss\
    

    Note the directory is marked as a hidden system folder. That’s a clue that you should not modify files in your bash environment using Windows File Explorer, console, or apps.

    Creating and/or modifying files in this location using Windows tools and apps corrupts the system because it is read-locked. Here is an explanation.

    Homebrew

  38. Install the LinuxBrew fork of Homebrew for macOS*

    “I found some apps that didn’t work well from apt-get worked flawlessly when installed with brew, like zplug. Inversely, I couldn’t get ranger to work with brew but got it working with apt-get. This very conveniently gives you multiple options for installing a package, potentially skipping the step of Googling vague errors.

    To fix the patchelf error:

    sudo apt-get update
    sudo apt-get install build-essential
    

    Docker

  39. TODO: Use brew to install Docker Desktop Edge, in Technical Preview as of this writing.

    bash-windows-docker

    See https://docs.docker.com/docker-for-windows/edge-release-notes/

    Sean Dearnaley provides pointers. It now supports Kubernetes, offers VPN-friendly networking, provides an updated Docker daemon, and many new features.

    Visual Studio Code

    VIDEO: Run and debug your Linux-based applications from within VSCode in Windows. Edit files in WSL or the mounted Windows filesystem (/mnt/c) without worrying about pathing issues, binary compatibility, or other cross-OS challenges.

  40. Install Visual Studio Code Remote — WSL extension at https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-wsl

    Git

  41. TODO: Use brew to install additional utilities: git, Python/pip, tree, jq, etc.

    https://medium.com/swlh/wsl-2-docker-edge-tech-preview-native-linux-containers-w-o-emulation-b41667e6dbef

  42. Use pip to install Virtualenv.

  43. Install keyboard aliases to use custom commands:

    TODO: git-utilities

    NOTE: To run Python for Selenium controlling Firefox, install Xming with gekoDriver.

    VHD size adjustment

    WSL 2 stores Linux files inside of a VHD (Virtual Hard Disk) using the ext4 file system. VHD has an initial max size of 256GB. If your distro grows beyond that you will see errors stating that you’ve run out of disk space. To expand VHD size:

  44. Terminate all WSL instances:

    wsl --shutdown
  45. Find your distro installation package name ‘PackageFamilyName’.

  46. In a powershell prompt (where ‘distro’ is your distribution name) construct:

    Get-AppxPackage -Name "*distro*" | Select PackageFamilyName
  47. Locate the VHD file fullpath used by your WSL 2 installation, this will be your ‘pathToVHD’:

    %LOCALAPPDATA%\Packages\PackageFamilyName\LocalState\disk.vhdx

    Resize your WSL 2 VHD:

  48. Open a command prompt Window with admin privileges and run the following commands:

    diskpart

    In the dialog:

    Select vdisk file=”pathToVHD

    expand vdisk maximum=”sizeInMegaBytes

  49. Launch your WSL distro.

  50. Make WSL is aware that it can expand its file system’s size by running these commands in your WSL distro:

    sudo mount -t devtmpfs none /dev
    mount | grep ext4
  51. Copy the name of this entry, which will look like: /dev/sdXX (with the X representing any other character), making sure to use the value you copied earlier.

    sudo resize2fs /dev/sdXX
  52. You may need to use:

    apt install resize2fs

References

Links and issues for this:

https://github.com/Microsoft/BashOnWindows

https://msdn.microsoft.com/en-us/commandline/wsl/install_guide

https://blogs.msdn.microsoft.com/wsl/2016/06/15/wsl-file-system-support published 15 June 2016 by Jack Hammons, who writes:
https://blogs.msdn.microsoft.com/wsl, the home page of WSL.

https://medium.com/better-programming/make-bash-on-ubuntu-on-windows-10-look-like-the-ubuntu-terminal-f7566008c5c2

https://medium.com/free-code-camp/how-to-set-up-docker-and-windows-subsystem-for-linux-a-love-story-35c856968991 “Yeah, it seemed to have uninstalled linuxbrew as soon as I closed the Ubuntu shell. Although my path references in .profile seemed to persist”

More on DevOps

This is one of a series on DevOps:

  1. DevOps_2.0
  2. ci-cd (Continuous Integration and Continuous Delivery)
  3. User Stories for DevOps

  4. Git and GitHub vs File Archival
  5. Git Commands and Statuses
  6. Git Commit, Tag, Push
  7. Git Utilities
  8. Data Security GitHub
  9. GitHub API
  10. TFS vs. GitHub

  11. Choices for DevOps Technologies
  12. Java DevOps Workflow
  13. AWS DevOps (CodeCommit, CodePipeline, CodeDeploy)
  14. AWS server deployment options

  15. Cloud services comparisons (across vendors)
  16. Cloud regions (across vendors)
  17. AWS Virtual Private Cloud

  18. Azure Cloud Onramp
  19. Azure Cloud
  20. Azure Cloud Powershell
  21. Bash Windows using Microsoft’s WSL (Windows Subystem for Linux)

  22. Digital Ocean
  23. Cloud Foundry

  24. Packer automation to build Vagrant images
  25. Terraform multi-cloud provisioning automation
  26. Hashicorp Vault and Consul to generate and hold secrets

  27. Powershell Ecosystem
  28. Powershell on MacOS
  29. Powershell Desired System Configuration

  30. Jenkins Server Setup
  31. Jenkins Plug-ins
  32. Jenkins Freestyle jobs
  33. Jenkins2 Pipeline jobs using Groovy code in Jenkinsfile

  34. Docker (Glossary, Ecosystem, Certification)
  35. Make Makefile for Docker
  36. Docker Setup and run Bash shell script
  37. Bash coding
  38. Docker Setup
  39. Dockerize apps
  40. Docker Registry

  41. Maven on MacOSX

  42. Ansible

  43. MySQL Setup

  44. SonarQube static code scan

  45. API Management Microsoft
  46. API Management Amazon

  47. Scenarios for load