Wilson Mar bio photo

Wilson Mar

Hello!

Calendar YouTube Github

LinkedIn

Make PowerShell your superpower on Windows and Macs, for fun and profit

US (English)   Norsk (Norwegian)   Español (Spanish)   Français (French)   Deutsch (German)   Italiano   Português   Estonian   اَلْعَرَبِيَّةُ (Egypt Arabic)   Napali   中文 (简体) Chinese (Simplified)   日本語 Japanese   한국어 Korean

Overview

Powershell icon

This article describes the use of PowerShell scripting on Mac and Linux.

“PowerShell” refers to both the command-line shell AND scripting language designed system administration. (When “PowerShell Core 6.0” was announced on January 10, 2018, the word “Powershell” on its own now refers to the decade-old “PowerShell” integrated into all recent versions of Microsoft’s Windows operating system.)

PowerShell is an object-centered “management engine” that can be hosted in an application program:

  • CMD in Windows
  • PowerShell ISE (from Microsoft)
  • PowerGUI
  • SAPIEN Technologies PowerShell Studio
  • Idera PowerShell Pro

PowerShell Core is available as a cross-platform application such that scripts written on MacOS will run on Windows, Linux, or other supported operating system. So it does not have commands associated with the .NET Framework (for Windows OS). Such is similar to the rebranding of .NET vs. .NET Core.

PowerShell cmdlets (pronounced “command-lets”) let you manage computers from the command line.

ISE (Integrated Scripting Environment) is a GUI program that provides popup tab completion and other assists.

PowerShell promises more consistency than the various commands added over time by various parties:

  • It reads Excel files natively as well as JSON, XML, and even ASCII.
  • Microsoft Deployment Toolkit
  • Microsoft System Center
  • IBM, etc.

The above and other scripting is covered in my blog wilsonmar.github.io/azure-cloud-powershell

Open source on Linux and MacOS

PowerShell is open-sourced for all OSs at https://github.com/PowerShell/PowerShell.

Install PowerShell on Windows

https://aka.ms/PSWindows for PowerShell 7 on Windows 11 installed to: $env:ProgramFiles\PowerShell\7

choco install powershell-core

Windows PowerShellGet Module if you don’t want to install these from the Web Platform Installer (wpilauncher.exe) at https://www.microsoft.com/web/downloads/platform.aspx

I have a separate blog wilsonmar.github.io/powershell-dsc

Install PowerShell Core on MacOS

PowerShell Core supports macOS 10.12 and higher. See this for other os

  1. This says needed as a pre-requisite is

    xcode-select --install
  2. PROTIP: Skip the manual install and install using Homebrew.

    Alternately, click to download the latest release for MacOS at:
    https://github.com/PowerShell/PowerShell

    Alternately, get back versions at
    https://github.com/PowerShell/PowerShell/releases

    Date File MB Size Space Cmds
    21 May 2021 7.2.0-preview.6 62.1 MB
    Apr, 2018 6.0.2 on brew 50.8 MB
    Sep 13, 2017 powershell-6.0.0-beta.7-osx.10.12-x64.pkg 50.8 MB
    Sep 13, 2016 powershell-6.0.0-alpha.10.pkg 28.2 MB ? MB 345
    Aug 10, 2016 powershell-6.0.0-alpha.9.pkg 37.1 MB 119.7 MB
    Jul 26, 2016 powershell-6.0.0-alpha.7.pkg 25.0 MB
    Jul 8, 2016 powershell-0.6.0.pkg 24.2 MB
  3. Open the .pkg file in the Downloads folder:
  4. Click Continue, etc.

    NOTE: For Windows: Microsoft Windows Management Framework 5.0

    Install using Homebrew

  5. Alternately, use Homebrew:

    brew install powershell

    ==> Caveats
    To use Homebrew in PowerShell, set:
      Add-Content -Path $PROFILE.CurrentUserAllHosts -Value '$(/usr/local/bin/brew shellenv) | Invoke-Expression'
     
    ==> Downloading https://github.com/PowerShell/PowerShell/releases/download/v7.3.
    ==> Downloading from https://objects.githubusercontent.com/github-production-rel
    ######################################################################### 100.0%
    All formula dependencies satisfied.
    ==> Installing Cask powershell
    ==> Running installer for powershell; your password may be necessary.
    Package installers may write to any location; options such as `--appdir` are ignored.
    Password: ____
    
  6. ATTENTION: Enter your laptop password when prompted.

    Response:

    installer: Package name is PowerShell - 7.4.1
    installer: Installing at base path /
    installer: The install was successful.
    🍺  powershell was successfully installed!
    

    Run PowerShell in Bash

  7. An example of command parameters within double-quotes:

    po
    

    “Hello World” would be the response.

  8. Double-quotes are not needed for a single command, such as this to list folders (child items):

    pwsh -command get-childitem
    
     Directory: /Users/...
     
    Mode                 LastWriteTime         Length Name
    ----                 -------------         ------ ----
    -----           5/12/2021  3:48 PM          11381 .dir
    

    In and Out

  9. Open a Terminal shell window to launch PowerShell
    on MacOS:

    pwsh
    

    On Windows:

    powershell
    

    The response:

    PowerShell 7.4.1
    PS /Users/...> 
    

    PS displayed means you are in the PowerShell shell.

    Gone is line: https://aka.ms/pscore6-docs

    Sometimes the response also has:

    A new PowerShell stable release is available: v7.4.1
    Upgrade now, or check out the release page at:       
      https://aka.ms/PowerShell-Release?tag=v7.3.7
    

    System Variables

  10. List all system variables:

    dir env:

    Alternately, to sort by name:

    gci env:* | sort-object name

    “gci” is short for:

    Get-ChildItem Env:* | Select-Object -Property Name,Value

    Notice there are “LOGNAME” and “USER” variables.

  11. The content of system environment variable name $USER can be identified quickest using a command on both Linux and Windows:

    uname
  12. To display just the value of the $HOME variable which defines the path where the “cd” command navigates to:

    Get-Variable HOME -valueOnly
    

    On MacOS and Linux, for example (replacing “johndoe” with your user name):

    /Users/johndoe

    On Windows:

    C:\Users\johndoe

    Customize command prompt

  13. The default prompt is defined by this:

    function prompt { 'PS ' + $(get-location) + '> ' }

    The default command prompt contains “>” after the current path.

    I don’t like this because the prompt appears in different positions, requiring me to spend time finding it. And there isn’t much space left for commands before forced wrapping.

    I would rather have the prompt be at the same position, such as this, which gives a lot of space for long commands:

    > _

    I don’t need “PS”. Above the prompt, I show time of day, [Git branch], and current folder:

    03:54:32 PM [master] _posts

    _ </tt>

  14. To achieve the above prompt,

    function prompt {"`n  "+$(Get-Date -UFormat "%r")+' ['+$(git rev-parse --abbrev-ref HEAD)+'] '+$((get-item $pwd ).Name)+"`n"+'> ' }

    PROTIP: In PowerShell, a new line is specified by the “back-tick” escape character (at the upper-left of most keyboards). Also notice double quotation marks are necessary. The back-tick is also used for line continuation, so don’t put a space after a back-tick or PowerShell will recognize it as an escape character rather than a line continuation.

    An alternative to obtaining the current folder is:

    $($executionContext.SessionState.Path.CurrentLocation | Split-Path -Leaf)

    NOTE: Others display different colors.

  15. To make the change permanant, change your user profile definition file at the path defined by the $PROFILE system environment variable:

    $PROFILE

    On MacOS and Linux:

    /Users/USER/.config/powershell/Microsoft.PowerShell_profile.ps1

    On Windows:

    C:\Users\USER\Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1

    NOTE: That file is one of six PowerShell profile files*

  16. Check if it’s true that you have a profile file:

    Test-Path $PROFILE

    If “False”, create the file “Microsoft.PowerShell_profile.ps1”

    New-Item -Path $PROFILE -Type File -force
  17. Edit the file “Microsoft.PowerShell_profile.ps1”:

    code $PROFILE
  18. Copy and paste the function prompt from above.
  19. Click the “…” at the right to Save and Close Editor (or press Command+S and Command+Q).

    Exit PowerShell

  20. To leave PowerShell for changes to take effect, it’s the same as in Bash scripts:

    exit

  21. Enter Powershell again, per above.

    Upgrade PowerShell

  22. To upgrade (within Bash or pwsh):

    brew upgrade --cask powershell

    Verify install by seeing version

  23. Check the version of PowerShell being used by calling a pre-defined variable:

    $psversiontable

    PROTIP: With PowerShell, a variable can act like a command.

    Response:

    Name                           Value
    ----                           -----
    PSVersion                      7.1.3
    PSEdition                      Core
    GitCommitId                    7.1.3
    OS                             Darwin 19.6.0 Darwin Kernel Version 19.6.0: Tue Jan 12 22:13:05 PST 2021; root:xnu-6153.141.16~1/RELEASE_X86_64
    Platform                       Unix
    PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
    PSRemotingProtocolVersion      2.3
    SerializationVersion           1.1.0.1
    WSManStackVersion              3.0
    

    QUESTION: What’s the CLRVersion?

    • 7.0
    • 6.0 for Mac/Linux in Windows 10 Anniversay Edition
    • 5.0 in 2015 for Visual Studio Code text editor
    • 4.0 in 2014 with Windows 10 and .NET Framework 4.0 and Windows Management Framework 3.0
    • 3.0 in 2012 with Windows 8/Server 2012
    • 2.0 appeared in 2009
    • 1.0 appeared in 2006
    • Monad Manifesto published by Jeff Stover.

    PROTIP: Know the PowerShell commands known not to work on Linux/macOS.

    Configure Terminal colors

  24. Configure your Terminal colors

    See https://sqlsunday.com/2019/03/04/how-to-set-up-a-beautiful-powershell-core-terminal-on-mac-os/

  25. Get back in PowerShell

    Clear PS Screen

    This doesn’t need any modules installed:

  26. Clear Screen:

    cls
    

    Get Help on commands

  27. Download help files:

    update-help -force
    
  28. Get help information for a command (such as stop-service):

    get-help stop-service

    Name                              Category  Module                    Synopsis
    ----                              --------  ------                    --------
    Register-ArgumentCompleter        Cmdlet    Microsoft.PowerShell.Core Registers a custom argument completer.
    about_If                          HelpFile                
    about_Pipelines                   HelpFile
    

    Output continuation & wide view

    PROTIP: To continue a line end (like back-slash in Bash), use “tick marks” (` at the upper-left on Mac keyboards) and use “vertical bar” ( at the right of Mac keyboards) :
    Get-Alias -Definition Invoke-* `
    | Format-Table -Property * -AutoSize `
    | Out-String -Width 4096 `
    | Out-File aliases.txt
    

Visual Studio Code Editor

One text editor built for PowerShell is Microsoft’s Visual Studio Code.

  1. Install Visual Studio Code (see https://chocolatey.org/packages/VisualStudioCode):

    choco install visualstudiocode -y

  2. Install the PowerShell add-in to VSCode:

    choco install vscode-powershell -y

  3. Install the PowerShell Editor Services extension by pressing Ctrl+P, then type “ext install PowerShell” for a list of add-ins.

    Ctrl+P is the universal search that also does “fuzzy search” of text in files open.

  4. Click “install” of the extension named “PowerShell”. The icon turns to “installing”.

  5. Open a directory containing PowerShell scripts and open the File menu and select “Open Folder …”. Select the folder containing your scripts.
    The scripts show up in the Explore tab of the Side Bar.

    PROTIP: One advantage using VS Code is its Side Bar enabling you to switch quickly among different files.

    Press Ctrl+B to hide and unhide the Side Bar.

  6. Press Ctrl+\ to open a new editor window.

    Up to three editor panes can be open at once.

    Press Ctrl+1, 2, or 3 to switch among the files.

  7. To edit user settings, press Ctrl+Shift+P, then type “user” and press enter.

  8. Click on “powershell.scriptAnalysis.enable”.

  9. Press Ctrl+Shift+&LT;period> to change value from true to false or back again.

Keith Hill notes debugging support provided by the PowerShell Editor Services extension currently runs only on Windows.

Install .NET Core

PowerShell is written on top of .NET. .NET’s previous dependencies on Windows components have been removed in .NET Core.

PowerShell errors occur if .NET Core is not installed, so:

  1. Go to web page https://www.microsoft.com/net/core#macos

  2. The web page asks for OpenSSL to be installed.

    On a Mac:

    brew update
    brew install openssl
    ln -s /usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib /usr/local/lib/
    ln -s /usr/local/opt/openssl/lib/libssl.1.0.0.dylib /usr/local/lib/
    
  3. Click the link to download the 50.3MB
    dotnet-dev-osx-x64.1.0.0-preview2-003131.pkg

    https://github.com/dotnet/core/blob/master/cli/known-issues.md

  4. Run the installer (for 106.3MB of space).

  5. Before installing anything or running through the update app, hit Command+i or pull down the File menu and choose “Show Files”:

    • ./shared - Microsoft .NET Core 1.0.1 - Runtime
    • ./host - Microsoft .NET Core 1.0.1 - Host FX Resolver
    • ./dotnet
    • ./sdk - Microsoft .NET Core 1.0.1 - SDK

    These are folders within folder /usr/local/share under “Macintosh HD”.

  6. Edit your Bash shell search PATH to include /usr/local/share/dotnet

    atom ~/.bash_profile

    An example:

    PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet
    
  7. Open a new Terminal shell window to run:

    dotnet
    

    The response:

    Microsoft .NET Core Shared Framework Host
      Version  : 1.0.1
      Build    : cee57bf6c981237d80aa1631cfe83cb9ba329f12
    Usage: dotnet [common-options] [[options] path-to-application]
    Common Options:
      --help                           Display .NET Core Shared Framework Host help.
      --version                        Display .NET Core Shared Framework Host version.
    Options:
      --fx-version            Version of the installed Shared Framework to use to run the application.
      --additionalprobingpath    Path containing probing policy and assemblies to probe for.
    Path to Application:
      The path to a .NET Core managed application, dll or exe file to execute.
    If you are debugging the Shared Framework Host, set 'COREHOST_TRACE' to '1' in your environment.
    To get started on developing applications for .NET Core, install .NET SDK from:
      http://go.microsoft.com/fwlink/?LinkID=798306&clcid=0x409
    </pre>   
    
    
  8. In a PowerShell invoke this to ensure that it can be done:

    
    $response = Invoke-WebRequest -Uri "www.microsoft.com" 
    $response.items
    

PowerShellGet Commands

See https://docs.microsoft.com/en-us/powershell/module/powershellget/?view=powershell-7.1

The PowerShellGet module provides commands for discovering, installing, updating and publishing PowerShell artifacts like Modules, DSC Resources, Role Capabilities, and Scripts.

Install-Module AZ -AllowClobber -Force

PowerShell commands

Find-Command - Finds PowerShell commands in modules.

Modules

Get-InstalledModule - Gets a list of modules on the computer that were installed by PowerShellGet.

Install-Module - Downloads one or more modules from a repository, and installs them on the local computer.

Uninstall-Module - Uninstalls a module.

Update-ModuleManifest - Updates a module manifest file.

Find-RoleCapability - Finds role capabilities in modules.

Update-Module - Downloads and installs the newest version of specified modules from an online gallery to the local computer.

Save-Module - Saves a module and its dependencies on the local computer but doesn’t install the module.

Publish-Module - Publishes a specified module from the local computer to an online gallery.

PS Script Linting

TOOL: https://github.com/PowerShell/PSScriptAnalyzer/ PSScriptAnalyzer for static linting of PS Module script code https://poshoholic.com/2015/05/21/powershell-script-analyzer/

  1. Install-Module -Name PSScriptAnalyzer
  2. Type A
  3. Run

    $targetPath="/usr/local/microsoft/powershell/7/Modules/PSReadLine"
    Invoke-ScriptAnalyzer `
      -Path     $targetPath/SamplePSReadLineProfile.ps1 `
      -Settings $targetPath/PSReadLine.psd1
    

    Results:

    Invoke-ScriptAnalyzer: aliasestoexport is not a valid key in the settings hashtable. Valid keys are CustomRulePath, ExcludeRules, IncludeRules, IncludeDefaultRules, RecurseCustomRulePath, Rules and Severity.
     

Invoke-ScriptAnalyzer [-Path] [-CustomRulePat3. <String[]>] [-RecurseCustomRulePath] [-ExcludeRule <String[]>] [-IncludeDefaultRules] [-IncludeRule <String[]>] [-Severity <String[]>] [-Recurse] [-SuppressedOnly] [-Fix] [-EnableExit] [-ReportSummary] [-Settings ] [-SaveDscDependency] []

Get-ScriptAnalyzerRule [-CustomRulePath <String[]>] [-RecurseCustomRulePath] [-Name <String[]>] [-Severity <String[]>] []

Invoke-ScriptAnalyzer [-ScriptDefinition] [-CustomRulePath <String[]>] [-RecurseCustomRulePath] [-ExcludeRule <String[]>] [-IncludeDefaultRules] [-IncludeRule <String[]>] [-Severity <String[]>] [-Recurse] [-SuppressedOnly] [-EnableExit] [-ReportSummary] [-Settings ] [-SaveDscDependency] []

Invoke-Formatter [-ScriptDefinition] [[-Settings] ] [[-Range] <Int32[]>] []

PS Repository

Get-PSRepository - Gets PowerShell repositories.

Set-PSRepository - Sets values for a registered repository.

Register-PSRepository - Registers a PowerShell repository.

Unregister-PSRepository - Unregisters a repository.

PS Script Files

Find-Script - Finds a script.

Install-Script - Installs a script.

Uninstall-Script - Uninstalls a script.

Get-InstalledScript - Gets an installed script.

Update-ScriptFileInfo - Updates information for a script.

New-ScriptFileInfo - Creates a script file with metadata.

Test-ScriptFileInfo - Validates a comment block for a script.

Save-Script - Saves a script.

Update-Script - Updates a script.

Publish-Script - Publishes a script.

DSC Resource commands

Find-DscResource - Finds Desired State Configuration (DSC) resources.


Make Imperative Commands

Windows PowerShell providers access data stores, such as the Windows Registry and certificate store, as easily as you access the file system.

  1. Install NuGet provider:

    Install-PackageProvider -Name NuGet -Force
    
  2. Get a count of how many commands for Azure module:

    Get-Command -Module Azure | Measure-Object
    

    I got a count of 697 commands for just Azure for ASM.

  3. List Azure commands containing “vm” (virtual machine):

    Get-Command -Module Azure -noun *vm*
    

Enable PS1 execution

On Windows machines, PowerShell commands can be script files with .ps1 file extension.

Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy Unrestricted
   

On a Mac:

Set-ExecutionPolicy: Operation is not supported on this platform.

Execute script file

I like using script files rather than typing because it allows me to focus on the latest in what is usually a long string of commands necessary in today’s complex world.

To call scripts, an example:

& ".\basics.ps1"
   

PROTIP: Make sure that when a file with .ps1 extension is clicked from Folder, the script is not launched to run, but that the script appears in a text editor.

A sample command to invoke the script including an execution policy :

Powershell -executionpolicy remotesigned 
-command { import-module ‘C:\Users\pm\Documents\WindowsPowerShell\Modules\MyTwitter’
 ;Send-Tweet -Message ‘Message_ Twitter2’}
   

Notice it’s “powershell” and not “powershell.exe” because Mac and Linux don’t recognize .exe.

When a script is signed, its location is locked to a specific full directory path, even when it’s in the current folder.

“remotesigned” is important because if this script has not been digitally signed, one needs to set PS execution policy to “RemoteSigned” (or “Unrestricted”) after reopening PowerShell as an Administrator to run:

By default PowerShell prevents the execution of PowerShell scripts on Windows systems.

Set-ExecutionPolicy RemoteSigned

Get a list of current security settings:

Get-ExecutionPolicy -List | Format-Table -AutoSize

See https://blog.netspi.com/15-ways-to-bypass-the-powershell-execution-policy/

https://github.com/MeshkDevs/InvokeTwitterAPIs

## Verify a signed script can be used #

  1. Set

    Set-ExecutionPolicy AllSigned

Install a signing cert on Mac

To add the CA root certificate (either PEM or DER format) into the macOS global keychain:

  1. Use Finder to navigate to your /System -> Library -> Keychains -> X509Anchors to your own Library -> Keychains.

  2. In a Terminal shell window, run command:

    certtool i mycertificate.crt k=X509Anchors

    Add a “d” at the end for DER format.

  3. Copy your Library -> Keychains -> X509Anchors back to /System -> Library -> Keychains.

    Use sudo.

Automatic transcript logging

Increasingly, hackers are using PowerShell to create havoc.

So it’s a good idea to automatically log:

start-transcript

PROTIP: This can use up a lot of space quickly, so some management of its use is necessary.

stop-transcript

BLAH: The sample script at https://github.com/wilsonmar/git-utilities/ps-auto-log.ps1, causes errors during execution of scripts.

Inside the file:

**********************
Windows PowerShell transcript start
Start time: 20161209084850
Username: \root
RunAs User: \root
Machine: macs-MacBook-Pro-4 (Microsoft Windows NT 1.0.0.0)
Host Application: 
Process ID: 40107
PSVersion: 6.0.0-alpha
PSEdition: Core
PSCompatibleVersions: 1.0, 2.0, 3.0, 4.0, 5.0, 5.1.10032.0, 6.0.0
BuildVersion: 3.0.0.0
GitCommitId: v6.0.0-alpha.13
WSManStackVersion: 3.0
PSRemotingProtocolVersion: 2.3
SerializationVersion: 1.1.0.1
**********************
Transcript started, output file is ~/Documents/PowerShell/Transcript\2016-12-09T08-48-45-local.txt
   

PS Verbs

All PowerShell cmdlets follow a standardized verb-noun naming convention that makes it easy to look up, find, and use cmdlets.

  1. For a list of all the verbs:

    get-verb

    Verb        AliasPrefix Group          Description
    ----        ----------- -----          -----------
    Add         a           Common         Adds a resource to a container, or attaches an item to another item
    ...
    

    REMEMBER: Capitalization counts within PowerShell.

    ??? AliasProfix

    The output is sorted within PS Command Groupings:

    • Common
    • Communication
    • Data
    • Diagnostic
    • Lifecycle
    • Other
    • Security

  2. List of all comdlets beginning with a verb:

    get-command -verb export

    CommandType     Name                                               Version    Source
    -----------     ----                                               -------    ------
    Alias           Export-AdlStoreChildItemProperties                 1.3.0      Az.DataLakeStore
    
  3. List of all comdlets beginning with a verb:

    get-command -verb export
    get-command -noun ACL

    Environment variables

    READ READ:

  4. List all environment variables, remember the colon at the end:

    Get-ChildItem Env:
    

    An alternate form:

    Get-ChildItem -Path Env:\
    

    Alternately, use the legacy .NET command:

    [System.Environment]::GetEnvironmentVariables()
    

    The first lines in response:

    Name                           Value
    ----                           -----
    _                              /usr/local/bin/pwsh
    __CF_USER_TEXT_ENCODING        0x1F5:0:0
    ...
    
  5. For the value to a specific system variable:

    $env:PATH
    

    Alternately:

    Get-ChildItem Env:PATH
    
  6. Define a temporary environment variable:

    $env:MyTempVariable = "A temporary test variable."
    
  7. Retrieve a User environment variable:

    $env:MyTempVariable
    
  8. To delete an Environment Variable, set its value to an empty string:

    $env:MyTempVariable = ''
    
  9. Define a new permanent environment variable on Windows, containing specified text:

    [Environment]::SetEnvironmentVariable("PermVariableName", "Remember This", "User")
    

    Instead of “User”, the option can be either “Machine”, “User”, or “Process”.

  10. Retrieve a User environment variable:

    [System.Environment]::GetEnvironmentVariable('appdata')
    

    Alternately:

    [Environment]::GetEnvironmentVariable("PermVariableName", "User")
    

NOTE: PowerShell has providers that creates one or more drives, which are hierarchical, file system-like structures that allow a user to manage various areas in Windows. One of those providers is for environment variables called Environment.

Built-in Providers for the Windows operating system:*:

  • Alias - Windows PowerShell aliases {Alias}
  • Certificate - X509 certificates for digital signatures {cert}
  • Environment - Windows environment variables {Env}
  • FileSystem - File system drives, directories and files {filesystem}
  • Function - Windows PowerShell functions {Function}
  • Registry - Windows registry {HKLM, HKCU}
  • Variable - Windows PowerShell variables {Variable}

Version Logic: If Then Else

NOTE: I haven’t found a way to have a Bash script that can also be run as a PowerShell script.

PROTIP: Switching from Bash to PowerShell means a one-time migration and there is no turning back unless you want to maintain parallel scripts.

This is largely because of differences in if/then/else coding. The same if/then/else syntax in PowerShell scripts for Mac and PC is needed for the same script file to be used.

On Bash:

if [ "$IsWindows" = True ]; then
       echo "Do Windows"
   fi
   

If there is a question whether a single PowerShell script can really run on both Mac and Windows. Do a parallel run.

For different actions in PowerShell according to type of operating system:

If ($IsWindows -eq True) { echo "IsWindows"}
   echo "Windows"
   # use "C:/Users/%USERNAME%/.ssh/id_rsa.pub"
ElseIf ($IsOSX -eq True) {"IsOSX"}
   echo "OSX"
   # use "~/.ssh/id_rsa.pub"
Else {"Something else"}
   

NOTE: Because braces define actions, there is no “end if” (“fi”) in PowerShell.

Comparison Operators

-eq / -ne / -ge

-Like / -NotLike wildcard string - $name -Like “*sh”

-Match / -NotMatch regular expression - $name -Match “sh$”

-Contains / -NotContains a value in array - $name -contains “jo”

-In / -NotIn Is a value in an array - “joe” -in $name

Logical operators

-And

-Or

-Xor = Logical exclusive or.

Tilde and Providers

PROTIP: Use $home instead of the tilde (~) in PowerShell because tilde does not always represent the the user’s home folder as in Linux. This is because PS has different “providers” that include HKLM and HKCU top-levels in the Windows Registry.

  1. Get a list of providers and disk space:

    Get-PSDrive

    The response:

    Name           Used (GB)     Free (GB) Provider      Root                      
    ----           ---------     --------- --------      ----                      
    /                 386.19         78.43 FileSystem    /                         
    Alias                                  Alias                                   
    Cert                                   Certificate   \                         
    Env                                    Environment                             
    Function                               Function                                
    Variable                               Variable                 
    

    PowerShell calls files “items” as a term that groups files with registry keys and variables.

    returns the Mode and LastWriteTime of the user.

  2. Instead of “mkdir” to create folders, use

    New-Item

  3. To list files in a folder, it’s the same as in Bash:

    ls -al
    PowerShell cmdlets (command-lets) 
    enables computers to be managed from the command line,
    much like Bash shell scripts on Linux machines.
    How many are there?
    
    (get-command).count
    
    
    At time of writing, the response is 3879.
    
    https://github.com/pester/Pester/wiki/Mock
    
    
    

Alias not parameters

Many Bash commands work in PowerShell (ls, cat, echo) because Aliases make many commands in Bash scripts work:

get-alias echo

The response is what is executed:

Alias           echo -> Write-Output

BLAH: Many parameters to aliases are not recognized. For example, this common command results in an error:

ls -al

Instead, use:

dir -File | format-table

NOTE: dir is an alias to Get-ChildItem.

Thus,

Write-Host $env:computername -foreground Green

”–passthru” means do not go through Pipeline.

PROTIP: You can reset a default alias.

Environment Variables

PROTIP: Environment variables defined in Bash scripts can be read by PowerShell scripts and visa-versa.

  1. List all OS environment variables:

    dir env:\
    

    Alternately, since the command “dir” is an alias of Get-ChildItem.

    Get-ChildItem Env:
    
  2. For the value of a single environment variable:

    Get-ChildItem Env:USER
    Get-ChildItem Env:AWS_DEFAULT_REGION

  3. Save a password data type:

    $mypassword = New-Object System.Management.Automation.PSCredential($username, $SecurePassword) 
    

    According to VIDEO:

    $mypassword = New-Object -TypeName Microoft.Open-AzureAd.Model.PasswordProfile
    $mypassword.Password = "ChangeMe"
    new-azureaduser -DisplayName "John" -PasswordProfile $userpassword -UserPrincipalName johndoe@mycorp.com
    

Handling secrets

PROTIP: Files containing secrets, such as passwords and certificates are NOT stored in GitHub nor script files, but in a separate location, and backed up among other local files.

The secrets are retrieved into the script at run-time.

See my tutorial on GitHub Data Security

Hash tables

BTW, keys in a hash table must be unique.

Hash tables are used throughout PowerShell.

An example of a REST API call:

$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("X-DATE", '9/29/2016')
$headers.Add("X-SIGNATURE", '234j123l4kl23j41l23k4j')
$headers.Add("X-API-KEY", 'testuser')
$response = Invoke-RestMethod 'http://example.com/api/people/1' -Headers $headers
   

Sort a hash tables using the GetEnumertor():

   $source.GetEnumerator() | Sort name | select -first 5
   

Objects

Get-Service m* | where {$_.status -eq ‘running’}
Get-Service m* | where status -eq ‘running’

The “$_” represents the current object in v2 can handle more complexity than v3 syntax:


Remove/Delete whole folder

Instead of “rm -rf” in Bash :

Remove-Item -path c:\* -Filter *text* -WhatIf
   

“-WhatIf” specifies a dry-run.

Combine text in files

Ro add the content of several files into a single text file:

Get-Content "directory path"\*.txt -Force | Set-Content "directory path"\results.txt
   

Cmdlets

PS has some smarter parameters, such as filtering for files only and running recursively into sub-folders:

dir c:\work*.ps1 -file -recurse

paths

Only 25% of cmdlets are shipped with paths.

Strings

PROTIP: Don’t use “+” for string concatenation.

.NET Framework members

  1. Since PowerShell was initially built on Microsoft’s .NET Framework, PowerShell can refer to a static .NET member within square brackets followed by two colons to specify Pi:

    [math]::pi

    Response:

    3.14159265358979

    NOTE: Unlike Python, it’s wonderful that PowerShell doesn’t require an echo command to display the value of objects.

  2. To delete a file in the .NET I/O directory object, such as “tests”:

    [io.directory]::Delete("C:\test*")
    

    The issue with using .NET objects is that they may execute in a different folder context than PowerShell.

    TODO: $prompt

Other pre-defined variables

  1. To count the number of cmdlets:

    $size/1MB
    

    0

    To get the current folder:

    $MyFileName = "data.txt"
    $filebase = $PSScriptRoot + "\" + $MyFileName
    

    Alternatively, use (since v2):

  2. Current folder path in PowerShell is a PathInfo object:

    (Resolve-Path .\).Path
    

    $scriptDir = Split-Path -Path $MyInvocation.MyCommand.Definition -Parent

Dates

Based on http://ss64.com/ps/syntax-dateformats.html

Get-Date -Format "yyyy-MM-dd HH:mm"
   
2021-03-07 05:22
  1. Define variables:

    Get-Date -Format "yyyy-MM-dd HH:mm"
    $time = (Get-Date).AddDays(-30) -Format "yyyy-MM-dd HH:mm"
    $dirName  = [io.path]::GetDirectoryName($path)
    $filename = [io.path]::GetFileNameWithoutExtension($path)
    $ext      = [io.path]::GetExtension($path)
    $newPath  = "$dirName\$filename $(get-date -f yyyy-MM-dd)$ext"
    

Zip files using functions

# http://www.adminarsenal.com/admin-arsenal-blog/powershell-zip-up-files-using-.net-and-add-type/
$SourceFolder    = "C:\temp\Zip This Folder"
$DestinationFile = "C:\temp\NewZip.zip"
$Compression     = "Optimal"  # Optimal, Fastest, NoCompression
 
Zip-Directory -DestinationFileName $DestinationFile `
    -SourceDirectory $SourceFolder `
    -CompressionLevel $Compression ` #Optional parameter
    -IncludeParentDir #Optional parameter
 
function Zip-Directory {
    Param(
      [Parameter(Mandatory=$True)][string]$DestinationFileName,
      [Parameter(Mandatory=$True)][string]$SourceDirectory,
      [Parameter(Mandatory=$False)][string]$CompressionLevel = "Optimal",
      [Parameter(Mandatory=$False)][switch]$IncludeParentDir
    )
    Add-Type -AssemblyName System.IO.Compression.FileSystem
    $CompressionLevel    = [System.IO.Compression.CompressionLevel]::$CompressionLevel  
    [System.IO.Compression.ZipFile]::CreateFromDirectory($SourceDirectory, $DestinationFileName, $CompressionLevel, $IncludeParentDir)
}
   

Pipelines

Instead of just parsing text (as *Nix shells do), PowerShell works with objects in a pipeline.

Piping:

To list all variables defined and their values:

Get-Variable | Out-String

PROTIP: With PowerShell, it’s best to use out-file instead of “>” redirect character:

dir -file -hidden | out-file -filepath rootfiles.txt

Error handling:

Use preference variables for stream redirection:

1> Success 2> Error 3> Warning 4> Verbose 5> Debug

NOTE: Can Only merge to the success stream.

$Error is the automatic array that stores the last 256 exceptions (objects in error) - the default $MaximumErrorCount.

Error action preferences:

0 = Silently Continue
1 = Stop
2 = Continue
3 = Inquire
4 = Ignore [parameter value only]

Module to call Twitter REST API

This suggests:

$J = Invoke-WebRequest `
   -Uri http://search.twitter.com/search.json?q=PowerShell `
    | ConvertFrom-Json
   

PROTIP: To press the trailing back-tick that breaks up a command into several lines, press the key at the upper left corner of the keyboard with your left hand while you press shift key using your right hand.

A space character is required before the tick.

PROTIP: Break up long text into a string block (which Microsoft calls here-string):

$string = @"
item1 = value1 
item2 = value2
"@
$hashtable = ConvertFrom-StringData -StringData $string
$hashtable 
   

The output is:

Name                           Value                                           
----                           -----                                           
item2                          value2                                          
item1                          value1   
   

From https://apps.twitter.com/ define a new app. In Permissions tab, select Read-only. Click Update Settings. In Key and Access Tokens tab, click “Create my access tokens”. Copy the Consumer Key (API key) and paste in ~/.passwords as TWITTER_TOKEN.

It takes many lines to mess with OAuth, so I make use of Adam’s library for Twitter’s v1.1 API described at:
http://www.adamtheautomator.com/twitter-module-powershell/

  1. https://gallery.technet.microsoft.com/scriptcenter/Tweet-and-send-Twitter-DMs-8c2d6f0a
    called “Tweet and send Twitter DMs with Powershell”.

    Adam’s “MyTwitter.psm1” I’ve download had 229 lines on 8/31/2014.

    PROTIP: The “.psm1” extension means it’s a PowerShell module.

    I used a text editor to edit the file to paste in variables for the 4 credentials from Twitter.

       [Parameter()]
       [string]$ApiKey    = $SECRETS.TWITTER_APIKEY,
       [Parameter()]
       [string]$ApiSecret = $SECRETS.TWITTER_APISECRET,
       [Parameter()]
       [string]$AccessToken = $SECRETS.TWITTER_ACCESSTOKEN,
       [Parameter()]
       [string]$AccessTokenSecret = $SECRETS.TWITTER_APISECRET
    

    I then saved the module in the same GitHub folder as my script, and added a command to pull the module into the script:

    Import-module "../MyTwitter.psm1"
    

    Alternately, Copy-install the module to your $env:PSModulePath

    See http://www.powershellgallery.com/gettingstarted

    PowerShellGet from the Windows PowerShell Framework 5.0

    The alternative is to put the module in the PSModulePath, which enables tab completion to complete the names of commands from modules that are not loaded.

    The module has these functions:

    • Get-OAuthAuthorization
    • Send-Tweet
    • Send-TwitterDm

  2. Paste in your PowerShell script:

    Send-Tweet -Message '@adbertram Thanks for the Powershell Twitter module'
    

    BTW, PowerShell cmdlets in https://github.com/Iristyle/Posh-GitHub is only for use on Windows.

Curl

curl is an alias for Invoke-WebRequest in PowerShell.

   Invoke-RestMethod `
   -Method Post `
   -Uri "$resource\new" `
   -Body (ConvertTo-Json $body) `
   -Header @{"X-ApiKey"=$apiKey}
   

See https://channel9.msdn.com/Blogs/trevor-powershell/Automating-the-GitHub-REST-API-Using-PowerShell

JSON from REST API

To extract out a key from the JSON file:

$x.Stuffs | where { $_.Name -eq "Darts" } 
   

https://www.pluralsight.com/courses/powershell-modules-advanced-functions-building

API calls

Corporate IT departments often use Group Policies.

   $Headers = "Authorization: token ${GITHUB_TOKEN}"
   echo "Headers=$Headers"  # DEBUGGING

   $Token=$GITHUBUSER +':'+ $SECRETS.GITHUB_TOKEN;
   $Base64Token=[System.Convert]::ToBase64String([char[]]$Token);
   $Headers = @{
      Authorization = 'Basic(0)' -f $Base64Token;
      };
      # -f is for substitution of (0).
      # See https://technet.microsoft.com/en-us/library/ee692795.aspx
      # Write-Host ("Headers="+$Headers.Authorization)
   $Headers = "{
      Authorization: = Basic $GITHUB_TOKEN
      }"
      # -f is for substitution of (0).
      # See https://technet.microsoft.com/en-us/library/ee692795.aspx
      Write-Host ("Headers="+$Headers)
   

Profile scripts

Jeff Hicks notes these profile scripts execute automatically at start:

  1. To view all profiles:

    $profile | select *

    Response:

Folder Script file Script name
C:\Windows\ System32\ WindowsPowerShell\ v1.0\ profile.ps1 AllUsersAllHosts
Microsoft.PowerShell.profile.ps1 AllUsersCurrentHost
Microsoft.PowerShellSE.profile.ps1 AllUsersCurrentHost (ISE)
C:\Users\<user>\Documents\ WindowsPowerShell\ or /Users/<user>/ .config/powershell/ Microsoft.PowerShell.profile.ps1 CurrentUsersAllHosts*
profile.ps1 CurrentUserCurrentHost
Microsoft.PowerShellSE.profile.ps1 CurrentUserCurrentHost (ISE)
  • = This is the one shown when $profile is typed in.

Iterate

  1. Stephane shows this command to move (pipe) png files from Desktop to Pictures folder:

    Get-ChildItem -Filter '*.png' | Move-Item -Destination '../Pictures'
    

    A variable can contain an array:

    
    $files = dir c:\script -file
    Foreach ($file in $files){
       $fileage = ((get-Date)) - $file.LastWriteTime )
       "$($file.name) = $fileage" | Out-File ...
    }
    

    Remotely restart a server

Read in CSV file

This blog gives an example of importing a CSV file:

$data = Import-CSV C:\scripts\moviedata.csv

PROTIP: Sorting by date requires creating a new property:


   $data | Add-Member -MemberType ScriptProperty `
   -Name "OpensIn" `
   -Value { [int32]((($this.ReleaseDate `
      -as [DateTime]) - (Get-Date)).TotalDays)  }
   

The new property persists, so can be used this way:


   $data | 
   Sort "OpensIn" |
   Select Title.ReleaseDate.OpensIn.Coments
   

PowerShell Remoting

https://docs.microsoft.com/en-us/powershell/scripting/learn/remoting/winrmsecurity?view=powershell-7.1 Security Considerations for PowerShell Remoting using WinRM

Social

https://powershell.slack.com

More PS Libraries

https://www.simple-talk.com/blogs/psyaml-powershell-yaml

References

“Practical PowerShell for IT Security”:

https://docs.broadcom.com/doc/increased-use-of-powershell-in-attacks-16-en

More on DevSecOps

This is one of a series on DevSecOps:

  1. DevOps_2.0
  2. ci-cd (Continuous Integration and Continuous Delivery)
  3. User Stories for DevOps
  4. Enterprise Software)

  5. Git and GitHub vs File Archival
  6. Git Commands and Statuses
  7. Git Commit, Tag, Push
  8. Git Utilities
  9. Data Security GitHub
  10. GitHub API
  11. TFS vs. GitHub

  12. Choices for DevOps Technologies
  13. Pulumi Infrastructure as Code (IaC)
  14. Java DevOps Workflow
  15. Okta for SSO & MFA

  16. AWS DevOps (CodeCommit, CodePipeline, CodeDeploy)
  17. AWS server deployment options
  18. AWS Load Balancers

  19. Cloud services comparisons (across vendors)
  20. Cloud regions (across vendors)
  21. AWS Virtual Private Cloud

  22. Azure Cloud Onramp (Subscriptions, Portal GUI, CLI)
  23. Azure Certifications
  24. Azure Cloud

  25. Azure Cloud Powershell
  26. Bash Windows using Microsoft’s WSL (Windows Subsystem for Linux)
  27. Azure KSQL (Kusto Query Language) for Azure Monitor, etc.

  28. Azure Networking
  29. Azure Storage
  30. Azure Compute
  31. Azure Monitoring

  32. Digital Ocean
  33. Cloud Foundry

  34. Packer automation to build Vagrant images
  35. Terraform multi-cloud provisioning automation
  36. Hashicorp Vault and Consul to generate and hold secrets

  37. Powershell Ecosystem
  38. Powershell on MacOS
  39. Powershell Desired System Configuration

  40. Jenkins Server Setup
  41. Jenkins Plug-ins
  42. Jenkins Freestyle jobs
  43. Jenkins2 Pipeline jobs using Groovy code in Jenkinsfile

  44. Docker (Glossary, Ecosystem, Certification)
  45. Make Makefile for Docker
  46. Docker Setup and run Bash shell script
  47. Bash coding
  48. Docker Setup
  49. Dockerize apps
  50. Docker Registry

  51. Maven on MacOSX

  52. Ansible
  53. Kubernetes Operators
  54. OPA (Open Policy Agent) in Rego language

  55. MySQL Setup

  56. Threat Modeling
  57. SonarQube & SonarSource static code scan

  58. API Management Microsoft
  59. API Management Amazon

  60. Scenarios for load
  61. Chaos Engineering