Make PowerShell your superpower on Windows and Macs, for fun and profit
Overview
- Open source on Linux and MacOS
- Install PowerShell on Windows
- Install PowerShell Core on MacOS
- Configure Terminal colors
- Output continuation & wide view
- Visual Studio Code Editor
- Install .NET Core
- PowerShellGet Commands
- Modules
- PS Script Linting
- Make Imperative Commands
- Execute script file
- Automatic transcript logging
- PS Verbs
- Environment variables
- Version Logic: If Then Else
- Tilde and Providers
- Alias not parameters
- Environment Variables
- Handling secrets
- Hash tables
- Objects
- Remove/Delete whole folder
- Combine text in files
- Cmdlets
- paths
- Strings
- Dates
- Zip files using functions
- Pipelines
- Module to call Twitter REST API
- JSON from REST API
- API calls
- Iterate
- Read in CSV file
- PowerShell Remoting
- Social
- More PS Libraries
- References
- More on DevSecOps
This article describes the use of PowerShell scripting on Mac and Linux.
“PowerShell” refers to both the command-line shell AND scripting language designed system administration. (When “PowerShell Core 6.0” was announced on January 10, 2018, the word “Powershell” on its own now refers to the decade-old “PowerShell” integrated into all recent versions of Microsoft’s Windows operating system.)
PowerShell is an object-centered “management engine” that can be hosted in an application program:
- CMD in Windows
- PowerShell ISE (from Microsoft)
- PowerGUI
- SAPIEN Technologies PowerShell Studio
- Idera PowerShell Pro
PowerShell Core is available as a cross-platform application such that scripts written on MacOS will run on Windows, Linux, or other supported operating system. So it does not have commands associated with the .NET Framework (for Windows OS). Such is similar to the rebranding of .NET vs. .NET Core.
PowerShell cmdlets (pronounced “command-lets”) let you manage computers from the command line.
ISE (Integrated Scripting Environment) is a GUI program that provides popup tab completion and other assists.
PowerShell promises more consistency than the various commands added over time by various parties:
- It reads Excel files natively as well as JSON, XML, and even ASCII.
- Microsoft Deployment Toolkit
- Microsoft System Center
- IBM, etc.
The above and other scripting is covered in my blog wilsonmar.github.io/azure-cloud-powershell
Open source on Linux and MacOS
PowerShell is open-sourced for all OSs at https://github.com/PowerShell/PowerShell.
Install PowerShell on Windows
https://aka.ms/PSWindows for PowerShell 7 on Windows 11 installed to: $env:ProgramFiles\PowerShell\7
choco install powershell-core
Windows PowerShellGet Module if you don’t want to install these from the Web Platform Installer (wpilauncher.exe) at https://www.microsoft.com/web/downloads/platform.aspx
I have a separate blog wilsonmar.github.io/powershell-dsc
Install PowerShell Core on MacOS
PowerShell Core supports macOS 10.12 and higher. See this for other os
-
This says needed as a pre-requisite is
xcode-select --install
-
PROTIP: Skip the manual install and install using Homebrew.
Alternately, click to download the latest release for MacOS at:
https://github.com/PowerShell/PowerShellAlternately, get back versions at
https://github.com/PowerShell/PowerShell/releasesDate File MB Size Space Cmds 21 May 2021 7.2.0-preview.6 62.1 MB Apr, 2018 6.0.2 on brew 50.8 MB Sep 13, 2017 powershell-6.0.0-beta.7-osx.10.12-x64.pkg 50.8 MB Sep 13, 2016 powershell-6.0.0-alpha.10.pkg 28.2 MB ? MB 345 Aug 10, 2016 powershell-6.0.0-alpha.9.pkg 37.1 MB 119.7 MB Jul 26, 2016 powershell-6.0.0-alpha.7.pkg 25.0 MB Jul 8, 2016 powershell-0.6.0.pkg 24.2 MB - Open the .pkg file in the Downloads folder:
-
Click Continue, etc.
NOTE: For Windows: Microsoft Windows Management Framework 5.0
Install using Homebrew
-
Alternately, use Homebrew:
brew install powershell
==> Caveats To use Homebrew in PowerShell, set: Add-Content -Path $PROFILE.CurrentUserAllHosts -Value '$(/usr/local/bin/brew shellenv) | Invoke-Expression' ==> Downloading https://github.com/PowerShell/PowerShell/releases/download/v7.3. ==> Downloading from https://objects.githubusercontent.com/github-production-rel ######################################################################### 100.0% All formula dependencies satisfied. ==> Installing Cask powershell ==> Running installer for powershell; your password may be necessary. Package installers may write to any location; options such as `--appdir` are ignored. Password: ____
-
ATTENTION: Enter your laptop password when prompted.
Response:
installer: Package name is PowerShell - 7.4.1 installer: Installing at base path / installer: The install was successful. 🍺 powershell was successfully installed!
Run PowerShell in Bash
-
An example of command parameters within double-quotes:
po
“Hello World” would be the response.
-
Double-quotes are not needed for a single command, such as this to list folders (child items):
pwsh -command get-childitem
Directory: /Users/... Mode LastWriteTime Length Name ---- ------------- ------ ---- ----- 5/12/2021 3:48 PM 11381 .dir
In and Out
-
Open a Terminal shell window to launch PowerShell
on MacOS:pwsh
On Windows:
powershell
The response:
PowerShell 7.4.1 PS /Users/...>
PS
displayed means you are in the PowerShell shell.Gone is line: https://aka.ms/pscore6-docs
Sometimes the response also has:
A new PowerShell stable release is available: v7.4.1 Upgrade now, or check out the release page at: https://aka.ms/PowerShell-Release?tag=v7.3.7
System Variables
-
List all system variables:
dir env:
Alternately, to sort by name:
gci env:* | sort-object name
“gci” is short for:
Get-ChildItem Env:* | Select-Object -Property Name,Value
Notice there are “LOGNAME” and “USER” variables.
-
The content of system environment variable name $USER can be identified quickest using a command on both Linux and Windows:
uname
-
To display just the value of the $HOME variable which defines the path where the “cd” command navigates to:
Get-Variable HOME -valueOnly
On MacOS and Linux, for example (replacing “johndoe” with your user name):
/Users/johndoe
On Windows:
C:\Users\johndoe
Customize command prompt
-
The default prompt is defined by this:
function prompt { 'PS ' + $(get-location) + '> ' }
The default command prompt contains “>” after the current path.
I don’t like this because the prompt appears in different positions, requiring me to spend time finding it. And there isn’t much space left for commands before forced wrapping.
I would rather have the prompt be at the same position, such as this, which gives a lot of space for long commands:
> _
I don’t need “PS”. Above the prompt, I show time of day, [Git branch], and current folder:
03:54:32 PM [master] _posts
_ </tt>
-
To achieve the above prompt,
function prompt {"`n "+$(Get-Date -UFormat "%r")+' ['+$(git rev-parse --abbrev-ref HEAD)+'] '+$((get-item $pwd ).Name)+"`n"+'> ' }
PROTIP: In PowerShell, a new line is specified by the “back-tick” escape character (at the upper-left of most keyboards). Also notice double quotation marks are necessary. The back-tick is also used for line continuation, so don’t put a space after a back-tick or PowerShell will recognize it as an escape character rather than a line continuation.
An alternative to obtaining the current folder is:
$($executionContext.SessionState.Path.CurrentLocation | Split-Path -Leaf)
NOTE: Others display different colors.
-
To make the change permanant, change your user profile definition file at the path defined by the $PROFILE system environment variable:
$PROFILE
On MacOS and Linux:
/Users/USER/.config/powershell/Microsoft.PowerShell_profile.ps1
On Windows:
C:\Users\USER\Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1
NOTE: That file is one of six PowerShell profile files*
-
Check if it’s true that you have a profile file:
Test-Path $PROFILE
If “False”, create the file “Microsoft.PowerShell_profile.ps1”
New-Item -Path $PROFILE -Type File -force
-
Edit the file “Microsoft.PowerShell_profile.ps1”:
code $PROFILE
- Copy and paste the function prompt from above.
-
Click the “…” at the right to Save and Close Editor (or press Command+S and Command+Q).
Exit PowerShell
-
To leave PowerShell for changes to take effect, it’s the same as in Bash scripts:
exit
-
Enter Powershell again, per above.
Upgrade PowerShell
-
To upgrade (within Bash or pwsh):
brew upgrade --cask powershell
Verify install by seeing version
-
Check the version of PowerShell being used by calling a pre-defined variable:
$psversiontable
PROTIP: With PowerShell, a variable can act like a command.
Response:
Name Value ---- ----- PSVersion 7.1.3 PSEdition Core GitCommitId 7.1.3 OS Darwin 19.6.0 Darwin Kernel Version 19.6.0: Tue Jan 12 22:13:05 PST 2021; root:xnu-6153.141.16~1/RELEASE_X86_64 Platform Unix PSCompatibleVersions {1.0, 2.0, 3.0, 4.0…} PSRemotingProtocolVersion 2.3 SerializationVersion 1.1.0.1 WSManStackVersion 3.0
QUESTION: What’s the CLRVersion?
- 7.0
- 6.0 for Mac/Linux in Windows 10 Anniversay Edition
- 5.0 in 2015 for Visual Studio Code text editor
- 4.0 in 2014 with Windows 10 and .NET Framework 4.0 and Windows Management Framework 3.0
- 3.0 in 2012 with Windows 8/Server 2012
- 2.0 appeared in 2009
- 1.0 appeared in 2006
- Monad Manifesto published by Jeff Stover.
PROTIP: Know the PowerShell commands known not to work on Linux/macOS.
Configure Terminal colors
-
Configure your Terminal colors
See https://sqlsunday.com/2019/03/04/how-to-set-up-a-beautiful-powershell-core-terminal-on-mac-os/
-
Clear PS Screen
This doesn’t need any modules installed:
-
Clear Screen:
cls
Get Help on commands
-
Download help files:
update-help -force
-
Get help information for a command (such as stop-service):
get-help stop-service
Name Category Module Synopsis ---- -------- ------ -------- Register-ArgumentCompleter Cmdlet Microsoft.PowerShell.Core Registers a custom argument completer. about_If HelpFile about_Pipelines HelpFile
Output continuation & wide view
PROTIP: To continue a line end (like back-slash in Bash), use “tick marks” (` at the upper-left on Mac keyboards) and use “vertical bar” ( at the right of Mac keyboards) : Get-Alias -Definition Invoke-* ` | Format-Table -Property * -AutoSize ` | Out-String -Width 4096 ` | Out-File aliases.txt
Visual Studio Code Editor
One text editor built for PowerShell is Microsoft’s Visual Studio Code.
-
Install Visual Studio Code (see https://chocolatey.org/packages/VisualStudioCode):
choco install visualstudiocode -y
-
Install the PowerShell add-in to VSCode:
choco install vscode-powershell -y
-
Install the PowerShell Editor Services extension by pressing Ctrl+P, then type “ext install PowerShell” for a list of add-ins.
Ctrl+P is the universal search that also does “fuzzy search” of text in files open.
-
Click “install” of the extension named “PowerShell”. The icon turns to “installing”.
-
Open a directory containing PowerShell scripts and open the File menu and select “Open Folder …”. Select the folder containing your scripts.
The scripts show up in the Explore tab of the Side Bar.PROTIP: One advantage using VS Code is its Side Bar enabling you to switch quickly among different files.
Press Ctrl+B to hide and unhide the Side Bar.
-
Press Ctrl+\ to open a new editor window.
Up to three editor panes can be open at once.
Press Ctrl+1, 2, or 3 to switch among the files.
-
To edit user settings, press Ctrl+Shift+P, then type “user” and press enter.
-
Click on “powershell.scriptAnalysis.enable”.
-
Press Ctrl+Shift+<period> to change value from true to false or back again.
Keith Hill notes debugging support provided by the PowerShell Editor Services extension currently runs only on Windows.
Install .NET Core
PowerShell is written on top of .NET. .NET’s previous dependencies on Windows components have been removed in .NET Core.
PowerShell errors occur if .NET Core is not installed, so:
-
Go to web page https://www.microsoft.com/net/core#macos
-
The web page asks for OpenSSL to be installed.
On a Mac:
brew update brew install openssl ln -s /usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib /usr/local/lib/ ln -s /usr/local/opt/openssl/lib/libssl.1.0.0.dylib /usr/local/lib/
-
Click the link to download the 50.3MB
dotnet-dev-osx-x64.1.0.0-preview2-003131.pkghttps://github.com/dotnet/core/blob/master/cli/known-issues.md
-
Run the installer (for 106.3MB of space).
-
Before installing anything or running through the update app, hit Command+i or pull down the File menu and choose “Show Files”:
- ./shared - Microsoft .NET Core 1.0.1 - Runtime
- ./host - Microsoft .NET Core 1.0.1 - Host FX Resolver
- ./dotnet
- ./sdk - Microsoft .NET Core 1.0.1 - SDK
These are folders within folder /usr/local/share under “Macintosh HD”.
-
Edit your Bash shell search PATH to include /usr/local/share/dotnet
atom ~/.bash_profile
An example:
PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet
-
Open a new Terminal shell window to run:
dotnet
The response:
Microsoft .NET Core Shared Framework Host Version : 1.0.1 Build : cee57bf6c981237d80aa1631cfe83cb9ba329f12 Usage: dotnet [common-options] [[options] path-to-application] Common Options: --help Display .NET Core Shared Framework Host help. --version Display .NET Core Shared Framework Host version. Options: --fx-version
Version of the installed Shared Framework to use to run the application. --additionalprobingpath Path containing probing policy and assemblies to probe for. Path to Application: The path to a .NET Core managed application, dll or exe file to execute. If you are debugging the Shared Framework Host, set 'COREHOST_TRACE' to '1' in your environment. To get started on developing applications for .NET Core, install .NET SDK from: http://go.microsoft.com/fwlink/?LinkID=798306&clcid=0x409 </pre> -
In a PowerShell invoke this to ensure that it can be done:
$response = Invoke-WebRequest -Uri "www.microsoft.com" $response.items
PowerShellGet Commands
See https://docs.microsoft.com/en-us/powershell/module/powershellget/?view=powershell-7.1
The PowerShellGet module provides commands for discovering, installing, updating and publishing PowerShell artifacts like Modules, DSC Resources, Role Capabilities, and Scripts.
Install-Module AZ -AllowClobber -Force
PowerShell commands
Find-Command - Finds PowerShell commands in modules.
Modules
Get-InstalledModule - Gets a list of modules on the computer that were installed by PowerShellGet.
Install-Module - Downloads one or more modules from a repository, and installs them on the local computer.
Uninstall-Module - Uninstalls a module.
Update-ModuleManifest - Updates a module manifest file.
Find-RoleCapability - Finds role capabilities in modules.
Update-Module - Downloads and installs the newest version of specified modules from an online gallery to the local computer.
Save-Module - Saves a module and its dependencies on the local computer but doesn’t install the module.
Publish-Module - Publishes a specified module from the local computer to an online gallery.
PS Script Linting
TOOL: https://github.com/PowerShell/PSScriptAnalyzer/ PSScriptAnalyzer for static linting of PS Module script code https://poshoholic.com/2015/05/21/powershell-script-analyzer/
- Install-Module -Name PSScriptAnalyzer
- Type A
-
Run
$targetPath="/usr/local/microsoft/powershell/7/Modules/PSReadLine" Invoke-ScriptAnalyzer ` -Path $targetPath/SamplePSReadLineProfile.ps1 ` -Settings $targetPath/PSReadLine.psd1
Results:
Invoke-ScriptAnalyzer: aliasestoexport is not a valid key in the settings hashtable. Valid keys are CustomRulePath, ExcludeRules, IncludeRules, IncludeDefaultRules, RecurseCustomRulePath, Rules and Severity.
Invoke-ScriptAnalyzer [-Path]
Get-ScriptAnalyzerRule [-CustomRulePath <String[]>] [-RecurseCustomRulePath] [-Name <String[]>] [-Severity <String[]>] [
Invoke-ScriptAnalyzer [-ScriptDefinition]
Invoke-Formatter [-ScriptDefinition]
PS Repository
Get-PSRepository - Gets PowerShell repositories.
Set-PSRepository - Sets values for a registered repository.
Register-PSRepository - Registers a PowerShell repository.
Unregister-PSRepository - Unregisters a repository.
PS Script Files
Find-Script - Finds a script.
Install-Script - Installs a script.
Uninstall-Script - Uninstalls a script.
Get-InstalledScript - Gets an installed script.
Update-ScriptFileInfo - Updates information for a script.
New-ScriptFileInfo - Creates a script file with metadata.
Test-ScriptFileInfo - Validates a comment block for a script.
Save-Script - Saves a script.
Update-Script - Updates a script.
Publish-Script - Publishes a script.
DSC Resource commands
Find-DscResource - Finds Desired State Configuration (DSC) resources.
Make Imperative Commands
Windows PowerShell providers access data stores, such as the Windows Registry and certificate store, as easily as you access the file system.
-
Install NuGet provider:
Install-PackageProvider -Name NuGet -Force
-
Get a count of how many commands for Azure module:
Get-Command -Module Azure | Measure-Object
I got a count of 697 commands for just Azure for ASM.
-
List Azure commands containing “vm” (virtual machine):
Get-Command -Module Azure -noun *vm*
Enable PS1 execution
On Windows machines, PowerShell commands can be script files with .ps1 file extension.
Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy Unrestricted
On a Mac:
Set-ExecutionPolicy: Operation is not supported on this platform.
Execute script file
I like using script files rather than typing because it allows me to focus on the latest in what is usually a long string of commands necessary in today’s complex world.
To call scripts, an example:
& ".\basics.ps1"
PROTIP: Make sure that when a file with .ps1 extension is clicked from Folder, the script is not launched to run, but that the script appears in a text editor.
A sample command to invoke the script including an execution policy :
Powershell -executionpolicy remotesigned -command { import-module ‘C:\Users\pm\Documents\WindowsPowerShell\Modules\MyTwitter’ ;Send-Tweet -Message ‘Message_ Twitter2’}
Notice it’s “powershell” and not “powershell.exe” because Mac and Linux don’t recognize .exe.
When a script is signed, its location is locked to a specific full directory path, even when it’s in the current folder.
“remotesigned” is important because if this script has not been digitally signed, one needs to set PS execution policy to “RemoteSigned” (or “Unrestricted”) after reopening PowerShell as an Administrator to run:
By default PowerShell prevents the execution of PowerShell scripts on Windows systems.
Set-ExecutionPolicy RemoteSigned
Get a list of current security settings:
Get-ExecutionPolicy -List | Format-Table -AutoSize
See https://blog.netspi.com/15-ways-to-bypass-the-powershell-execution-policy/
https://github.com/MeshkDevs/InvokeTwitterAPIs
## Verify a signed script can be used #
-
Set
Set-ExecutionPolicy AllSigned
Install a signing cert on Mac
To add the CA root certificate (either PEM or DER format) into the macOS global keychain:
-
Use Finder to navigate to your /System -> Library -> Keychains -> X509Anchors to your own Library -> Keychains.
-
In a Terminal shell window, run command:
certtool i mycertificate.crt k=X509Anchors
Add a “d” at the end for DER format.
-
Copy your Library -> Keychains -> X509Anchors back to /System -> Library -> Keychains.
Use sudo.
Automatic transcript logging
Increasingly, hackers are using PowerShell to create havoc.
So it’s a good idea to automatically log:
start-transcript
PROTIP: This can use up a lot of space quickly, so some management of its use is necessary.
stop-transcript
BLAH: The sample script at https://github.com/wilsonmar/git-utilities/ps-auto-log.ps1, causes errors during execution of scripts.
Inside the file:
********************** Windows PowerShell transcript start Start time: 20161209084850 Username: \root RunAs User: \root Machine: macs-MacBook-Pro-4 (Microsoft Windows NT 1.0.0.0) Host Application: Process ID: 40107 PSVersion: 6.0.0-alpha PSEdition: Core PSCompatibleVersions: 1.0, 2.0, 3.0, 4.0, 5.0, 5.1.10032.0, 6.0.0 BuildVersion: 3.0.0.0 GitCommitId: v6.0.0-alpha.13 WSManStackVersion: 3.0 PSRemotingProtocolVersion: 2.3 SerializationVersion: 1.1.0.1 ********************** Transcript started, output file is ~/Documents/PowerShell/Transcript\2016-12-09T08-48-45-local.txt
PS Verbs
All PowerShell cmdlets follow a standardized verb-noun naming convention that makes it easy to look up, find, and use cmdlets.
-
For a list of all the verbs:
get-verb
Verb AliasPrefix Group Description ---- ----------- ----- ----------- Add a Common Adds a resource to a container, or attaches an item to another item ...
REMEMBER: Capitalization counts within PowerShell.
??? AliasProfix
The output is sorted within PS Command Groupings:
- Common
- Communication
- Data
- Diagnostic
- Lifecycle
- Other
- Security
-
List of all comdlets beginning with a verb:
get-command -verb export
CommandType Name Version Source ----------- ---- ------- ------ Alias Export-AdlStoreChildItemProperties 1.3.0 Az.DataLakeStore
-
List of all comdlets beginning with a verb:
get-command -verb export
get-command -noun ACLEnvironment variables
-
List all environment variables, remember the colon at the end:
Get-ChildItem Env:
An alternate form:
Get-ChildItem -Path Env:\
Alternately, use the legacy .NET command:
[System.Environment]::GetEnvironmentVariables()
The first lines in response:
Name Value ---- ----- _ /usr/local/bin/pwsh __CF_USER_TEXT_ENCODING 0x1F5:0:0 ...
-
For the value to a specific system variable:
$env:PATH
Alternately:
Get-ChildItem Env:PATH
-
Define a temporary environment variable:
$env:MyTempVariable = "A temporary test variable."
-
Retrieve a User environment variable:
$env:MyTempVariable
-
To delete an Environment Variable, set its value to an empty string:
$env:MyTempVariable = ''
-
Define a new permanent environment variable on Windows, containing specified text:
[Environment]::SetEnvironmentVariable("PermVariableName", "Remember This", "User")
Instead of “User”, the option can be either “Machine”, “User”, or “Process”.
-
Retrieve a User environment variable:
[System.Environment]::GetEnvironmentVariable('appdata')
Alternately:
[Environment]::GetEnvironmentVariable("PermVariableName", "User")
NOTE: PowerShell has providers that creates one or more drives, which are hierarchical, file system-like structures that allow a user to manage various areas in Windows. One of those providers is for environment variables called Environment.
Built-in Providers for the Windows operating system:*:
- Alias - Windows PowerShell aliases {Alias}
- Certificate - X509 certificates for digital signatures {cert}
- Environment - Windows environment variables {Env}
- FileSystem - File system drives, directories and files {filesystem}
- Function - Windows PowerShell functions {Function}
- Registry - Windows registry {HKLM, HKCU}
- Variable - Windows PowerShell variables {Variable}
Version Logic: If Then Else
NOTE: I haven’t found a way to have a Bash script that can also be run as a PowerShell script.
PROTIP: Switching from Bash to PowerShell means a one-time migration and there is no turning back unless you want to maintain parallel scripts.
This is largely because of differences in if/then/else coding. The same if/then/else syntax in PowerShell scripts for Mac and PC is needed for the same script file to be used.
On Bash:
if [ "$IsWindows" = True ]; then echo "Do Windows" fi
If there is a question whether a single PowerShell script can really run on both Mac and Windows. Do a parallel run.
For different actions in PowerShell according to type of operating system:
If ($IsWindows -eq True) { echo "IsWindows"} echo "Windows" # use "C:/Users/%USERNAME%/.ssh/id_rsa.pub" ElseIf ($IsOSX -eq True) {"IsOSX"} echo "OSX" # use "~/.ssh/id_rsa.pub" Else {"Something else"}
NOTE: Because braces define actions, there is no “end if” (“fi”) in PowerShell.
Comparison Operators
-eq / -ne / -ge
-Like / -NotLike wildcard string - $name -Like “*sh”
-Match / -NotMatch regular expression - $name -Match “sh$”
-Contains / -NotContains a value in array - $name -contains “jo”
-In / -NotIn Is a value in an array - “joe” -in $name
Logical operators
-And
-Or
-Xor = Logical exclusive or.
Tilde and Providers
PROTIP: Use $home instead of the tilde (~) in PowerShell because tilde does not always represent the the user’s home folder as in Linux. This is because PS has different “providers” that include HKLM and HKCU top-levels in the Windows Registry.
-
Get a list of providers and disk space:
Get-PSDrive
The response:
Name Used (GB) Free (GB) Provider Root ---- --------- --------- -------- ---- / 386.19 78.43 FileSystem / Alias Alias Cert Certificate \ Env Environment Function Function Variable Variable
PowerShell calls files “items” as a term that groups files with registry keys and variables.
returns the Mode and LastWriteTime of the user.
-
Instead of “mkdir” to create folders, use
New-Item
-
To list files in a folder, it’s the same as in Bash:
ls -al
PowerShell cmdlets (command-lets) enables computers to be managed from the command line, much like Bash shell scripts on Linux machines. How many are there? (get-command).count At time of writing, the response is 3879. https://github.com/pester/Pester/wiki/Mock
Alias not parameters
Many Bash commands work in PowerShell (ls, cat, echo) because Aliases make many commands in Bash scripts work:
get-alias echo
The response is what is executed:
Alias echo -> Write-Output
BLAH: Many parameters to aliases are not recognized. For example, this common command results in an error:
ls -al
Instead, use:
dir -File | format-table
NOTE: dir is an alias to Get-ChildItem.
Thus,
Write-Host $env:computername -foreground Green
”–passthru” means do not go through Pipeline.
PROTIP: You can reset a default alias.
Environment Variables
PROTIP: Environment variables defined in Bash scripts can be read by PowerShell scripts and visa-versa.
-
List all OS environment variables:
dir env:\
Alternately, since the command “dir” is an alias of Get-ChildItem.
Get-ChildItem Env:
-
For the value of a single environment variable:
Get-ChildItem Env:USER
Get-ChildItem Env:AWS_DEFAULT_REGION -
Save a password data type:
$mypassword = New-Object System.Management.Automation.PSCredential($username, $SecurePassword)
According to VIDEO:
$mypassword = New-Object -TypeName Microoft.Open-AzureAd.Model.PasswordProfile $mypassword.Password = "ChangeMe" new-azureaduser -DisplayName "John" -PasswordProfile $userpassword -UserPrincipalName johndoe@mycorp.com
Handling secrets
PROTIP: Files containing secrets, such as passwords and certificates are NOT stored in GitHub nor script files, but in a separate location, and backed up among other local files.
The secrets are retrieved into the script at run-time.
See my tutorial on GitHub Data Security
Hash tables
BTW, keys in a hash table must be unique.
Hash tables are used throughout PowerShell.
An example of a REST API call:
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]" $headers.Add("X-DATE", '9/29/2016') $headers.Add("X-SIGNATURE", '234j123l4kl23j41l23k4j') $headers.Add("X-API-KEY", 'testuser') $response = Invoke-RestMethod 'http://example.com/api/people/1' -Headers $headers
Sort a hash tables using the GetEnumertor():
$source.GetEnumerator() | Sort name | select -first 5
Objects
Get-Service m* | where {$_.status -eq ‘running’}
Get-Service m* | where status -eq ‘running’
The “$_” represents the current object in v2 can handle more complexity than v3 syntax:
Remove/Delete whole folder
Instead of “rm -rf” in Bash :
Remove-Item -path c:\* -Filter *text* -WhatIf
“-WhatIf” specifies a dry-run.
Combine text in files
Ro add the content of several files into a single text file:
Get-Content "directory path"\*.txt -Force | Set-Content "directory path"\results.txt
Cmdlets
PS has some smarter parameters, such as filtering for files only and running recursively into sub-folders:
dir c:\work*.ps1 -file -recurse
paths
Only 25% of cmdlets are shipped with paths.
Strings
PROTIP: Don’t use “+” for string concatenation.
.NET Framework members
-
Since PowerShell was initially built on Microsoft’s .NET Framework, PowerShell can refer to a static .NET member within square brackets followed by two colons to specify Pi:
[math]::pi
Response:
3.14159265358979
NOTE: Unlike Python, it’s wonderful that PowerShell doesn’t require an echo command to display the value of objects.
-
To delete a file in the .NET I/O directory object, such as “tests”:
[io.directory]::Delete("C:\test*")
The issue with using .NET objects is that they may execute in a different folder context than PowerShell.
TODO: $prompt
Other pre-defined variables
-
To count the number of cmdlets:
$size/1MB
0
To get the current folder:
$MyFileName = "data.txt" $filebase = $PSScriptRoot + "\" + $MyFileName
Alternatively, use (since v2):
-
Current folder path in PowerShell is a PathInfo object:
(Resolve-Path .\).Path
$scriptDir = Split-Path -Path $MyInvocation.MyCommand.Definition -Parent
Dates
Based on http://ss64.com/ps/syntax-dateformats.html
Get-Date -Format "yyyy-MM-dd HH:mm"
2021-03-07 05:22
-
Define variables:
Get-Date -Format "yyyy-MM-dd HH:mm" $time = (Get-Date).AddDays(-30) -Format "yyyy-MM-dd HH:mm" $dirName = [io.path]::GetDirectoryName($path) $filename = [io.path]::GetFileNameWithoutExtension($path) $ext = [io.path]::GetExtension($path) $newPath = "$dirName\$filename $(get-date -f yyyy-MM-dd)$ext"
Zip files using functions
# http://www.adminarsenal.com/admin-arsenal-blog/powershell-zip-up-files-using-.net-and-add-type/ $SourceFolder = "C:\temp\Zip This Folder" $DestinationFile = "C:\temp\NewZip.zip" $Compression = "Optimal" # Optimal, Fastest, NoCompression Zip-Directory -DestinationFileName $DestinationFile ` -SourceDirectory $SourceFolder ` -CompressionLevel $Compression ` #Optional parameter -IncludeParentDir #Optional parameter function Zip-Directory { Param( [Parameter(Mandatory=$True)][string]$DestinationFileName, [Parameter(Mandatory=$True)][string]$SourceDirectory, [Parameter(Mandatory=$False)][string]$CompressionLevel = "Optimal", [Parameter(Mandatory=$False)][switch]$IncludeParentDir ) Add-Type -AssemblyName System.IO.Compression.FileSystem $CompressionLevel = [System.IO.Compression.CompressionLevel]::$CompressionLevel [System.IO.Compression.ZipFile]::CreateFromDirectory($SourceDirectory, $DestinationFileName, $CompressionLevel, $IncludeParentDir) }
Pipelines
Instead of just parsing text (as *Nix shells do), PowerShell works with objects in a pipeline.
Piping:
To list all variables defined and their values:
Get-Variable | Out-String
PROTIP: With PowerShell, it’s best to use out-file instead of “>” redirect character:
dir -file -hidden | out-file -filepath rootfiles.txt
Error handling:
Use preference variables for stream redirection:
1> Success 2> Error 3> Warning 4> Verbose 5> Debug
NOTE: Can Only merge to the success stream.
$Error is the automatic array that stores the last 256 exceptions (objects in error) - the default $MaximumErrorCount.
Error action preferences:
0 = Silently Continue
1 = Stop
2 = Continue
3 = Inquire
4 = Ignore [parameter value only]
Module to call Twitter REST API
This suggests:
$J = Invoke-WebRequest ` -Uri http://search.twitter.com/search.json?q=PowerShell ` | ConvertFrom-Json
PROTIP: To press the trailing back-tick that breaks up a command into several lines, press the key at the upper left corner of the keyboard with your left hand while you press shift key using your right hand.
A space character is required before the tick.
PROTIP: Break up long text into a string block (which Microsoft calls here-string):
$string = @" item1 = value1 item2 = value2 "@ $hashtable = ConvertFrom-StringData -StringData $string $hashtable
The output is:
Name Value ---- ----- item2 value2 item1 value1
From https://apps.twitter.com/ define a new app. In Permissions tab, select Read-only. Click Update Settings. In Key and Access Tokens tab, click “Create my access tokens”. Copy the Consumer Key (API key) and paste in ~/.passwords as TWITTER_TOKEN.
It takes
many lines to mess with OAuth,
so I make use of Adam’s library for Twitter’s v1.1 API described at:
http://www.adamtheautomator.com/twitter-module-powershell/
-
https://gallery.technet.microsoft.com/scriptcenter/Tweet-and-send-Twitter-DMs-8c2d6f0a
called “Tweet and send Twitter DMs with Powershell”.Adam’s “MyTwitter.psm1” I’ve download had 229 lines on 8/31/2014.
PROTIP: The “.psm1” extension means it’s a PowerShell module.
I used a text editor to edit the file to paste in variables for the 4 credentials from Twitter.
[Parameter()] [string]$ApiKey = $SECRETS.TWITTER_APIKEY, [Parameter()] [string]$ApiSecret = $SECRETS.TWITTER_APISECRET, [Parameter()] [string]$AccessToken = $SECRETS.TWITTER_ACCESSTOKEN, [Parameter()] [string]$AccessTokenSecret = $SECRETS.TWITTER_APISECRET
I then saved the module in the same GitHub folder as my script, and added a command to pull the module into the script:
Import-module "../MyTwitter.psm1"
Alternately, Copy-install the module to your $env:PSModulePath
See http://www.powershellgallery.com/gettingstarted
PowerShellGet from the Windows PowerShell Framework 5.0
The alternative is to put the module in the PSModulePath, which enables tab completion to complete the names of commands from modules that are not loaded.
The module has these functions:
- Get-OAuthAuthorization
- Send-Tweet
- Send-TwitterDm
-
Paste in your PowerShell script:
Send-Tweet -Message '@adbertram Thanks for the Powershell Twitter module'
BTW, PowerShell cmdlets in https://github.com/Iristyle/Posh-GitHub is only for use on Windows.
-
Trevor Sullivan (@pcgeek86) made a 20:40 video Mar 17, 2016
-
A PowerShell Module for manipulating PowerShell Profiles by Thomas Malkewitz
Curl
curl is an alias for Invoke-WebRequest in PowerShell.
Invoke-RestMethod ` -Method Post ` -Uri "$resource\new" ` -Body (ConvertTo-Json $body) ` -Header @{"X-ApiKey"=$apiKey}
See https://channel9.msdn.com/Blogs/trevor-powershell/Automating-the-GitHub-REST-API-Using-PowerShell
JSON from REST API
To extract out a key from the JSON file:
$x.Stuffs | where { $_.Name -eq "Darts" }
https://www.pluralsight.com/courses/powershell-modules-advanced-functions-building
API calls
Corporate IT departments often use Group Policies.
$Headers = "Authorization: token ${GITHUB_TOKEN}" echo "Headers=$Headers" # DEBUGGING $Token=$GITHUBUSER +':'+ $SECRETS.GITHUB_TOKEN; $Base64Token=[System.Convert]::ToBase64String([char[]]$Token); $Headers = @{ Authorization = 'Basic(0)' -f $Base64Token; }; # -f is for substitution of (0). # See https://technet.microsoft.com/en-us/library/ee692795.aspx # Write-Host ("Headers="+$Headers.Authorization) $Headers = "{ Authorization: = Basic $GITHUB_TOKEN }" # -f is for substitution of (0). # See https://technet.microsoft.com/en-us/library/ee692795.aspx Write-Host ("Headers="+$Headers)
Profile scripts
Jeff Hicks notes these profile scripts execute automatically at start:
-
To view all profiles:
$profile | select *
Response:
Folder | Script file | Script name |
---|---|---|
C:\Windows\ System32\ WindowsPowerShell\ v1.0\ | profile.ps1 | AllUsersAllHosts |
Microsoft.PowerShell.profile.ps1 | AllUsersCurrentHost | |
Microsoft.PowerShellSE.profile.ps1 | AllUsersCurrentHost (ISE) | |
C:\Users\<user>\Documents\ WindowsPowerShell\ or /Users/<user>/ .config/powershell/ | Microsoft.PowerShell.profile.ps1 | CurrentUsersAllHosts* |
profile.ps1 | CurrentUserCurrentHost | |
Microsoft.PowerShellSE.profile.ps1 | CurrentUserCurrentHost (ISE) |
- = This is the one shown when $profile is typed in.
Iterate
-
Stephane shows this command to move (pipe) png files from Desktop to Pictures folder:
Get-ChildItem -Filter '*.png' | Move-Item -Destination '../Pictures'
A variable can contain an array:
$files = dir c:\script -file Foreach ($file in $files){ $fileage = ((get-Date)) - $file.LastWriteTime ) "$($file.name) = $fileage" | Out-File ... }
Read in CSV file
This blog gives an example of importing a CSV file:
$data = Import-CSV C:\scripts\moviedata.csv
PROTIP: Sorting by date requires creating a new property:
$data | Add-Member -MemberType ScriptProperty ` -Name "OpensIn" ` -Value { [int32]((($this.ReleaseDate ` -as [DateTime]) - (Get-Date)).TotalDays) }
The new property persists, so can be used this way:
$data | Sort "OpensIn" | Select Title.ReleaseDate.OpensIn.Coments
PowerShell Remoting
https://docs.microsoft.com/en-us/powershell/scripting/learn/remoting/winrmsecurity?view=powershell-7.1 Security Considerations for PowerShell Remoting using WinRM
Social
More PS Libraries
https://www.simple-talk.com/blogs/psyaml-powershell-yaml
References
“Practical PowerShell for IT Security”:
- Part I: File Event Monitoring
- Part II: File Access Analytics (FAA)
- Part III: Classification on a Budget
- Part IV: Security Scripting Platform (SSP)
- Part V: Security Scripting Platform Gets a Makeover
https://docs.broadcom.com/doc/increased-use-of-powershell-in-attacks-16-en
More on DevSecOps
This is one of a series on DevSecOps:
- DevOps_2.0
- ci-cd (Continuous Integration and Continuous Delivery)
- User Stories for DevOps
- Git and GitHub vs File Archival
- Git Commands and Statuses
- Git Commit, Tag, Push
- Git Utilities
- Data Security GitHub
- GitHub API
- Choices for DevOps Technologies
- Pulumi Infrastructure as Code (IaC)
- Java DevOps Workflow
- AWS DevOps (CodeCommit, CodePipeline, CodeDeploy)
- AWS server deployment options
- Cloud services comparisons (across vendors)
- Cloud regions (across vendors)
- Azure Cloud Onramp (Subscriptions, Portal GUI, CLI)
- Azure Certifications
- Azure Cloud Powershell
- Bash Windows using Microsoft’s WSL (Windows Subsystem for Linux)
- Azure Networking
- Azure Storage
- Azure Compute
- Digital Ocean
- Packer automation to build Vagrant images
- Terraform multi-cloud provisioning automation
-
Hashicorp Vault and Consul to generate and hold secrets
- Powershell Ecosystem
- Powershell on MacOS
- Jenkins Server Setup
- Jenkins Plug-ins
- Jenkins Freestyle jobs
- Docker (Glossary, Ecosystem, Certification)
- Make Makefile for Docker
- Docker Setup and run Bash shell script
- Bash coding
- Docker Setup
- Dockerize apps
- Ansible
- Kubernetes Operators
- Threat Modeling
- API Management Microsoft
- Scenarios for load
- Chaos Engineering