Mac setup

Quickly build and update your macOS laptop with the most useful software for developers – no more dreading losing your laptop, using my shell script.

Overview

Invoke Terminal.app
Phases
Basics: Menu: Copy, Paste, Run
- Default menu of parameters
- Edit mac-setup.env parameter values
Steps in script mac-setup.zsh
First line Shebang and comments
Disable Shellcheck Linting Rules
- Program Version and change description
Time end - start = elapsed
- Indent code 3 spaces
Arguments into script
Text color codes in messages
Custom functions to echo text to screen
Feature Flags (Run Parameters)
Set variables associated with each parameter flag
- Clear screen echo
- Set “Strict Mode”
- Read config flag
Operating System Detection
Version of Bash installed
apt-get install function
Bash Traps
Display OS Info
Define utility functions: Keep Alive, pskill
Install Installers
Install utilities
ShellCheck Shell script linter
Get secrets from a clear-text file in $HOME folder
Projects folder
Obtain repository from GitHub
- Unencrypt GitSecret
Pipenv
Connect to Google Cloud
Connect to AWS Cloud
Install Azure</a>
Install K8S minikube
Install EKS using eksctl
- Report Timings : Disk space free capacity
Alternatives to automation
Dot files and Ansible
- Mac Programs Google Sheet
Homebrew
Cask
Dotfiles
- Symlinks
Edit playbook.yml
Edit System Settings
NPM
Inventory of Hosts
- Other Ansible Playbooks
Run Installation
Remove app
Apple Store programs
Ansible Tower
Others like this
Resources
Ansible on Azure
References
More on macOS
More on DevOps

This article introduces automation that installs and securely configures macOS laptops.

Files in https://github.com/wilsonmar/mac-setup
enables the quickest onboarding experience so people can get to work without waiting, mistakes, and needing support. Quickly get a new laptop just the way you like it after theft or loss.

I run again each week to reinstall software with the latest security patches.

NOTE: Content here are my personal opinions, and not intended to represent any employer (past or present). “PROTIP:” here highlight information I haven’t seen elsewhere on the internet because it is hard-won, little-know but significant facts based on my personal research and experience.

Invoke Terminal.app

Invoke the built-in Terminal program. The script will install alternative apps, if configured.

Hold down the Command key and press spacebar to pop up the Spotlight Search form.
Type on top of “Spotlight Search” enough of Terminal for that to appear in the drop-down.
When “Terminal.app” is highlighted, click it with your mouse or press the return key to launch the Terminal program.

NOTE: We will see later in this article how to customize the prompt shown.
Type pwd to see the current folder you are in. The path shown is also stored in a variable named $HOME.
Type ls -al to see the files in the current folder. The -al parameters are passed to the ls command to show all files in a list.

NOTE: Later in this article we will see how to use keyboard aliases to reduce typing.

Phases

Automation at https://github.com/wilsonmar/mac-setup is used in phases:

Navigate to a temporary working folder and
run the mac-setup.zsh shell script without parameters to display a menu of parameters. Follow the specific steps below copy and run this:
```
zsh -c "$(curl -fsSL https://raw.githubusercontent.com/wilsonmar/mac-setup/master/mac-setup.zsh)"
```
Navigate to your $HOME folder to
run the mac-setup.zsh shell script with parameters to download the mac-setup.env file to that folder:
```
zsh -c "$(curl -fsSL https://raw.githubusercontent.com/wilsonmar/mac-setup/master/mac-setup.zsh)" -envd
```
Edit the mac-setup.env file with values you want to keep secret and to customize software you want installed.
Install software (such as XCode Command Line Utilities, Homebrew, Docker, Python, etc.) based on your specifications by -I
```
zsh -c "$(curl -fsSL https://raw.githubusercontent.com/wilsonmar/mac-setup/master/mac-setup.zsh)" -I -v
```

Run the mac-setup.zsh with various command parameters to run Python and other programs.

zsh -c "$(curl -fsSL https://raw.githubusercontent.com/wilsonmar/mac-setup/master/mac-setup.zsh)" -v

See a list of the different -usage commands parameters:

zsh -c "$(curl -fsSL https://raw.githubusercontent.com/wilsonmar/mac-setup/master/mac-setup.zsh)" -u

Configure what you want installed.
- Edit browser add-ons
Update (reininstall) all utilities installed:
```
zsh -c "$(curl -fsSL https://raw.githubusercontent.com/wilsonmar/mac-setup/master/mac-setup.zsh)" -I -U -v
```
Here is where

Open each of the apps to ensure that they still work.

Press command+Tab to switch among the Terminal.app and the browser window.
If you don’t have a Terminal open, press command+spacebar, then type enough of Terminal.app to select it.

You can be in any folder for this because the script creates and navigates to a Projects folder in your $HOME folder, like many programs do.

NOTE: Some prefer using iTerm2 or Hyper instead of Apple’s built-in Terminal.app, which this script can also install.
Expand the width of your Terminal window by dragging the right edge so you see any more line wrapping.
Highlight the line here below: use your mouse to triple-click zsh at the front of the command, then press command+C to copy it to your Clipboard:
```
zsh -c "$(curl -fsSL https://raw.githubusercontent.com/wilsonmar/mac-setup/master/mac-setup.zsh)" 
```
CAUTION: Do not click on the https:// link since the terminal program will then attempt to open a browser to that URL.
On the Terminal, press Enter Return/Enter on your keyboard to begin execution.

When no parameter is specified, the script returns a menu of parameters options.
```
zsh -c "$(curl -fsSL https://raw.githubusercontent.com/wilsonmar/mac-setup/master/mac-setup.zsh)" -v
```
Default menu of parameters

-v is a parameter that specifies optional verbose log output.

Additional parameters are covered later in this article below.

This modular approach is designed to enable quick experimentation.
Paste the command again and add more parameters to additional command executions. See “Customize mac-setup specifications to your liking”

-I specifies install of base utilities and app most popular among developers using macOS.

-go specifies install of Go language.

etc.

Again press Return to start the script running using the new parameters.

If each utility program in the list is not installed, the script installs it.

We created the script to automatically take care of workarounds to issues.

Edit mac-setup.env parameter values

The behavior described below can be stopped by specifying the
-nenv run-time parameter.
Here’s that <a exception noted earlier: WARNING: The mac-setup.zsh script, by default, downloads (using curl) a mac-setup.env file in your user $HOME to hold configuration settings. That’s needed so that, in future runs, the script makes use of values in variables within that file rather than default values hard-coded in the mac-setup.zsh script file. View it online:

https://github.com/wilsonmar/mac-setup/blob/main/mck-setup.env

NOTE: Secrets are kept in a different file.

Once downloaded, the script does not replace the file with another download.

The script loads this file. However, at run-time, an alternative name can be specified dynamically (for a particular run), such as:
-env “~/alt-mac-setup.env”
After your favorite text editor is installed, edit that mac-setup.env file and change some values:

We expect that you change each “wilson” sample value, since that is never applicable to you.

The script uses a folder based on what is specified in the PROJECTS_CONTAINER_PATH variable.

~/Projects/mac-setup folder on your machine and downloads the mac-setup.zsh script file into it.
If you have not created an account on GitHub.com, please do so before continuing. Most actions by developers make use of the GitHub versioned change management utility on the web. It is reached by the git utility the script installs.
If you (and your team) want a different set of configurations (utilities and apps to install), save the script.
Alternately, edit the configuration files to run Ansible, a whole other approach.

Steps in script mac-setup.zsh

Click each section for an explanation of the coding:

The first lines:

The first “shebang” line.
Define bash ShellCheck rules that need to be disabled.
SCRIPT_VERSION and last change description text.

01. Capture time stamps to later calculate how long the script runs, no matter how it ends.

02. Display a menu if no parameter is specified in the command line

03. Custom functions to echo text to screen. Each echo function displays in different colors. -vv causes sample output to be displayed.

04. Download config settings file to $HOME/mac-setup.env (away from GitHub)

05. Define variables for use as “feature flags”

06. Set variables dynamically based on each parameter flag

07. Display run variables

08. Obtain and show information about the operating system in use to define which package manager to use

09. Set traps to display information if script is interrupted.

10. Set Continue on Error and Trace

11. Print run Operating environment information

12. Keep-alive: update existing sudo time stamp until .osx has finished

13. Install installers (brew, apt-get), depending on operating system

14. Install ShellCheck

15. Install basic utilities (git, jq, yq, tree, etc.) used by many

yq takes a YAML file as input and can: https://learnk8s.io/templating-yaml-with-code See https://mikefarah.gitbook.io/yq/

read values from the file
add new values
updated existing values
generate new YAML files
covert YAML into JSON
merge two or more YAML files

16. Override defaults in Apple macOS System Preferences:”

17. Image SD card

Reserved.

21. Configure project folder location where files are created by the run

21. Obtain repository from GitHub

21. Reveal secrets stored within .gitsecret folder within repo from GitHub

22. Pipenv and Pyenv to install Python and its modules

23. Connect to Google Cloud, if requested

24. Connect to AWS

25. Install Azure

26. Install K8S minikube

27. Install EKS using eksctl

Open

29. Use CircleCI SaaS

30. Use Yubikey

31. Use GitHub

32. Use HashiCorp Vault

33. Use HashiCorp Vault

34. Put secret in HashiCorp Vault

35. Install NodeJs

36. Install Virtualenv

37. Configure Pyenv with virtualenv

43. Finish RUN_VIRTUALENV

48. RUN_ACTUAL within Docker

49. UPDATE_GITHUB

50. REMOVE_GITHUB_AFTER folder after run

51. KEEP_PROCESSES after run

52. Delete Docker containers in memory after run …

53. REMOVE_DOCKER_IMAGES downloaded

54. Report Timings

Now a technical dive into each topic:

First line Shebang and comments

#!/usr/bin/env zsh

# is a comment in Bash scripts.

#! is called the “Shebang”.

/usr/bin is the folder that holds the executable program env.

/env is the name of the program that obtains the appropriate shell based on the next nugget. It select the appropriate version.

zsh and bash specify the interpreter. (python is used in Python scripts.)

Unlike the Windows operating system, which decides what program is used to open a file based on the file name “extension” behind the dot, Linux systems ignores the file name and looks into the file to see the first line.

Historically, up to MacOS High Sierra, the “Bourne-compliant” shebang for the Bash v3.2 shell installed in folder /bin by default on . Thus:

#!/bin/bash

/ means the folder is from the root level, above where the operating system stores home files for specific users.

However, Bash v4 is installed (for parallel operation) in another folder:

#!/usr/local/bin/bash

This blog describes what is improved by version 4, such as “associative arrays”.

Disable Shellcheck Linting Rules

This comment line disables (excludes) ShellCheck linter check SC2034 in the file:
```
shellcheck disable=SC2001 # See if you can use ${variable//search/replace} instead.
```
After ShellCheck version 0.4.6, the line can be added anywhere for the next line in the script.

Alternately, in the script define the code associated with each rule to -exclude from checking:
```
export SHELLCHECK_OPTS="-e SC2001 -e SC2059 -e SC2034 -e SC1090"
```
Also, the entire script can be copied and pasted online for checking at shellcheck.net, but that can be a security violation. So we install it for local running:

Install ShellCheck from https://github.com/koalaman/shellcheck
```
bash install shellcheck
```
This script runs ShellCheck to lint itself.
```
shellcheck sample.sh
```
No response text is issued if no errors were found.

PROTIP: If ShellCheck finds an issue, the script stops.

Program Version and change description

SCRIPT_VERSION=”v0.82” # Renumber steps

PROTIP: Each time I edit the file, I increment the SCRIPT_VERSION and type a short description of what I intend to change. That text is often used in the git commit message.

Time end - start = elapsed

To determine elapsed time, START time stamps are captured as soon as the script starts.

When the script ends, END time stamps are captured to calculate the elapsed time.

There are two time stamp formats.
```
EPOCH_START="$(date -u +%s)"  # such as 1572634619
```
captures the number of minutes since the Jan 1, 1970 epoch point in time.
```
LOG_DATETIME=$(date +%Y-%m-%dT%H:%M:%S%z)-$((1 + RANDOM % 1000))
```
captures the date in a human-readable year-month-day-hour-minutes “ISO 8601” format which also includes the hours and minutes from UTC/GMT, such as “-600” (for US Central Time) in this sample output:

sample.sh.2018-04-22T19:26:20-0600-18

An additional RANDOM number is added to ensure uniqueness among several instances running.

PROTIP: Values stored in variables during a run do not persist.

The number of seconds is rounded DOWN, so a run that takes less than a second is measured as 0 seconds.

Indent code 3 spaces

It’s an asthetic choice.

Google’s Style Guide calls for two spaces.

But three spaces make the line indent under if align better. And the if statement is the most common in the script.

Arguments into script

The args_prompt() function defines text that is echoed to the console if the script is invoked with no arguments, such as:
```
./mac-setup.zsh -h -v -I -U -c -s -r -a -o
```
Checking for whether parameters were added is done by this code:
```
if [ $# -eq 0 ]; then  # display if no parameters are provided:
 args_prompt
fi
```
A sample response was shown above.

The USAGE example shows the various parameters that need to be added to specific actions taken by the script.

This design ensures the flexibility of the script.

Flags not associated with a text string specification (such as Verbose) default to false and get switched to true when specified.

Text variables are defined first, then exported in a separate step as recommended by Shellcheck.

Text color codes in messages

The Unix operating system (on which today’s Linux distributions are based) “streams” text to the Console. Colors (colours) and other effects are specified by inserting “toggles” (attributes) that change the appearing of text following it. A reset sets all text to display in the default appearance.

The color and other text attributes described above are specified within functions called to display message text to the console.

On macOS the approach is to define variables containing ANSI escape numbers:

blink="\e[5m"
blue="\e[34m"
bold="\e[1m"
dim="\e[2m"
green="\e[32m"
red="\e[31m"
reset="\e[0m"
underline="\e[4m"

Different Linux distributions and platforms recognize different toggle codes. So use the alternate approach using the tput utility which works on all *nix systems to display attribute variables.

# Set less cryptic color attributes names using tput common to all Linux distributions: 
   blink=$(tput blink)         # 5 as in ANSI 5 in "\e[5m"
   bold=$(tput bold)           # 1
   dim=$(tput dim)             # 2 (faint)
   underline=$(tput smul)      # 4
   end_underline=$(tput rmul)
   reverse=$(tput rev)         # 7
# Foreground colors:
   red=$(tput setaf 1)         # 31
   green=$(tput setaf 2)       # 32
   yellow=$(tput setaf 3)      # 33
   blue=$(tput setaf 4)        # 34
   purple=$(tput setaf 5)      # 35
   cyan=$(tput setaf 6)        # 36
   white=$(tput setaf 7)       # 37
   reset=$(tput setaf 0)       # 39 default
# Background colors:
   b_red=$(tput setb 1)        # 41
   b_green=$(tput setb 2)      # 42
   b_yellow=$(tput setb 3)     # 43
   b_blue=$(tput setb 4)       # 44
   b_purple=$(tput setb 5)     # 45
   b_cyan=$(tput setb 6)       # 46
   b_white=$(tput setb 7)      # 47
   b_reset=$(tput setb 0)      # 49 default
# Reset all to defaults:
   reset=$(tput sgr0)

BTW To test how the codes, put this in a script:

echo "${green}Success! ${dim}dimmed${reset} "
echo "${red}Failure ${bold}bolded${reset}"
echo "${blink}${f_yellow}Caution ${bold}bolded${reset} bad"
echo "${blue}Note${reset} blue on black is annoying"
echo "${underline}${purple}Alert${reset} magenta underlined"
echo "${reverse}${cyan}Info${reset} cyan reversed"
echo "${white}Whatever white${reset} this is"

Custom functions to echo text to screen

To format output, this code is used:

h2() {     # heading
   printf "\n${bold}>>> %s${reset}\n" "$(echo "$@" | sed '/./,$!d')"
}
info() {   # output on every run
   printf "${dim}\n➜ %s${reset}\n" "$(echo "$@" | sed '/./,$!d')"
}
note() { if [ "${RUN_VERBOSE}" = true ]; then
   printf "${bold}${cyan} ${reset} ${cyan}%s${reset}\n" "$(echo "$@" | sed '/./,$!d')"
   fi
}
success() {
   printf "${green}✔ %s${reset}\n" "$(echo "$@" | sed '/./,$!d')"
}
error() {       # ☓
   printf "${red}${bold}✖ %s${reset}\n" "$(echo "$@" | sed '/./,$!d')"
}
warnNotice() {  # ☛
   printf "${cyan}☛ %s${reset}\n" "$(echo "$@" | sed '/./,$!d')"
}
warnError() {   # Skull: ☠  # Star: ★ ★ U+02606  # Toxic: ☢
   printf "${red}☢ %s${reset}\n" "$(echo "$@" | sed '/./,$!d')"
}

“h2” is a homage to HTML heading names. The other functions correspond to the different levels of verbosity used by the log4j library (in the npm aws-code-deploy repo).

The printf command is used instead of echo for compatibility with all versions of Bash.

PROTIP: Notice there are HTML/CSS icons within text, so the file must be stored in UTF-8 format.

-vv sets debugging on to print how the codes look:

h2 "Header here"
info "info"
note "note"
success "success!"
error "error"
warning "warning (warnNotice)"
fatal "fatal (warnError)"

Feature Flags (Run Parameters)

Change what each run of the script does by changing parameters invoking the command, such as -v -I -U -c -s -r -a -o
```
bash -c "$(curl -fsSL https://raw.githubusercontent.com/wilsonmar/DevSecOps/master/bash/sample.sh)" -v -I -U -c -s -r -a -o
```
The script ends with a message like this:
```
✔ End of script after 1883 seconds and 677960 bytes of disk space.
```
-v for -verbosity adds additional notes.

-vv for debugging verbosity such as a display example log types.

-q for -quiet suppression of headers and footers that appear by default, such as when running in production mode.

-t for -testing mode, which runs local Vault and app servers.

-I runs installers, but installs each only if it is not already installed.

-I and -U updates installers even though each is installed. Some installers are invoked only if the feature is also specified. But Homebrew and git are updated if no other utilities are specified.

-o -opens the sample app in your default browser.

The rest of this article describes coding tricks used and how you might customize the script.

Set variables associated with each parameter flag

There are several ways for shell script to process parameters. The one shown seems the easiest for me to read.

Parameters are processed in the order coded. Multi-character options are coded first.

Clear screen echo

To show responses at the top of the terminal, delete the comment # to enable clear # screen (but not history).

However, a lot of output would scroll past, so it is rather useless. Better to print a long string as a visual marker to differentiate between different runs:
```
echo "========================= $SCRIPT_VERSION"
```
The SCRIPT_VERSION is shown at the beginning and the end to detect whether a cached version of the script was used. That happens.

Set “Strict Mode”

At the beginning of the script file is:

set -e  # exits script when a command fails
# set -eu pipefail  # pipefail counts as a parameter

Others are there for convenience, to copy and paste to a specific point in the script where commands need to be visible for debugging:

# set -x to show commands for specific issues.
# set -o nounset

Some put them all in one line:

set -o nounset -o pipefail -o errexit  # "strict mode"

pipefail means that when the program encounters an exit code != 0, the exit code for the pipeline (Bash script) becomes != 0. E.g. pipefail can be useful to ensure curl does-not-exist-aaaaaaa.com | wc -c doesn’t exit with exit code 0..!>

Some toggle tracing on and off by defining export DEBUG=TRUE and add in the code:

DEBUG="TRUE"
...
if [[ "${DEBUG:-FALSE}" != "FALSE" ]]; then
  set -o xtrace
fi

Read config flag

settings file to $HOME/mac-setup.env (away from GitHub)

Operating System Detection

We code shell scripts to operate in macOS and various distributions of Linux so that developers can focus on processing sequence which are similar on all platforms.

uname is supposed to be available on all versions of Linux and macOS.

Darwin is the internal name of the current macOS operating system. It is based on the NeXTSTEP operating system Steve Jobs brought into Apple upon his return to Apple in 1998. [Wikipedia explains its roots in BSD]

brew is the command used by the Homebrew package manager used by macOS.

Different Linux distributions use different file names to store its version information. And different Linux distributions have their own package manager. Thus we need to obtain the PACKAGE_MANAGER used by the script.

# Check what operating system is in use:
   OS_TYPE="$( uname )"
   OS_DETAILS=""  # default blank.
if [ "$(uname)" == "Darwin" ]; then  # it's on a Mac:
      OS_TYPE="macOS"
      PACKAGE_MANAGER="brew"
elif [ "$(uname)" == "Linux" ]; then  # it's on a Mac:
   if command -v lsb_release ; then
      lsb_release -a
      OS_TYPE="Ubuntu"
      PACKAGE_MANAGER="apt-get"

      silent-apt-get-install(){  # "$1" refers to parameter of package to install:
         sudo DEBIAN_FRONTEND=noninteractive apt-get install -qq "$1" < /dev/null > /dev/null
      }
   elif [ -f "/etc/os-release" ]; then
      OS_DETAILS=$( cat "/etc/os-release" )  # ID_LIKE="rhel fedora"
      OS_TYPE="Fedora"
      PACKAGE_MANAGER="yum"
   elif [ -f "/etc/redhat-release" ]; then
      OS_DETAILS=$( cat "/etc/redhat-release" )
      OS_TYPE="RedHat"
      PACKAGE_MANAGER="yum"
   elif [ -f "/etc/centos-release" ]; then
      OS_TYPE="CentOS"
      PACKAGE_MANAGER="yum"
   else
      error "Linux distribution not anticipated. Please update script. Aborting."
      exit 0
   fi
else 
   error "Operating system not anticipated. Please update script. Aborting."
   exit 0
fi

Version of Bash installed

Some commands make use of a more recent version of Bash than the operating system may have by default. So the script updates the bash processor if the -U flag is specified. Follow along manually:

Be at a macOS Terminal.
Test what version of Bash is installed on your Mac by typing this:
```
bash --version
```
If you see the below, you are using Bash version 3.x, which macOS first released in 2007.
```
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin16)
```
In macOS Mojave version, Apple still ships that old thing due to licensing issues.

Install the latest version of the Bash shell, using Homebrew:

brew install bash

Bash 4.0 was released in 2009.

As of this writing, the response is:

GNU bash, version 5.0.11(1)-release (x86_64-apple-darwin18.6.0)
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
 
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

If you want to see just the version line, pipe the response to the grep utility built into macOS:
```
bash --version | grep 'bash'
```
Hold down the left Shift key to press the | (called pipe) key at the upper-right of the keyboard.

grep ‘bash’ filters out lines that do not contain the word “bash” in the response.

apt-get install function

apt-get install commands in this script use a custom function which feeds in the package name to be installed:

silent-apt-get-install "git"

The function is defined where the operating system and package manager is recognized:

silent-apt-get-install(){  # "$1" refers to parameter of package to install:
sudo DEBIAN_FRONTEND=noninteractive apt-get install -qq git htop < /dev/null > /dev/null

DEBIAN_FRONTEND=noninteractive gets rid of “(Reading database … 5%” output.

-qq is there to not request manual confirmation messages such as:

Need to get 260 MB of archives.
After this operation, 308 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y

The -qq parameter combines the effect of the -y (yes) and -q (quiet) parameter, plus more suppression.

The output that remains is from dpkg which was forked by apt-get. So > /dev/null pipes the standard output (stdout) to nothing so you don’t see them. However, you’ll still see error messages, which go out thru stderr.

< /dev/null pipes stdin standard output to nothing. Explained here.

Bash Traps

The Bash trap command catches signals so it can execute some commands when appropriate, such as cleaning up temp files before the script finishes, called an exit trap.

cleanup() {
    err=$?
    echo "Cleaning stuff up..."
    trap '' EXIT INT TERM
    exit $err 
}
sig_cleanup() {
    trap '' EXIT # some shells will call EXIT after the INT handler
    false # sets $?
    cleanup
}

The above cleanup function is invoked when INT TERM occurs to trigger the function, at the bottom of the script:

trap cleanup EXIT
trap sig_cleanup INT QUIT TERM

This statement in the script…

trap 'ret=$?; test $ret -ne 0 && printf "failed\n\n" >&2; exit $ret' EXIT

Display OS Info

Define utility functions: Keep Alive, pskill

pskill is not a native function.

Install Installers

For online pair programming where several people can change a file at once, this article notes that there is a plug-in for each of the popular text editors:

About VSCode, have vscode.dev on the browser read your local files.
Sublime Text
etc.

Install utilities

ShellCheck Shell script linter

Get secrets from a clear-text file in $HOME folder

Projects folder

There are two places to download files from GitHub:

Into the GITHUB_ACCOUNT_PATH specified by -gap or default “$HOME/gh-wm” folder referenced in ~/.ssh.
Used when local changes will be uploaded to GitHub.
Into the PROJECTS_CONTAINER_PATH specified by -pcp or default $HOME/Projects”.
Used when changes will NOT be uploaded to GitHub. This is the less common approach.

The script downloads into one the other if there is a specification of a folder:

Into the GITHUB_REPO_FOLDER specified by -grf or default GitHub repo name.
Into the PROJECT_FOLDER_NAME specified by -pn or default GitHub repo name.

This provides a place for files to be created away from the GitHub folder. This prevents inadvertant upload to public GitHub repo.

Obtain repository from GitHub

First, make sure variables needed by the rest of the script are valid:

There is no sense going forward if the GitHub location is not defined or exists, so the script can fatally exit here.

The GITHUB_REPO_URL used to download can be of two formats:

“https://github.com/wilsonmar/xxx.git” for public repos
“git@github.com:wilsonmar/mac-setup.git” if parameter -ssh was specified to use private repos

If GITHUB_REPO_URL is not specified explicitly using parameter -ghr, it is constructed by combining:
“https://github.com/” or “git@github.com:” (depending on -ssh) +
GITHUB_ACCOUNT name (such as “wilsonmar”) specified using parameter -gan +
GITHUB_REPO_FOLDER (name) specified using parameter -grf.

If a GitHub repo already exists, it won’t be overridden from GitHub unless the -c CLONE_GITHUB flag is specified to clone.

There is no sense going forward if the GitHub repo does not exist, so the script can fatally exit here.

# curl -s -O https://raw.GitHubusercontent.com/wilsonmar/build-a-saas-app-with-flask/master/mac-setup.zsh # git remote add upstream https://github.com/nickjj/build-a-saas-app-with-flask # git pull upstream master

Unencrypt GitSecret

This involves storing secrets in GitHub within files encrypted with long-running secrets. This is not very secure because bad actors would have time to break the encryption.

Pipenv

Connect to Google Cloud

Connect to AWS Cloud

Install Azure</a>

Install K8S minikube

Install EKS using eksctl

Report Timings : Disk space free capacity

We want to know how much disk space is available at the beginning of the run, and the amount of space taken during the run.

On macOS and other BSD operating systems, the “disk free” command df -P / outputs a “standardized” number of 512 byte blocks in the “/” mount:

Filesystem   512-blocks      Used  Available Capacity  Mounted on
/dev/disk1s1 1953595632 521869880 1417651624    27%    /

We use this methodology to obtain the percentage of disk free, which obtains the 12th text item -delimited by a space (which include heading items):

DISK_PCT_FREE=$(read -d '' -ra df_arr < <(LC_ALL=C df -P /); echo "${df_arr[11]}" )

The blocks Available is the 10th text item.

FREE_DISKBLOCKS_START=$(read -d '' -ra df_arr < <(LC_ALL=C df -P /); echo "${df_arr[10]}" )

This uses bash arrays which became available since Bash version 4.

NOTE: We don’t use “-m” for megabytes or “-k” for kilobytes which result in mesuring small amounts of space used as zero.

This captures the starting count:

FREE_DISKBLOCKS_START="$( df . | cut -d' ' -f 6 )"   # e.g. 254781 MB Used

TODO: Within cloud environments such as Amazon AWS EC2 or Azure, this may still be relevant.

df is the disk free command used to obtain the number of blocks Used and Available for each storage device mounted.

. specifies calculation of the number of 512 byte blocks in the current device:

Filesystem   512-blocks      Used  Available Capacity iused               ifree %iused  Mounted on
/dev/disk1s1 1953595632 521825264 1417696240    27% 1480293 9223372036853295514    0%   /

| cut -d' ' -f 6 pipes to cut using a space to -demarkate the 6th column. The response is an integer such as “254781”. Divided by 1024 means 248 Gigabytes.

At the end of the script, another variable is obtained when the END variable is obtained for use in calculating the time and disk space used during the script run.

Alternatives to automation

If you’ve already run this, click here to skip to view and edit the install specifications further down this tutorial.

To automate preparation of third-party software for mass deployment onto managed clients:

Autopkg at https://github.com/autopkg/autopkg (requies Git) aims to define the steps in a “Recipe” which can be defined in an understood (plist XML-based) format, run automatically instead of by hand, and shared with others.
Software distribution server software: Munki (pronounced like “monkey”), Casper, Absolute Manage, etc.

There are two phases to this:

Phase one - supply your password to install Xcode, Homebrew, Tap, and Cask, and Ansible
Phase two - supply your password again for Ansible to install what is not commented out in specification files:

Dot files and Ansible

I haven’t seen it elsewhere on the internet, so I created an explanation of MacOS customizations and “dot files” that set them automatically.

Copy from below this shell script call (highlight the line, then hold down command and press C to the invisible Clipboard):
```
sh -c "$(curl -fsSL https://raw.githubusercontent.com/wilsonmar/ansible-macos-setup/master/install.sh)"
```
Make a folder and navigate into it:
```
cd ~
mkdir gits
cd gits
mkdir siyelo
cd siyelo
```
You can create another folder name if you prefer, but folder names in this tutorial will be wrong.
In the Terminal window, paste the call from clipboard (hold down command and press V).
Press the enter/return key to invoke the command which causes files to be downloaded and folders created:
- Xcode
- Homebrew, Tap, and Cask
- Ansible
The command also downloads folder laptop from https://github.com/siyelo/laptop

NOTE: Download and installation is skipped if it’s already installed.
When SUDO password: appears, hold down the control key and press C. This message should appear (in red):

“[ERROR]: User interrupted execution failed”
While still in the Terminal window from above, change to the directory just created:

cd laptop pwd

You should now be at:
```
/Users/mac/gits/siyelo/laptop
```
Instead of “mac” above, it would show your account name.

The siyelo playbook.yml file lists the applications to be loaded.

The scripts/system_settings.sh file are commands to configure the Mac operating system for better security and productivity. It would take hours to set them all manually.

Mac Programs Google Sheet

Switch to use an internet browser to see my list of programs for macOS at this Google spreadsheet online.

The _installer column identifies the technique to install to install each program named:

brew (formulas) are installed using the brew command. Programs installed this way are invoked using a text-based Terminal command-line interface.
tap uses the brew tap command to install a package within another repository. interesting brew tap.
cask packages are installed using the brew cask command which is an extension to Homebrew that allows management of applications with a graphical user interface (GUI).
pip are Python plug-ins.
npm are Node modules (such as Express).
store indicates manual installation using the Apple iTunes program accessing the Apple Store.
zip programs are installed after a download, requiring an Ansible task to replace manual clicks and data entry.

WARNING: Each program added uses up more disk space.

WARNING: This may make obsolete documentation based on default Mac settings.

Ansible configurations are “idempotent”, which means that if you run it again, the result is the same. With declarative specifications, items specified for install are not re-installed.

The other columns in the spreadsheet:

_want contains Y for the generator to include in Ansible file.

Homebrew

Cask

Dotfiles

GitHub’s Dotfiles manage files containing preferences for command-line programs.

For example, the configuration file for Zsh is .zshrc. The SSH configuration folder is .ssh. And on.

Such files have a dot because they are hidden.

Famous collections of dotfiles from:

https://github.com/holman/dotfiles from Zach Holman
Matthias Bynens
Paul Irish
https://github.com/mattstauffer/ohmyzsh-dotfiles

Features added by dotfiles:

Color grep output
Boost ls commands
Adding plugins to vim

Symlinks

Symlinks to files allow you to git pull and watch your dotfiles instantly update.

One stores .ssh-config file in his Dropbox folder which he symlinks:

   touch ~/Dropbox/.ssh-config
   ln -s ~/Dropbox/.ssh-config ~/.ssh/config

The .ssh-config file contains:

# My Awesome Web Site
Host awesome
    Hostname 141.141.141.141
    User me_duh
    IdentityFile=/Users/me/.ssh/id_for_awesome_site.rsa

The above enables him to invoke:


   ssh awesome

No more remembering ip addresses, fumbling with command line switches for multiple SSH IDs, or even remembering your ssh usernames.

https://mattstauffer.co/blog/setting-up-a-new-os-x-development-machine-part-3-dotfiles-rc-files-and-ssh-config

Edit playbook.yml

Use the vi text editor program to edit the file:

vi ~/gits/siyelo/laptop/playbook.yml
- To quit without editing type :q
- To save changes type :wq (colon, w, and q)
- To enter insert mode, press I.
- To escape insert mode, press the Esc key.
Have a # (pound sign) in front of each package based on your evaluation.

Click on each link to open a web page about each. Some links take you to a section lower in this tutorial.

NOTE: The playbook.yml file contains code to run the system_settings.sh file.

Edit System Settings

List the files and change directory into the scripts folder:

ls -al cd scripts

The above list the files and change directory into the scripts folder.
Use the vi text editor program to edit the file:

vi ~/gits/siyelo/laptop/scripts/system_settings.sh
- To quit without editing type :q
- To save changes type :wq (colon, w, and q)

NPM

Search for packages at this site where resources are downloaded for install:
- https://www.npmjs.com/package/generator-webapp

Example:

npm install -g yo

The -g globally installs for access from any folder.

Inventory of Hosts

The inventory of hosts can be defined within /etc/ansible/hosts containing .ini format Microsoft uses:

mail.example.com

[webservers]
foo.example.com
bar.example.com:5309

[dbservers]
one.example.com
two.example.com
three.example.com

Items in square brackets define group names.

Alternately, a YAML format:

Other Ansible Playbooks

Consider other Ansible playbooks for Mac:

github.com/MWGriffin/ansible-playbooks is one of the first to use Ansible to install Macs.

The last change was made in 2013. It provides playbooks for installation of php and others from zip file that may now be available as Homebrew packages.

It copies from URLS such as https://copy.com/8WKZ3x1ZWOIC/VLC-2.0.6.dmg?download=1 into the /.ansible-downloads/ folder it creates.
https://bitbucket.org/samfisher/mac-ansible-provisioning/src/
Mr. Jeff Geeling I call “Master of the Ansible Galaxy” because he has contributed both the most downloaded roles and garnered the highest ratings. His mac-dev-playbook contains:

|-- LICENSE
|-- README.md
|-- files
|   |-- etc
|   |   `-- sudoers
|   |-- sublime
|   |   `-- Library
|   |       `-- Packages
|   |           `-- User
|   |               |-- DashDoc.sublime-settings
|   |               |-- Package\ Control.sublime-settings
|   |               `-- Preferences.sublime-settings
|   `-- terminal
|       `-- JJG-Term.terminal
|-- inventory
|-- main.yml
|-- requirements.txt
|-- tasks
|   |-- ansible-setup.yml
|   `-- preferences.yml
`-- vars
    `-- main.yml

The requirements.txt file specifies files to download from the Ansible Gaxlaxy sharing site.

geerlingguy.dotfiles sets up preferences and configurations for all the built-in services and apps.

It references http://secrets.blacktree.com/?showapp=com.apple.finder

Use a text editor to edit the main.yml file at the root. Change user: value from “jgeeling” to your Mac account name.

---
- hosts: localhost
  user: jgeerling
  connection: local

  vars_files:
    - vars/main.yml

  roles:
    - geerlingguy.homebrew
    - geerlingguy.dotfiles

  tasks:
    - include: tasks/ansible-setup.yml
    - include: tasks/preferences.yml

    # TODO: Use sudo once .osx can be run via root with no user interaction.
    - name: Run .osx dotfiles.
      shell: ~/.osx --no-restart
      changed_when: false

Under the vars folder main.yml file is a list of several Homebrew packages.

Run Installation

Open a Termial, go to the directory:

cd ~/gits/siyelo/laptop/
Run Ansible:

ansible-playbook playbook.yml -i hosts –ask-sudo-pass -vvvv
Press the return key to invoke the command.
A restart is needed at the end.

Remove app

QUESTION: How to remove apps

Apple Store programs

Programs are installed into the Applications folder. Two different ones:

cd /Applications
cd ~/Applications to see programs installed by clicking browser links, such as GoToMeeting, etc.

To get rid of the GarageBand music program, which comes with Apple MacOS:

In a Terminal, go to where Apple stores its default apps at:
```
cd "/Library/Application Support/GarageBand/"
```
The quotes are needed because there is a space in the folder name.
See how much disk space it takes.
```
ls -al
```
The number of bytes is to the left of the date:
```
-rw-rw-r--   1 root  admin  31211520 Oct 25  2013 Themes.db
```
PROTIP: Delete the contents, but leave the folder name.
```
rm -rf *.* /s
```

Ansible Tower

Ansible Tower is a hub for automation tasks. It is a commercial product supported by Red Hat, Inc. but open sourced as AWX in September 2017 at https://github.com/ansible/awx.

An open source alternative to Tower is Semaphore, written in Go.

Since Ansible was acquired by Red Hat, Inc. in October 2015, the company also controls certification and provides three training for it.

AUTOMATION WITH ANSIBLE I (DO407) for $2,890 over 90 days or 4 days. was retired on DEC 31st 2020.
AUTOMATION WITH ANSIBLE II: RED HAT ANSIBLE TOWER (DO409) for $1,445 over 90 days
RED HAT CERTIFICATE OF EXPERTISE IN ANSIBLE AUTOMATION (EX407) was retired on DEC 31st 2020.
Red Hat Certified Specialist in Advanced Automation: Ansible Best Practices was retired.

Others like this

The script described here was based on reviews of other similar “Mac booststrap” scripts:

https://gist.github.com/zeekay/7394565
https://github.com/fs/osx-bootstrap is a setup for Rails developement, including http://fs.github.io/fs-tool/ for creating pull-requests from command line and Ruby in Heroku.

ImageMagick
https://github.com/divio/osx-bootstrap also asks for RSA keys to GitHub
https://github.com/monfresh/laptop provides a set of shell scripts.

Resources

https://en.wikipedia.org/wiki/Ansible_(software)

Red Hat’s YouTube channel has all products together.

https://code.tutsplus.com/tutorials/setting-up-a-mac-dev-machine-from-zero-to-hero-with-dotfiles–net-35449

To uninstall, select it and press Command+Delete or drag the program into the Trash.

http://osxdaily.com/2014/01/02/delete-garageband-imovie-iphoto-mac-os-x/

http://osxdaily.com/2014/01/01/4-free-os-x-utilities-mac/

https://www.ansible.com/videos-ansiblefest-sf-2016

5 minutes Kubernetes Cluster on Mac

Ansible on Azure

READ: Ansible & Azure: Automating the Basic Building Blocks of the Azure Cloud

References

https://courses.cs.washington.edu/courses/cse154/21sp/resources/assets/vscode-git-tutorial/macosx/index.html

VIDEO: comparison of Dotfiles managers at https://www.chezmoi.io by Tom Payne, identifies Go-based MIT-licensed brew install chezmoi as the only one with Password manager integration!

More on macOS

This is one of a series on macOS:

More on DevOps

This is one of a series on DevOps:

Wilson Mar