Pass Microsoft’s comprehensive AZ-400 on DevOps for automated CI/CD pipelines in the Azure cloud as well as GitHub Actions
Overview
- Your settings (Naming Conventions)
- Azure DevOps Product components
- Related Products and Services
- Rebranding History
- Competition
- AZ-400 certification
- Create Azure account
- CLI/PowerShell Automation
- GUI QuickStart: dev.azure.com was aex.dev.azure.com
- Workflows (Reference Architectures)
- You must install Visual Studio thick client
- Azure Service Endpoints
- Extensions to Azure DevOps Services
- Instrumentation (Logging) Strategy
- SRE Strategy
- Security Plan
- Source Controls
- Facilitating Communication and Collaboration
- Define and Implement Continuous Integration
- Define and implement a continuous delivery and release management strategy
- API to DevOps
- Azure Boards
- Azure Repos
- Azure Artifacts
- Azure Test Plans
- Azure Pipelines
- Azure DevTest Labs
- Resource Groups
- Azure Templates
- Ansible
- Terraform
- GitHub repositories
- Microsoft Rock Stars
- AZ-400 Practice exams
- Other video tutorials on AZ-400:
- Videos
- Previous Course (Abandonded)
- Social Media around Azure DevOps
- Azure’s Partners
- Independents
- References
- More on DevOps
- More about Azure
There are a lot of summary/high-level tutorials and videos. But this article skips the generalized sales hype and presents a “deep dive” with automation with logical and succinct hands-on steps so you get perficient quickly.
NOTE: Content here are my personal opinions, and not intended to represent any employer (past or present). “PROTIP:” here highlight information I haven’t seen elsewhere on the internet because it is hard-won, little-know but significant facts based on my personal research and experience.
Your settings (Naming Conventions)
PROTIP: What makes my tutorial here unique are (CLI and PowerShell) automation scripts I created to do (quickly and repeatedly) what manual clicking through GUI portal. Running the scripts locally on your laptop requires some installation.
But whether you use GUI or CLI, before diving in, define custom values to be used by Azure, by replacing defaults below with your own:
export AZDEVOPS_ORG_NAME="contoso" export AZDEVOPS_PROJ_NAME="ContosoWebApp" export AZDEVOPS_REST_VER="6.0" export AZDEVOPS_GITHUB_PAT="12928342342982342347abcdf2324234" export AZDEVOPS_USER_EMAIL="johndoe@gmail.com"
DEFINITION: The combination of {organization} / {project} is called a “route”.
Documentation below references the settings above.
Azure DevOps Product components
If you don’t need the history nor to create an Azure account, begin with my QuickStart tour with commentary (below).
Each widget on the Dashboard can be added or deleted using an API.
https://status.dev.azure.com/_history lists events and status in each geographical area (region) by product:
1). Azure Boards (like Jira) to plan, track, and discuss Work Items across teams using Kanban boards (to deliver value to users faster)
2). Azure Repos (like GitHub and AWS Code Commit, etc.) to use Git for source version, collaborate using pull requests, and file management
3). Azure Pipelines (like Jenkins, AWS CodeDeploy, etc.) for CI/CD automation
4). Azure Test Plans to manage manual and exploratory testing
5). Azure Artifacts (like Artifactory, Nexxus, etc.) for binary package consumption package management
Azure DevOps Services (https://azure.com/devops) adds to the above this product offering:
6). Azure DevTest Labs (like Sauce Labs) provides self-service creation of pre-provisioned sandbox environments from private custom images in VHD.
QUESTION: Do “Other services” include extensions installed from https://marketplace.visualstudio.com/items, such as Code Search, Analytics, etc.
Pricing
Each product above has its own pricing.
The Aure DevOps Release Notes presents Features Timeline (product roadmap) for additional “Areas”:
- Administration
- Notifications
Sign into aka.ms/AzureDevOpsForum = Developer Community Problems and Features (Active | Newest | Votes) |
NOTE: Azure DevOps works with more languages than just C# (Java, etc.).
Related Products and Services
Azure DevOps makes use of the following products (services and client executables):
NOTE: Microsoft exited the software testing market in 2019 by retiring their products.
-
Azure IAM access manager
-
AEX.dev.azure.com replaces Visual Studio at https://app.vssps.visualstudio.com/
-
Azure Resource Manager (ARM) carries out specifications entered in the Azure Dashboard. It deploys, updates, or deletes cloud resources in a single, coordinated operation. Resources can include virtual machines, storage accounts, virtual networks, services, or any component that you are managing. NOTE: VIDEO: ARM vs. Classic Azure Service Management
-
Azure Key Vault provides an HSM (vs. HashiCorp Vault, which enforces automatic revocation of leases with key usage audit and key rolling). KMS is focused on securely storing encryption keys and supporting cryptographic operations (encrypt and decrypt) using those keys. It supports access controls and auditing as well.
-
Azure Container Service (AKS) - uses Docker to make dynamic scaling easy on Kubernetes, Docker Swarm, or Mesos DC/OS. AKS handles auto upgrades, patching, and self-healing of Kubernetes clusters.
https://docs.microsoft.com/en-us/azure/aks/intro-kubernetes Introduction to Azure Container Service (AKS)
https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough Deploy an Azure Container Service (AKS) cluster
https://azure.microsoft.com/en-us/resources/videos/episode-198-azure-container-service-with-ross-gardler/ Cloud Cover Episode 198: Azure Container Service with Ross Gardler
-
Azure Container Registry (ACR) stores images for different types of container deployments (Swarm, DC/OS and Kubernetes, etc.) as well as Azure services such as App Service, Batch, and Service Fabric.
Unlike Docker Hub, ACR provides more control over who can see and use images, which ACR can sign cryptographically (to detect corruption) and encrypt at rest. The Premium SKU of Container Registry includes 500 GiB of storage that is geo-replicated.
ACR Tasks dynamically discovers base image dependencies when it builds a container image, so when changes are detected, automatically rebuilds application images.
-
Log Analytics (generally available)
-
Application Insights - the APM service works with Azure Service Profiler for ASP.NET code. Its API and SDKs (in .NET, Java, node.js) and TelemetryClient obtain metrics about requests, pages views, dependency calls, trace (log) messages, exceptions, and more. (Add Retrace to collect first chance exceptions)
Monitor Web Apps using Azure Application Insights Nov 18, 2016
Summary: DevOps tools overview for Azure DevOps
Rebranding History
“Azure DevOps” was a big deal for Microsoft when it was first announced Sep 16, 2018.
VIDEO: Buck Hodges (@tfsbuck), Director of Engineering for Microsoft VSTS, Jun 27, 2018 VIDEO: Global DevOps Bootcamp 2018 Keynote how Microsoft evolved from on-premise TFS into VSTS as a service shipping in 3 week sprints, from the same code repository. Add SPS (Shared Platform Service) for account, identity, profile, licensing.
dev.azure.com was VisualStudio.com
Back in Sept 2018 there was a name upgrade from Visual Studio Online (VSO) http://www.visualstudio.com which included capabilities in Visual Studio Team Services (VSTS), which began as a performance testing server and on-premise Team Foundation Server (TFS), now called “Visual Studio Server” on-premises.
“Visual Studio App Center” is for mobile iPhone/Android app development and integration (using Xamarin) so not in scope for this topic here. (@vsappcenter)
Competition
Forrester’s Continuous Delivery And Release Automation” market assesement ranked Microsoft among leaders in 2020:
Although Microsoft has gained “Market Presence” from rankings in 2018:
Microsoft’s “current offerings” continue to lag behind IBM and Cloudbees/Jenkins (which has shot up to the top ranking) after acquiring Electric Cloud. NOTE: IBM’s product is called “Urban Code”. IBM also acquired Red Hat and its Ansible portfolio. Broadcom bought CA. Digital.ai bought Xebia.
The acquisition of GitHub in 2020 has caused confusion about how to prepare for Microsoft’s future direction with Azure DevOps.
Microsoft doesn’t use the term “DevSecOps” for their product because it includes 3rd-party (from Marketplace) security testing tools rather than in-house tools.
AZ-400 certification
You need to have passed EITHER exam AZ-104 “Administrator Associate” OR AZ-203/204 “Developer Associate” before paying $165 to pass exam AZ-400 “Implementing Azure DevOps Solutions” to make you a “Microsoft Certified: Azure DevOps Engineer Expert”.
NOTE: Exam AZ-400 replaces exam “Implementing Microsoft Azure DevOps Solutions” 70-538. Although there are still free online courses from Microsoft at at EDx.org, the 9 video courses (of 8-16 hours each) Microsoft Professional Program for DevOps is now gone. It had covered these technologies:
- VSTS (Visual Studio Team System) - gone
- “Visual Studio” 2017/2019
- Azure Container Service (ACS) is like Docker and rkct (from Red Hat, pronounced like “rocket”).
- Application Insights
- Selenium (for functional testing)
- Operations Management Suite (OMS)
Thresholds for passing the AZ-400 exam:
A minimum score of 70 percent on the overall exam.
A minimum score of 35 percent on each exam domain.
Microsoft’s AZ-400 tutorials
Microsoft’s free textual tutorials and labs uses the same structure as the exam:
- Develop an instrumentation strategy (5-10%)
- Develop a Site Reliability Engineering (SRE) strategy (5-10%)
- Develop a security and compliance plan (10-15%)
- Manage source control (10-15%)
- Facilitate communication and collaboration (10-15%)
- Define and implement continuous integration (20-25%)
- Define and implement a continuous delivery and release management strategy (10-15%)
AZ-400 Certification Class Labs
PROTIP: I think the most useful learning experience is Microsoft’s 5-day live AZ-4100T00 class by Robert Tichelman & Vishal Nigam, but I highly recommend everyone get hands-on on the lab instructions in Github by Unai Huete Beloki and Michael Kenntenich.
Microsoft’s class Modules and labs at aka.ms/az400labs generally shows use of the GUI Console rather than CLI commands, which I have prepared in https://github.com/wilsonmar/azure-quickly, based on https://docs.microsoft.com/en-us/cli/azure/keyvault?view=azure-cli-latest
Module 0. Lab 00: Validate lab environment
-
- STAR: github.com/wilsonmar/azure-quickly
- Visual Studio Code
Module 1. Planning for DevOps
-
Lab : Agile Planning and Portfolio Management with Azure Boards
PROTIP: Instead of https://aex.devops.azure.com, use https://devops.azure.com.
- Transformation Planning
- Project Selection
- Team Structures
- Migrating to Azure DevOps
Module 2. Getting Started with Source Control [Azure Repos]
-
Lab : Version Controlling with Git in Azure Repos
- Git is the default version control provider for new projects.
- Clone an existing repository
- Save work with commits
- Review history of changes
-
Work with branches by using Visual Studio Code
- Introduction to GitHub
- Migrating from Team Foundation Version Control (TFVC) to Git in Azure Repos
Module 3. Managing Technical Debt
-
Lab : Sharing Team Knowledge using Azure Project Wikis
- Identifying Technical Debt
- Knowledge Sharing within Teams
- Modernizing Development Environments with Codespaces
Module 4. Working with Git for Enterprise DevOps
-
Lab : Version Controlling with Git in Azure Repos
- How to Structure Your Git Repo
- Git Branching Workflows
- Collaborating with Pull Requests in Azure Repos
- Why Care About Git Hooks
- Fostering Inner Source
- Managing Git Repositories
Module 5. Configuring Azure Pipelines
Module 6. Implementing Continuous Integration using Azure Pipelines
Module 7. Managing Application Configuration and Secrets
Module 8. Implementing Continuous Integration with GitHub Actions
Module 9. Designing and Implementing a Dependency Management Strategy
Module 10. Designing a Release Strategy
Module 11. Implementing Continuous Deployment using Azure Pipelines
Module 12. Implementing an Appropriate Deployment Pattern
Module 13. Managing Infrastructure and Configuration using Azure Tools
Module 14. Third Party Infrastructure as Code Tools Available with Azure
-
Lab 14b: Automating infrastructure deployments in the Cloud with Terraform and Azure Pipelines
- wilsonmar.github.io/terraform
- Azure DevOps Services Demo Generator provisions projects with pre-populated sample content based on a template of your choice. Each includes source code, work items, iterations, service connections, and build and release pipelines. Sign in at at https://azuredevopsdemogenerator.azurewebsites.net/. Click Sign-In.
Module 15. Managing Containers using Docker
Module 16. Creating and Managing Kubernetes Service Infrastructure
-
Lab : Deploying a Multi-Container Application to Azure Kubernetes Service
- Azure Kubernetes Service
- Kubernetes Tooling
- Integrating AKS with Pipelines
Module 17. Implementing Feedback for Development Teams
-
Lab 17: Monitoring Application Performance with Application Insights
- Implement Tools to Track System Usage, Feature Usage, and Flow
- Implement Routing for Mobile Application Crash Report Data
- Develop Monitoring and Status Dashboards
- Integrate and Configure Ticketing Systems
Module 18. Implementing System Feedback Mechanisms
-
Lab 18: Integration between Azure DevOps and Teams
- Site Reliability Engineering
- Design Practices to Measure End-User Satisfaction
- Design Processes to Capture and Analyze User Feedback
- Design Processes to Automate Application Analytics
- Managing Alerts
- Blameless Retrospectives and a Just Culture
Module 19. Implementing Security in DevOps Projects
-
Lab 19: Implement Security and Compliance in an Azure DevOps pipeline
-
Bolt is a lightweight open source security and management solution developed specifically for integration with Azure DevOps and Azure DevOps Server. It works per project and does not offer real-time alert capabilities, which requires Full platform, generally recommended for larger development teams that want to automate their open source management throughout the entire software development lifecycle (from the repositories to post-deployment stages) and across all projects and products. https://www.whitesourcesoftware.com/ integrates into build processes, irrespective of your programming languages, build tools, or development environments. It works automatically, continuously, and silently in the background, checking the security, licensing, and quality of your open source components against WhiteSource constantly-updated definitive database of open source repositories.
- Security in the Pipeline
- Azure Security Center
-
Module 20. Validating Code Bases for Compliance
-
Lab 20: Managing technical debt with SonarCloud and Azure DevOps
- https://sonarcloud.io/
- Open-Source Software
- Managing Security and Compliance Policies
- Integrating License and Vulnerability Scans
Create Azure account
There are the options:
Use your benefits
- Click Use your benefits.
- Click “Register” for 50 percent off the cost of a Microsoft Certification exam by completing your challenge within 30 days.
- Click “Register now”.
- Click “Get started” under “DevOps Engineer”.
- Click “Begin now”.
-
Work through Microsoft’s 27 free tutorial challenge:
- Capture Web Application Logs with App Service Diagnostics Logging - 55 min
- Control and organize Azure resources with Azure Resource Manager - 46 min
- Introduction to App Center - 49 min
- Deploy Spring microservices to Azure - 38 min
- Microsoft Azure Well-Architected Framework - Performance efficiency - 48 min
- Microsoft Azure Well-Architected Framework - Operational excellence - 54 min
- Analyze your Azure infrastructure by using Azure Monitor logs - 36 min
- Capture and view page load times in your Azure web app with Application Insights - 45 min
- Instrument server-side web application code with Application Insights - 34 min
- Monitor cloud resources - 40 min
- React to state changes in your Azure services by using Event Grid - 33 min
-
Design a holistic monitoring oy on Azure - 57 min
- Introduction to GitHub - 1 hr 12 min
- Migrate your repository by using GitHub best practices - 43 min
- Upload your project by using GitHub best practices - 43 min
- Manage repository changes by using pull requests on GitHub - 49 min
- Settle competing commits by using merge conflict resolution on GitHub - 52 min
- Search and organize repository history by using GitHub - 38 min
- Manage an InnerSource program by using GitHub - 50 min
- Communicate effectively on GitHub by using Markdown - 1 hr 2 min
- Maintain a secure repository by using GitHub best practices - 1 hr 9 min
- Automate DevOps processes by using GitHub Apps - 1 hr 8 min
- Automate GitHub by using GitHub Script - 25 min
-
Manage software delivery by using a release based workflow on GitHub - 1 hr 44 min
- Build continuous integration (CI) workflows by using GitHub Actions - 1 hr 7 min
- Build and deploy applications to Azure by using GitHub Actions - 59 min
- Implement a code workflow in your build pipeline by using Git and GitHub - 1 hr 27 min
Azure account and project
Alternately, these steps are based on Start using Azure DevOps but with my additional commentary:
-
Get signed up with Azure and Microsoft Learn accounts. Use of
https://aka.ms/aft-iot to Create your Azure free is described at
https://wilsonmar.github.io/azure-quicklyCreate your organization with a personal Microsoft account or a work or school accountdocs.microsoft.com
CLI/PowerShell Automation
If you prefer using ARM YAML, see: https://docs.microsoft.com/en-us/azure/devops/cli/azure-devops-cli-in-yaml?view=azure-devops
PROTIP: YAML runs fail if it’s not perfectly formatted (with no extra spaces, etc.).
Local client install for DevOps
Get prepared by installing CLI command programs for use in your Terminal sessions:
-
To obtain the .NET Core command-line interface (CLI) tool on a Mac, download file dotnet-install.sh stored in https://github.com/dotnet/cli, then run it.
chmod +x dotnet-install.sh ./dotnet-install.sh
PROTIP: This is a very well-written bash script.
The response:
dotnet-install: Downloading link: https://dotnetcli.azureedge.net/dotnet/Sdk/2.1.701/dotnet-sdk-2.1.701-osx-x64.tar.gz dotnet-install: Extracting zip from https://dotnetcli.azureedge.net/dotnet/Sdk/2.1.701/dotnet-sdk-2.1.701-osx-x64.tar.gz dotnet-install: Adding to current process PATH: `/Users/wilsonmar/.dotnet`. Note: This change will be visible only when sourcing script. dotnet-install: Installation finished successfully.
-
Verify:
dotnet --version
3.1.201
-
For more verbose:
dotnet --info
-
Node and NPM:
node
npm install bower -g v9.11.1 npm install grunt-cli -g
-
Verify your machine can get to the bottle resources from Homebrew:
curl -O https://formulae.brew.sh curl -O https://homebrew.bintray.com
-
To enable az CLI commands:
brew install azure-cli
Alternately, if you already have it installed and want to upgrade to the latest version:
brew upgrade azure-cli
-
Analytics:
brew info azure-cli
azure-cli: stable 2.15.1 (bottled), HEAD Microsoft Azure CLI 2.0 https://docs.microsoft.com/cli/azure/overview /usr/local/Cellar/azure-cli/2.15.1 (16,636 files, 221.9MB) * Poured from bottle on 2020-12-04 at 15:31:19 From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/azure-cli.rb License: MIT ==> Dependencies Required: openssl@1.1 ✔, python@3.8 ✔ ==> Options --HEAD Install HEAD version ==> Caveats Bash completion has been installed to: /usr/local/etc/bash_completion.d ==> Analytics install: 38,000 (30 days), 103,343 (90 days), 318,336 (365 days) install-on-request: 37,613 (30 days), 102,290 (90 days), 313,119 (365 days) build-error: 0 (30 days)
-
Analyze and act on caveats in response such as:
==> python Python has been installed as /usr/local/bin/python3 Unversioned symlinks `python`, `python-config`, `pip` etc. pointing to `python3`, `python3-config`, `pip3` etc., respectively, have been installed into /usr/local/opt/python/libexec/bin If you need Homebrew's Python 2.7 run brew install python@2 You can install Python packages with pip3 install
They will install into the site-package directory /usr/local/lib/python3.7/site-packages See: https://docs.brew.sh/Homebrew-and-Python ==> azure-cli Bash completion has been installed to: /usr/local/etc/bash_completion.d </pre> -
Confirm: (based on docs for macOS, Ubuntu, Windows 10, Azure Cloud Shell):
az --version
PROTIP: The CLI doesn’t assume a Python virtual environment, so it relies on finding the installed Python version.
azure-cli 2.24.2 * core 2.24.2 * telemetry 1.0.6 Python location '/usr/local/Cellar/azure-cli/2.24.2/libexec/bin/python' Extensions directory '/Users/wilsonmar/.azure/cliextensions' Python (Darwin) 3.8.10 (default, May 4 2021, 03:04:19) [Clang 11.0.0 (clang-1100.0.33.17)] Legal docs and information: aka.ms/AzureCliLegal You have 2 updates available. Consider updating your CLI installation with 'az upgrade' Please let us know how we are doing: https://aka.ms/azureclihats and let us know if you're interested in trying out our newest features: https://aka.ms/CLIUXstudy
Previously:
Extensions: azure-cli-iot-ext 0.8.7
-
Upgrade az if requested.
This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus Your current Azure CLI version is 2.24.2. Latest version available is 2.25.0. Please check the release notes first: https://docs.microsoft.com/cli/azure/release-notes-azure-cli Do you want to continue? (Y/n): y
docs.microsoft.com/en-us/cli/azure/get-started-with-azure-cli?view=azure-cli-latest to Get started with Azure CLI
Azure DevOps CLI extension
Based on: https://docs.microsoft.com/en-us/azure/devops/cli/?view=azure-devops
-
Optionally: Add CLI extension:
az extension add --name azure-devops
NOTE: The extension was built from https://github.com/Azure/azure-devops-cli-extension
PROTIP: Instead of the above command, run this next command and it prompts you to install if it’s not installed. A neat feature!
-
Widen your Terminal to list az devops commands detailed at docs.microsoft.com/en-us/azure/devops/cli/quick-reference?view=azure-devops
az devops -h
az devops : Manage Azure DevOps organization level operations. Related Groups az pipelines: Manage Azure Pipelines az boards: Manage Azure Boards az repos: Manage Azure Repos az artifacts: Manage Azure Artifacts. Subgroups: admin : Manage administration operations. extension : Manage extensions. project : Manage team projects. security : Manage security related operations. service-endpoint : Manage service endpoints/service connections. team : Manage teams. user : Manage users. wiki : Manage wikis. Commands: configure : Configure the Azure DevOps CLI or view your configuration. feedback : Displays information on how to provide feedback to the Azure DevOps CLI team. invoke : This command will invoke request for any DevOps area and resource. Please use only json output as the response of this command is not fixed. Helpful docs - https://docs.microsoft.com/rest/api/azure/devops/. login : Set the credential (PAT) to use for a particular organization. logout : Clear the credential for all or a particular organization.
NOTE: Reviews of this command -
Configure default configuration for your organization and project using your settings discussed in this document (above)
az devops configure --defaults organization="https://dev.azure.com/$AZDEVOPS_ORG_NAME" \ project="$AZDEVOPS_PROJ_NAME"
NOTE: If you’re connecting to an on-prem Azure DevOps Server, the URL would instead be:
organization=https://ServerName/CollectionName
-
To confirm the installation:
az extension show --name azure-devops
{ "extensionType": "whl", "metadata": { "author": "Microsoft", "author_email": "VSTS_Social@microsoft.com", "azext.minCliCoreVersion": "2.2.0", "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "Intended Audience :: System Administrators", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "License :: OSI Approved :: MIT License" ], "description": "Microsoft DevOps CLI Extension for Windows, Mac and Linux\n=========================================================\n\n1.0.0\n---------------------\n\n* Initial preview release.\n\n", "filename": "/Users/wilsonmar/.azure/cliextensions/azure-devops/azure_devops-0.18.0.dist-info", "home_page": "https://github.com/Microsoft/azure-devops-cli-extension", "license": "MIT", "metadata_version": "2.0", "name": "azure-devops", "platforms": [ "UNKNOWN" ], "requires_dist": [ "distro (==1.3.0)", "python-dateutil (==2.7.3)", "msrest (<0.7.0,>=0.6.0)" ], "summary": "Tools for managing Azure DevOps.", "version": "0.18.0" }, "name": "azure-devops", "path": "/Users/wilsonmar/.azure/cliextensions/azure-devops", "version": "0.18.0" }
QUESTION: “Programming Language :: Python :: 3.6” is the latest supported?
-
Set the memory variable “msrest (<0.7.0,>=0.6.0)”:
export AZDEVOPS_REST_VER="6.0"
devops login
-
In GitHub obtain a PAT (Personal Access Token) for the service account used to run pipelines and save it in a private place:
export AZDEVOPS_GITHUB_PAT="12928342342982342347abcdf2324234"
-
Sign in using the Azure CLI az login command or an Azure DevOps Personal Access Token (PAT)
az devops login --organization "http://dev.azure.com/$AZDEVOPS_ORG_NAME"
-
When prompted. paste the PAT. See https://docs.microsoft.com/en-us/azure/devops/cli/log-in-via-pat?view=azure-devops&tabs=windows
See https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=preview-page#create-a-pat
More CLI commands
https://stackoverflow.com/questions/64502148/how-to-securely-login-in-az-cli-from-a-devops-pipeline
STAR: VIDEO: references scaffoling script examples at https://github.com/Azure/azure-devops-cli-extension/tree/master/examples/Scaffolding
STAR: Also see https://www.dotnetcurry.com/devops/1528/azure-devops-cli
https://www.keithjenneke.com/getting-started-with-the-azure-devops-cli/
Invoke REST API from CLI
When the Azure DevOps CLI doesn’t cover a resource or action, can use the “catch-all” command to call the Azure DevOps REST API:
GET https://vsrm.dev.azure.com/{organization}/{project}/_apis/release/releases?api-version=6.0
az devops invoke --area release --resource releases \ --route --api-version "6.0-preview" -o json
-
To get a list of REST API releases:DOCS:
curl GET "https://vsrm.dev.azure.com/$AZDEVOPS_ORG_NAME/$AZDEVOPS_PROJ_NAME/_apis/release/releases?api-version=$AZDEVOPS_REST_VER"
https://vsrm.dev.azure/route/route/_apis/area/resource/api version
Add users to project team
Assuming members we want to add already exists within the Organization and the Project Group:
-
We have a Security Group.
-
Get Group Descriptor:
$groupDescriptor = az devops security group list \ --query "graphGroups[?contains(principalName,'Example1 Team')].[descriptor]" -o tsv
-
Get Member Descriptor:
$memberDescriptor = az devops user show \ --user "$AZDEVOPS_USER_EMAIL" \ --query user.descriptor -o tsv
-
Add members to the group:
az devops security group membership add \ --group-id $groupDescriptor --member-id $memberDescriptor
Show Build in browser
-
To show details of a specific build (id 1) in the default browser:
az pipelines build show --id 1 --open
Git branches
-
To make sure you have the correct branch selected:
David Tesar (@dtzar, host of DevOps-Dimension on channel9.msdn) wrote the two branches which visually display the same front-end website content with a SQL Azure back-end (all PaaS):
-
Branch aspnet45 contains code for ASP.NET 4.5 used on older “bare metal” machines. Don’t use this.
-
Branch master contains code for ASP.NET Core used within Containers. Use this.
-
GUI QuickStart: dev.azure.com was aex.dev.azure.com
-
In a browser, use a profile for each one of your emails (corporate employee, gmail, hotmail, outlook, etc.)
PROTIP: I use different browser programs so I can quickly press command+Tab (on a Mac) to switch between documentation (on Brave browser), apps (on Chrome), and Azure DevOps (on Microsoft Edge):
-
PROTIP: Click on your browser’s avatar picture to select the browser profile housing the browser cookies and history associated with the email address you are using (one email for corporate/client work, one for your personal gmail, one for hotmail/outlook, etc.).
-
In the Microsoft Edge browser, go to the “Azure Dev Essentials” site hosting Azure DevOps at:
-
If you’re using a browser profile, you would be logged in automatically based on prior session’s history. Otherwise, sign in using the same account you use for Azure billing.
-
Select “Default Directory” if you don’t have alternative tenants.
Account Profile GUI
-
Click on the “User Settings” icon at the top menu to the left of your account avatar:
TODO: Change profile items.
Create Organization
-
Click the blue “Create new organization” under title “Get started with Azure DevOps”.
PROTIP: Several organizations can be created under an account.
- Click “Continue” in the pop-up “Get started with Azure DevOps” with Terms and Conditions checked.
-
Optionally, change the Azure DevOps organization name Azure generates based on your username, such as:
https://dev.azure.com/wilsonmar0412
Many admins change the user name to a team name if several others will be using it.
-
“We’ll host your projects in” provides a limited number of locations/regions:
- West Europe
- East Asia
- Central US (the only one in the US)
- Brazil South
- Canada Central
- Australia East
- UK South
- South India
PROTIP: Create your other resources in the region selected to avoid cross-region network charges.
- Enter the CAPTCHA (two lines of the graphic are enter as one line).
-
Click the blue “Continue” for a “Create a project to get started” dialog.
You’ll get an email about the new organization.
Organization-level Settings
-
Click on “Organization Settings” at the lower-left corner for the Organization’s menu.
In the Overview page is where the default Region is specified for all projects.
-
To switch organizations or configure an organization, click the “Azure DevOps” icon at the upper-left corner:
Notice that work requests and pull requests are under an organization rather its projects.
-
Click on the organization name.
-
Configure profile, Security, Usage, Notification settings, Theme, etc.
Find what permissions you or a team member have, including project-level, collection-level, and object-level…docs.microsoft.com in Quick Start: View permissions for yourself or others - Azure DevOps & TFS
- Privacy URL. ???
-
Time zone: leave as default “UTC”, which doesn’t have Daylight Savings Time (Summer Time).
-
Navigate to Organization settings -> Billing -> Setup billing -> Select an Azure subscription. Select the Azure Pass subscription which results in the green “Subscription is valid”. Click “Save”.
CAUTION: That green “valid” line doesn’t appear if you now click “Change billing”. You’ll need to remove billing to see it again.
-
Set the “MS Hosted CI/CD” field “Paid parallel jobs” to 1 (from the default 0). Scroll down to the bottom and click “Save” (leaving all other defaults, such as 5 Basic users, etc.).
-
While waiting for new settings to be reflected in the back end (at least 3 hours before using CI/CD capabilities), you will see this message at ???
This agent is not running because you have reached the maximum number of requests…
Create project using CLI
-
Create project using the basic process template:
az devops project create --name "$AZDEVOPS_PROJ_NAME" \ --process basic \ --description "Az Devops CLI Example1"
Defaults: -s or –source-control (git)
–visibility (private)Project Generator
-
To create a new pre-defined project with pre-populated sample content (which include source code, work items, iterations, service endpoints, build and release definitions) based on a template you choose, open a new browser tab to:
https://azuredevopsdemogenerator.azurewebsites.net
NOTE: The site was built using github.com/microsoft/AzureDevOpsDemoGenerator
-
Click the blue “Sign In” for the same email you used in the steps above.
Troubleshooting
Close your browser windows and then open a new browser and navigate to
https://aka.ms/vssignoutClose this browser, open a new browser and navigate to https://aex.dev.azure.com/me, and sign-in using the desired tenant to reach “Get started with Azure DevOps”.
Create a new Organization at a Location of your choice.
Return to step 1 above to try the generator again.
- Click “Accept” after memorizing the default permission granted ;)
-
For New Project Name: include in the project name the template name to be selected below:
TailwindTraders
For production project, specify your product, according to your Naming Conventions.
- Select Organization: the one created above.
-
Leave unchecked “I want to fork this repository”.
Create project using a Template
NOTE: This is based on from code at https://github.com/CanarysAutomationsInternal/AppCenterDemoGenerator
https://azuredevopsdemogenerator.azurewebsites.net/environment/createproject
-
For “Selected template”: click “Select Template” then “TailwindTraders” because steps below require it.
NOTE: Don’t choose the others:
- TailwindTraders.com is described as a fictitious retail company created using an ensemble of reference apps using all-Microsoft tech running in Azure. It showcases “intelligent application experiences made smarter through data and AI”.
VIDEO: “we keep it fresh”:
- https://github.com/microsoft/tailwindtraders-website
-
https://github.com/microsoft/tailwindTraders-Backend
STAR: github.com/Microsoft/TailwindTraders lists step-by-step demo scripts, such as for Integrating Azure DevOps, Microsoft Teams and GitHub.
NOTE: The template creates a SQL database back-end at
github.com/microsoft/TailwindTraders-Backend -
“PartsUnlimited” (“PU”) at github.com/Microsoft/PartsUnlimited and PartsUnlimitedYAML defines an ASP.NET app with customized CI/CD pipelines. See below
-
SmartHotel360 and SmartHotel360-IoT open sourced on GitHub are now “retired, archived, and no longer supported”. It contained a complete ASP.NET 2 web mobile and desktop business apps for a hotel. The mobile app was built using Xamarin web-mobile app for a hotel, all deployed using Docker containers within AKS (Azure Kubernetes Service):
VIDEO: IoT demo shows the app powered by Azure Digital Twins to control lights and temperature of the hotel rooms, Dynamics 365, mobile Hololens for wayfinding. Demo Apps and Architecture. See the Facilities Management website running live here (admin/admin).
STAR: Azure superfan Gregor Suttie How to get started with Azure DevOps describes the steps using SmartHotel360. -
“MyHealthClinic” provisions a scrum based team project with code, work items for a sample ASP.NET Core web application-My Health Clinic. The template also includes pipeline definition to build and deploy the web app to Azure App Service.
-
“MyShuttle” defines a Java This template contains work items and source code for MyShuttle, a sample Java application backed by a MySQL database. This template also contains pipeline definitions to build the maven project and deploy the app to Tomcat server on Azure app service. Lab URL: Deploying a Java application on Tomcat
-
ContosoAir is a sample airline booking application with a Node.js based frontend with a CosmosDB database. This template provisions an Agile based project with user stories, features, epics and a CI&CD pipeline that builds and deploys code from GitHub.
- QUESTION: How to get “Mercury Health Group” sample app?
- TailwindTraders.com is described as a fictitious retail company created using an ensemble of reference apps using all-Microsoft tech running in Azure. It showcases “intelligent application experiences made smarter through data and AI”.
-
Right-click on ARM Outputs for a new browser tab to ARM Outputs extension page for Visual Studio (licensed product not available on MacOS).
“This extension reads the output values of an ARM deployment and sets them as Azure Pipelines variable”
- Click the green “Get it free”. Optionally: DOCS for use with Visual Studio.
- Click Install for your Organization.
-
Click “Proceed to organization”, such as “https://dev.azure.com/e4830gs/”
-
Return to the Azure DevOps Demo browser page.
-
Check the box about “third party” and click the blue “Create Project”
- Project gend created
- Required extensions are installed
- 1 team(s) created
- Board-Column, Swimlanes, Styles updated
- Created Wiki
- Build definition created
-
When “Congratulations” appears, click the blue “Navigate to project”.
Notice another tab is created with counts of work items created and completed in Azure Boards:
Run a test build
- To run a test build.
Parts Unlimited demo apps
- http://microsoft.github.io/PartsUnlimited (http://aka.ms/pumrplabs) describes the app.
- Front end service runs Apache Tomcat and talks to order service
- Order and Integration service runs Java and calls MongoDB
- Integration service integrates with Parts Unlimited Website
- Includes a Dockerfile and sample publishing profile to publish to a Docker container
- Includes ARM JSON templates and PowerShell automation scripts to easily build and provision your environment
https://github.com/Microsoft/PartsUnlimitedMRP
(PUMRP) is a fictional outsourced Manufacturing Resource Planning (MRP) app built entirely using open source software: Linux, Java, Apache, and MongoDB which creates a web front end, an order service, and an integration service.
- http://microsoft.github.io/PartsUnlimitedMRP describes the app.
-
When successful, you’ll get an email.
Create Project
-
Learn how to structure a project, manage users, and more to support your software development teams:
Quick Start: Get started as a project admin or organization owner in Azure DevOps
Sign up for Azure DevOps and invite teammates - Azure DevOps Quickstart guide to signing up and inviting others to join a project in Azure DevOpsdocs.microsoft.com
-
Type a project name, such as “PU”. Naming convention: The format of the URL is:
https://dev.azure.com/YourOrganization/project
NOTE: You can use the slash character.
- Leave Visibility as the default “Private” until it’s ready for Public viewing.
- Click “Advanced” to reveal more choices.
- Leave Version control default of “Git”.
-
Leave Work item process default of “Agile” for Azure Board terminology:
- Agile
- Basic (the default)
- CMMI
- Scrum
-
PROTIP: Bookmark the URL on the page for use in the future, such as:
https://dev.azure.com/wilsonmar0014/PUTo view this sample project populated, click it for a full-screen:
Project GUI menu
Notice the hierarchy at the top breadcrumb:
wilsonmar0964 / whatever1 / Overview / Summary
Expanded as the default are Overview menu items for the selected project (Summary, Dashboards, Wiki).
-
Clicking on a project menu category (such a “Repos”) contracts the previously selected menu items. So …
-
Click “«” at the lower-left corner to reveal the “Project settings” menu.
PROTIP: I leave the left menu minimized because mousing over the icon reveals its sub-menu .
Work Items
-
Notice that “My work items” and “My pull requests” are at the Organization level.
Import repo from GitHub
-
Check whether a Git Repo exists:
az repos list --query "[].name" -o tsv
-
In the GUI, be in your project.
- Click on Repos. Several options are shown.
- Click “Import”.
-
Paste the URL to where the repository is located. For this tutorial you have two choices which VIDEO: Microsoft has created several sample apps (with source code) implements the eCommerce website described as “Project Unicorn” in chapters 31-35 of The Phoenix Project by Gene Kim, Kevin Behr, and George Spafford, © 2013.
-
Click on the project name.
Notice that the project menu remains minimized.
Project Settings
Remember that the cog icon is associated with “Project settings”.
Workflows (Reference Architectures)
The work above kickstarts the first two steps in this sequence of work (from CI/CD for Azure VMs) described in this flowchart:
Click on diagram for full-screen pop-up
- Engineer uses Visual Studio to Git Commit changes into Azure Repos.
-
Commit application code and Azure Resource Manager template into Azure Repos.
PartsUnlimited Manual Deployment to Azure App Services with Visual Studio and ARM Templates (PartsUnlimited) provides a hands-on, step-by-step instructions:
- Setup Local Visual Studio 2017 environment
- Create Azure App Service Infrastructure from Visual Studio uing ARM Templates
- Publish the website to Azure App Service Production URL
- Publish the website to Azure App Service dev and staging sites
- Swap dev and staging site content
- Continuous integration triggers application build and unit tests in Azure DevOps Pipelines.
- Continuous Deployment trigger orchestrates deployment of application artifacts with environment-specific parameters.
- Deployment to QA environment using Azure DevTest Labs
- Deployment to Staging environment using Azure Virtual Machines.
- Deployment to production environment using Azure Virtual Machines.
- Azure Application Insights collects and analyzes health, performance, and usage data.
- Engineer reviews health, performance, and usage information.
- Update backlog item as complete in Azure DevOps Boards.
Other “Reference Architectures” with workflows:
- CI/CD for Containers
- CI/CD for Azure Web Apps
- Java CI/CD using Jenkins and Azure Web Apps
- Immutable Infrastructure CI/CD using Jenkins and Terraform
PROTIP: Notice that all of them have Visual Studio thick client (at the lower-left corner) as the interface for engineers rather than Visual Studio Code (VSCode).
You must install Visual Studio thick client
VIDEO: Tutorial: PartsUnlimited Setup with Visual Studio 2017 is obsolete because it uses VSTS. QUESTION: Difference with Visual Studio 2109 for Mac?
Enable “ASP. NET and web development” and “Azure development” during installation or Tools > Get Tools and Features, Modify.
-
PartsUnlimited Setup with Visual Studio (PartsUnlimited) to standardizing environments [80 minutes]
- Import Source Code into your Azure DevOps Account with Git.
- Create local Git repo from your Azure DevOps Git repo.
- Set up Service Endpoint in Azure DevOps.
- Import Continuous Integration Build pipeline into Azure DevOps and kick off a build.
- Import Continuous Deployment Release pipeline into Azure DevOps.
- Export Build and Release pipelines from Azure DevOps and commit changes to Azure DevOps repo to kick off CI and CD.
- Confirm successful deployment to Azure.
-
Set up Parts Unlimited MRP with VSTS (PartsUnlimitedMRP)is obsolete because VSTS is no longer available. -
Create CI infratructure using PartsUnlimitedMRP from GitHub, Python 3.5 within venv, and Travis CI to build and deploy to production. VIDEO
- Create an application using Python using Virtualenv for the environment isolation
- Create Unit Tests
- Declare Depenendcies
- Define the continuous build and test in our travis.yml file.
- Create a Github repository for this application and link it to our Travis CI account.
- Setup Travis CI
- Managing Pull Requests
-
Create a simple REST API and perform Load Tests using Locust (PartsUnlimitedMRP) on a minimal RESTful API using the Python 3.5 Flask framework.
-
CI and CD with VSTS - Quickstart (PartsUnlimited) to build and deploy to production [80 minutes]
Azure Service Endpoints
Extensions to Azure DevOps Services
-
Find and install free extensions for Azure DevOps Services on https://marketplace.visualstudio.com/azuredevops based on Quick Start: Install free extensions for Azure DevOps Services
-
If you click “Visual Studio”, the browser issues a URL such as:
vsweb://vs/?Product=Visual_Studio&EncFormat=UTF8&tfslink=dnN0Z…
This should open the Visual Studio for Mac app installed on your machine.
This is perhaps the most disheartening as the product currently does a clumsy job working with Git and GitHub.
- Install Visual Studio 2017 or 2019 within a Windows machine.
- In Tools, Get Tools and Features, select “ASP.NET and web development” and “Azure development”. These two should be check in “Workloads”
- Install PowerShell. The scripts folder contains .ps1 PowerShell scripts
- Download and install Node v6.12.3 (which has build tools taken out in v8).
- Install Bower and Grunt (ignore deprecated warning).
- ASP.NET 5 support for Linux and Mono
- Updated to .NET Core 2.0 in Jan 2018
- Modern HTML5 responsive layout using bootstrap for mobile, tablet, and PC
- Includes a Dockerfile and sample publishing profile to publish to a Docker container
- Supports multiple authentication options including Azure Active Directory, Google, and Facebook
- Azure Machine Learning product recommendations based on Order History
- Designed for Azure Websites, including Testing in Production, Staging slots and environment variables for feature flags (to turn off recommendations)
- Includes Grunt tasks for publishing assets to Azure Storage for CDN ingestion for faster performance
- Entity Framework code-first using SQL Azure or an in-memory database (Mono)
- Basic administration pages to add or edit product information
- Includes Azure RM JSON templates and PowerShell automation scripts to easily build and provision your environment
Instrumentation (Logging) Strategy
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-designing-implementing-logging Logging
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-designing-implementing-telemetry Design and implement Telemetry metrics (Application insights) alert rules, user behavior insights
SRE Strategy
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-developing-actionable-alerting-strategy Develop an actionable alerting strategy
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-designing-failure-prediction-strategy Design a Failure Prediction strategy
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-designing-implementing-health-checks Implement Health checks
Security Plan
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-designing-authentication-authorization-strategy
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-designing-sensitive-information-strategy
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-developing-security-compliance
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-designing-governance-enforcement-mechanisms
Source Controls
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-developing-modern-source-control-strategy
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-integrating-source-control-tools
Facilitating Communication and Collaboration
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-communicating-collaborating-stakeholders
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-generating-devops-process-documentation
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-automating-communication
Define and Implement Continuous Integration
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-designing-build-automation
https://app.pluralsight.com/library/courses/microsoft-dev-ops-solutions-designing-package-management-strategy
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-designing-application-infrastructure-management-strategy
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-implementing-maintaining-standardizing-build-strategies
Define and implement a continuous delivery and release management strategy
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-developing-deployment-scripts-templates
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-implementing-orchestration-automation-solutions
https://app.pluralsight.com/library/courses/microsoft-devops-solutions-planning-deployment-environment-strategies
API to DevOps
Azure DevOps Services REST API Reference v5 was defined in 2016 with libraries:
- .NET conceptual documentation and .NET reference documentation by extending TFS via C#
- Node.js
- Python
- Swagger 2.0 specification
- Web Extensions SDK
QUESTION: Were there SOAPUI or Postman files built for this API?
Resources associated with a particular product from Microsoft Learn are provided below.
Azure Boards
Websites: Product | Docs | Get started
VIDEO: Using Azure Boards with GitHub
Quickstart guide to plan and track work - Azure Boards Plan and track work in your new team project on Azure Boardsdocs.microsoft.com
Start using Azure Boards (Agile process)
-
Add & update work items, approve releases, view work tracking progress with Stakeholder accessdocs.microsoft.com
Azure Repos
Websites: Product |
Quick Start: Code with Git in Azure DevOps - Azure DevOps Services & TFS - Learn how to share code in a Git repo and new projectdocs.microsoft.com
GitHub Actions
More than just hooks.
VIDEO: Using GitHub Actions to Deploy to Azure Mar 28, 2019
See https://github.com/azure/github-actions
Azure Artifacts
Websites: Product |
To host private Nuget (Windows), npm (NodeJs), Maven (Java), Python packages with builds.
Package management such as Artifactory.
-
Quickly access artifacts by favoriting them in Azure DevOps Services & Team Foundation
Azure Test Plans
Websites: Product |
Azure Pipelines
Websites: Product | User Guide
Azure DevOps Repos and Pipelines are free starting prices for up to 5 people running 10 parallel jobs in 1 hosted job up to 1,800 minutes per month. I
“WhiteSource Bolt” is a security analysis tool (like Black Duck license rating, SonarQube, Open Web Application Security Project)
https://github.com/rfennell/AzurePipelines
VIDEO: azure-pipelines.yml files
CLI can be called from within an Azure Pipeline:*
- task: AzureCLI@2 displayName: Publish Function inputs: azureSubscription: Name of the Azure Resource Manager service connection scriptType: ps scriptLocation: inlineScript inlineScript: | func azure publish function-name
References:
-
Azure Pipelines overview showing PartsUnlimited “Piplines work with many languages” by Mickey Gousset
-
https://pleasereleaseme.net/deploy-a-dockerized-application-to-azure-kubernetes-service-using-azure-yaml-pipelines-4-running-a-dockerized-application-locally/
Azure DevTest Labs
Websites: Product
- Quickly provision development and test environments
- Minimize waste with quotas and policies
- Set automated shutdowns to minimize costs
- Build Windows and Linux environments
Resource Groups
Factors to consider when defining resource groups (containers):
- All the resources in a group should share the same lifecycle. You will deploy, update, and delete them together. If one resource, such as a database server, needs to exist on a different deployment cycle, it should be in another resource group.
- Each resource can only exist in one resource group at a time.
- You can add or remove a resource to a resource group at any time.
- You can move a resource from one resource group to another group.
- A resource group can contain resources that reside in different regions.
- A resource group can be used to scope access control for administrative actions.
- A resource can be linked to a resource in another resource group when the two resources must interact with each other, but they do not share the same lifecycle (for example, multiple apps connecting to a database).
Up to 15 tags per ARM resource to logically organize resources for cost accounting. Tag names are limited to 512 characters. Tag values are limited to 256 characters.
https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-authoring-templates Understand the structure and syntax of Azure Resource Manager templates
See https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-using-tags
https://azure.microsoft.com/en-us/documentation/articles/resource-group-overview/#access-control
https://www.youtube.com/watch?v=h0UDIcRnPog
QUESTION: How are ARM Templates version controlled?
RBAC (Role-based Access Control)
Access to specific actions are limited to each user based on his/her assigned role.
https://azure.microsoft.com/en-us/documentation/articles/resource-group-overview/#access-control
In addition to built-in roles such as “Reader”, add custom roles using templates such as:
{ "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#", "contentVersion": "", "parameters": { }, "variables": { }, "resources": [ ], "outputs": { } }
- $schema: The location of the JSON schema file that describes the version of the template language.
- contentVersion: The version of the template (such as 1.0.0.0).
- parameters: The optional values that are provided when deployment is executed to customize resource deployment.
- variables: The values that are used as JSON fragments in the template to simplify template language expressions.
- resources: A manageable item that is available through Azure. Some common resources are a virtual machine, storage account, web app, database, and virtual network, but there are many more.
- outputs: The values that are returned after deployment
Azure Templates
Deploy predefined templates from the Azure Marketplace, the https://github.com/Azure/azure-quickstart-templates QuickStart Templates repo, or as a local private in PowerShell commands such as:
New-AzureRmResourceGroupDeployment -Name ExampleDeployment -ResourceGroupName ExampleResourceGroup TemplateFile PathToTemplate
Alternately, include parameters inline string:
New-AzureRmResourceGroupDeployment -Name ExampleDeployment -ResourceGroupName ExampleResourceGroup -myParameterName "parameterValue"
https://docs.microsoft.com/en-us/azure/active-directory/role-based-access-control-configure
Ansible
Create a basic virtual machine in Azure with Ansible
Install and configure Ansible to manage virtual machines in Azure
Below is an sample playbook that creates an Azure VM and configures SSH credentials.
- name: Create Azure VM hosts: localhost connection: local tasks: - name: Create VM azure_rm_virtualmachine: resource_group: myResourceGroup name: myVM vm_size: Standard_GS5-8 admin_username: azureuser ssh_password_enabled: false ssh_public_keys: - path: /home/azureuser/.ssh/authorized_keys key_data: "ssh-rsa CCDDV2aZ...WXhad10h" image: offer: UbuntuServer publisher: Canonical sku: '16.04-LTS' version: latest
https://docs.ansible.com/ansible/2.9/modules/list_of_cloud_modules.html#azure
Terraform
Install and configure Terraform to provision VMs and other infrastructure into Azure
Create a complete Linux virtual machine infrastructure in Azure with Terraform
GitHub repositories
https://github.com/Microsoft/azure-devops-dotnet-samples contains C# samples that show how to integrate with Azure DevOps Services and Azure using our public client libraries, service hooks, and more.
https://github.com/Microsoft/DevOps-Architecture has been archived in 2019. It was a (hands-on) guide on building a robust professional devops environment for ASP.NET Core using Azure DevOps by Jeff Palermo (podcast) identifies these for a “professional-grade DevOps environment”:
- Private build
- Continuous integration build
- Static code analysis
- Release candidate versioning and packaging
- Environment provisioning and configuration
- Minimum of a three-tier deployment pipeline
- Production diagnostics managed by development team
- Insanely short cycle time through the previous steps
Quick-reference posters on Azure DevOps Architecting, Getting Started, Habits, Practices, Technology from 2019.
https://github.com/mpeder/azdevopssecurity Azure DevOps: Recommended Practices for Secure Pipelines
https://github.com/Microsoft/devops-project-samples https://github.com/Azure/azure-cli https://github.com/Microsoft/azure-devops-auth-samples https://github.com/Azure/azure-devops-cli-extension https://github.com/Azure/azure-devops-utils
https://github.com/Microsoft/azure-devops-node-api https://github.com/Microsoft/azure-devops-python-api https://github.com/benmatselby/go-azuredevops https://github.com/Dinomite-Studios/unity-azure-pipelines-tasks
https://github.com/Azure/azure-api-management-devops-resource-kit
From London http://www.dwyl.io/ https://github.com/dwyl/learn-devops https://github.com/dwyl/learn-microsoft-azure
PowerShell Cmdlets for Azure DevOps and Team Foundation Server
https://github.com/Azure/DevOps-For-AI-Apps
Microsoft Rock Stars
Sam Guckenheimer, Product Owner, Visual Studio Cloud Services at Jeffery Palermo’s VIDEO: AzureDevopsPodcast.clear-measure.com State of Devops report
- Decision cycle: observe, orient, decide, and act
- Change Agents for DevOps
- Reflecting on the DevOps Journey - http://aka.ms/OurDevOpsJourney.
- WinOps 2017 Sam Guckenheimer - Moving 70,000 Microsofties to DevOps on the Public Cloud at the WinOps Conference [51:03]
- Rugged DevOps and DevOps Anti-Patterns on Channel9
DevOps Cloud Developer Advocates
was abandoned.
Donovan Brown (@DonovanBrown, http://donovanbrown.com: slide-decks.
Abel Wang (@AbelSquidHead, abelsquidhead.com) on development
- with Aplitools Apr 15, 2019
- Using the new Basic Process in Azure DevOps with Dan Hellem Apr 8, 2019 [8:17]
Jessica Deen (@jldeen, jessicadeen.com) on Kubernetes, open-source, Linux
Steve Marascky (https://twitter.com/StevenMurawski, https://stevenmurawski.com) on DSC, SRE
Damien Brady (@) on Octopus Deploy
Others in Microsoft
YOUTUBE: Continuous Integration, Continuous Deployment (CI-CD) with Azure DevOps Nov 1, 2018 by Frank Boucher in Montreal
Microsoft Developer Advocate and AI enthusiast Aaron (Ari) Bornstein (@pythiccoder, on Medium)
https://docs.microsoft.com/en-us/learn
https://docs.microsoft.com/en-us/azure/devops/learn/
AZ-400 Practice exams
-
Whizlabs???
Case Studies:
Other video tutorials on AZ-400:
-
VIDEO: John Saville has video courses on Pluralsight with his “Master Class” on YouTube with code in his GitHub, which he’s updating to include DevOps.
-
Paul Hacker created on 7/11/2019 for LinkedIn Learning
Videos
Azure DevOps YouTube channel lists ppt’s with videos on the DevOps Events and Talks webpage :
-
Azure DevOps Launch Sep 10, 2018 (6,000+ subscribers)
DevOps For ASP.NET Developers series on Channel9 by by Donovan Brown, Cecil Phillip, Rich Lander, Jeremy Likness, AbelSquidHead
-
Pt.1 - What is DevOps? Apr 15, 2019 by Donovan Brown, Rich Lander, Jeremy Likness. AbelSquidHead convinces (hilariously) skeptical developer Cecil Phillip about putting the SmartHotel360 app into DevOps.
-
Pt.2 - Source Control Apr 15, 2019 by Donovan Brown, Cecil Phillip, Rich Lander, Jeremy Likness, AbelSquidHead. You have the option of a centralized version control with TFVC or distributed version control with Git.
-
Pt. 3 - Work Item Tracking Apr 16, 2019 Being able to visualize the work ahead is an integral part to the success of any software project. With Azure Boards, you can quickly and easily start tracking tasks, features, and bugs associated with runfaster2000.
DevOps courses in Microsoft Virtual Academy (until April 30, 2019) http://aka.ms/DevOpsLearn
-
Meet the Engineers: Building Line-of-Business Apps at Microsoft with CI/CD 29 June 2018 [57:17]
-
Track updates made to a work item or pull request by following it when using Azure Boards or Team Foundation Serverdocs.microsoft.com
-
Add custom security groups, change permissions for groups or individuals tutorialdocs.microsoft.com
-
Set permissions to grant or restrict access to select build, version control, or work tracking functionsdocs.microsoft.com
Grant or restrict access to select features - Azure DevOps & TFS
-
Connect a client to the cloud service Azure DevOps Services or on-premises Team Foundation Server (TFS)docs.microsoft.com
Connect to a project from a web browser or supported client in Azure DevOps - Azure DevOps & TFS
See http://stories.visualstudio.com/devops
Previous Course (Abandonded)
Alas, EdX has removed their DEVOPS200.5x set of classes by Faculty: Eamonn Kelly, Samantha Lindsey Ahmed, Steve Borg (@stevenborg, steven.borg@nwcadence.com), Sachi Williamson
-
Introduction to DevOps Practices
DEVOPS200.1x by Steven Borg
Learning objectives: Define DevOps and describe its value, history, and building blocks. Define a sustainable DevOps process. Identify appropriate compliance, security, and secrets management strategies. Identify ways to motivate key stakeholders along the DevOps journey.
- Team autonomy and enterprise alignment
- Rigorous management of technical debt
- Focus on flow of customer value
- Hypothesis-driven development
- Evidence gathered in production
- Live-site culture
- Manage infrastructure as a flexible resource
principles and practices: State of DevOps report from Puppet video.
- https://www.devopsassessment.net/ - Microsoft’s DevOps Self-Assessment
- How to deploy to IaaS and PaaS environments in Microsoft Azure using ARM templates, Desired State Configuration (DSC) and other deployment tools.
- Build and deploy applications automatically to Dev, Test, and Production environments.
- Continuous learning from production to improve and scale business results.
- Unit tests, Integration tests, load tests, UI tests and test driven-development
- Data retention strategies, exposure control strategies, and hypothesis-driven development.
- VIDEO: “snowflake” servers that sticks around (also called “pets”) differs from other servers.
-
DEVOPS200.2x
- Azure Automation
- ARM templates
- Desired State Configuration (DSC) [Powershell]
- DevTest labs
- Chef deployments in Azure
- Puppet deployments in Azure
DEVOPS200.3x
- Configure Git and TFVC version control options for Continuous Integration
- How to manage Technical Debt and the sources and impact of it.
- How to create a continuous integration build using VSTS
- Package Management Integration options such as NuGet, SemVer, GitVersion and others.
- Agents and Pipelines, including how to release pipelines
- Continuous Delivery and Release Management strategies
- Integrating automated testing into your release pipelines using Selenium, Coded UI Testing, Microsoft Test Manager
- Understand the various availability and performance testing options
- Automated provisioning and de-provisioning of infrastructure and databases D
- Deployment groups, feature flags and recovery automation
- Database deployment in release pipelines
- Continuous deployment with Jenkins and VSTS
-
Configuration Management for Containerized Delivery
DEVOPS200.4x
- Create container images
- Manage multiple containers using tools like Docker
- How to set up a production cluster to host your containers
- Deploy containerized applications to different orchestrators that are available in Azure Container Service (ACS) i.e. DC/OS, Docker swarm and Kubernetes
- How to scale up the clusters, manage data and set up monitoring to proactively keep track of the clusters health and its deployed applications
- Create, monitor and manage a Service Fabric cluster.
- Manage and maintain Azure hosted clusters and containers in a better way.
- Deploy containerized applications to one of the available cluster container solutions, using continuous delivery pipelines.
- Manage these clusters in production scenarios.
Videos:
DEVOPS200.5x
Test Driven Development Studio: https://aka.ms/edx-devops200.5x-tdds Complete DevOps Solution: https://aka.ms/edx-devops200.5x-cdos
- Understand various testing types and usage scenarios
- Test-Driven development and it’s benefits
- How to create and implement Unit tests and the elements of a good test
- How to create basic API tests (or Integration tests)
- Performance testing and how to create a performance test using Visual Studio and Visual Studio team services (VSTS)
- How to analyze performance test results using Application Insights
- Exploratory testing in the context of visual Studio and Microsoft Test Manager
-
DEVOPS200.6x
- Define DevOps
- Identify the challenges of using databases that are separate from other software languages and platforms
- Include your database code alongside other application code in a version control system (VCS)
- Set up a Continuous Integration (CI) platform for your database code
- Write and include automated unit tests for your database code
- Develop an automated release process that deploys database changes to both on premise and cloud databases
- Implement branching and merging for your database code
- Instrument and monitor the database after deployment
-
Application Monitoring and Feedback Loops
DEVOPS200.7x by Tiago Pascoal
- Understand general application monitoring and feedback loop practices and principles.
- The different kinds of feedback and how they are used in different stages of the value stream, and their benefits.
- How to set up up monitoring with Azure Application Insights
- Monitor web application availability
- Search and analyze monitoring data in Application Insights
- How to use and query Application Insights data and Application Maps
- How to set up, configure, query and analyze data collection on Operations Management Suite (OMS) Log Analytics
- How to set up alerts in OMS and integrate Application Insights
- Configure and monitor a web application with New Relic
- Configure and use Loggly
Choose either 8 or 9:
DEVOPS200.9x
- Distributed cloud application fundamentals, including Why Cloud Apps?, embracing failure, orchestrators, when to split a monolith into microservices, 12-factor services, and when and how to use Containers.
- Networking communication, including service scalability and availability, how to define/manage/version service endpoint APIs, and how to perform fault-tolerant network communication.
- Messaging communication, including the benefits of messaging with queues and fault-tolerant message processing.
- Versioning, Upgrading, and Configuration, including various was to version your service’s code, how to shut down a service instance gracefully, and how to configure and share secrets with a running service.
- Data storage services, including storage service considerations, object/file storage services, relational and nonrelational databases, partitioning, replicas, eventual consistency patterns (CQRS, Event sourcing, Saga), concurrency patterns, and data schema versioning.
- Disaster recovery, including backup/restore, recovery point and time objectives, as well as Active/Passive and Active/Active architectures.
DEVOPS200.10x
- Automating Infrastructure using Azure Resource Manager (ARM) Templates
- Implementing Continuous Integration solutions
- Implementing continuous delivery and continuous deployment solutions with Visual Studio Team Services (VSTS)
- Implementing Testing solutions such as Unit Tests and Testing in Production
- Implementing Application Monitoring solutions using Application Insights
50% of the grade is to pass (by 70%) in five chances two Validated labs within 120 minutes each:
Microsoft Professional Program (MPP) in DevOps provides only two attempts to complete labs using PartsUnlimited sample reposistories.
1: Continuous Integration with Azure DevOps
- At https://visualstudio.microsoft.com/ Create a Azure DevOps account, a new Azure DevOps team project, and a personal access token
- Setting up repo in Azure DevOps
- Setting up continuous integration in Azure DevOps
2: Continuous Deployment
- Modify the CI build to include ARM Templates to be used for deployment to Azure
- Create an Service Principle Name service Endpoint to allow you deploy to your subscription in Azure
- Create release Definition > Deploy Infrastructure
- Create release Definition > Publish PU App to Deployed Azure App Service Infrastructure
- Create release Definition > Clone Dev Environment to Staging and Production
- Verify release Definition Deployment
Social Media around Azure DevOps
Issues with MicrosoftDocs are reported as issues within
https://github.com/MicrosoftDocs/feedback/issues.
Sign into aka.ms/AzureDevOpsForum = Developer Community Problems and Features (Active | Newest | Votes) |
Podcasts:
DevOps Interviews
The DevOps Lab
LinkedIn?
Azure’s Partners
In addition to Marketplace developers:
https://projectum.com/technologies/microsoft-azure-devops/
https://www.telerik.com/blogs/microsoft-azure-devops-what-you-need-to-know
https://www.preemptive.com/blog/article/1055-automating-and-scaling-app-protection-with-azure-devops/90-dotfuscator
https://www.mobilize.net/blog/vbuc-azure-devops
Independents
YOUTUBE: Intro to Azure DevOps - Source Control, CI/CD, Automation, and more Dec 17, 2018 by IAmTimCorey
https://medium.com/devops-cloud-it-career/microsoft-azure-devops-start-here-10c46efa4a76
References
Test Driven Development in Visual Studio - An environment for practicing Kent Beck style TDD [F.I.R.S.T. Unit Tests, fast builds, No Mocks, Hexagonal architecture] Open source alternative to nCrunch
The Salt platform at https://github.com/saltstack/salt comes with different components such as Salt Masters, Salt Minions, Top Files, and Salt Cloud. Its axis: Remote execution Configuration automation Cloud control Event-driven orchestration See https://www.microsoft.com/developerblog/2017/05/09/provision-configure-infrastructure-azure-using-saltstack/
https://devops.com/devops-help-hinder-compliance/
https://www.youtube.com/watch?v=Bo_84yKsxuc
Configuring and Operating Microsoft Azure Stack (INF240x) 25-30 hour enrollment closed on 6/25/19.
https://testingindevops.org/
Coursera guided project: Executing Selenium Test Automation With Azure DevOps
References:
https://www.youtube.com/watch?v=VNHmX8_McqI Watch Me Code - Debugging a Broken DevOps Pipeline
More on DevOps
This is one of a series on DevOps:
- DevOps_2.0
- ci-cd (Continuous Integration and Continuous Delivery)
- User Stories for DevOps
- Git and GitHub vs File Archival
- Git Commands and Statuses
- Git Commit, Tag, Push
- Git Utilities
- Data Security GitHub
- GitHub API
- Choices for DevOps Technologies
- Pulumi Infrastructure as Code (IaC)
- Java DevOps Workflow
- AWS DevOps (CodeCommit, CodePipeline, CodeDeploy)
- AWS server deployment options
- Cloud services comparisons (across vendors)
- Cloud regions (across vendors)
- Azure Cloud Onramp (Subscriptions, Portal GUI, CLI)
- Azure Certifications
- Azure Cloud Powershell
- Bash Windows using Microsoft’s WSL (Windows Subsystem for Linux)
- Azure Networking
- Azure Storage
- Azure Compute
- Digital Ocean
- Packer automation to build Vagrant images
- Terraform multi-cloud provisioning automation
-
Hashicorp Vault and Consul to generate and hold secrets
- Powershell Ecosystem
- Powershell on MacOS
- Jenkins Server Setup
- Jenkins Plug-ins
- Jenkins Freestyle jobs
- Docker (Glossary, Ecosystem, Certification)
- Make Makefile for Docker
- Docker Setup and run Bash shell script
- Bash coding
- Docker Setup
- Dockerize apps
- Ansible
- Kubernetes Operators
- Threat Modeling
- API Management Microsoft
- Scenarios for load
- Chaos Engineering
More about Azure
This is one of a series about Azure cloud:
- Azure cloud introduction
- Azure Cloud Onramp (Subscriptions, Portal GUI, CLI)
- RDP client to access servers
- Bash Windows using Microsoft’s WSL (Windows Subsystem for Linux)
- Microsoft PowerShell ecosystem
- Azure Cloud Powershell
- PowerShell DSC (Desired State Configuration)
- PowerShell Modules
- Azure Networking
- Azure Storage
- Azure Compute
- Azure cloud DevOps
- Dockerize apps
- Kubernetes container engine
- Hashicorp Vault and Consul for keeping secrets
- Hashicorp Terraform
- Ansible
- Microsoft AI in Azure cloud
- Azure Monitoring
- Azure KSQL (Kusto Query Language) for Azure Monitor, etc.
- Dynatrace cloud monitoring
- Cloud Performance testing/engineering
- Cloud JMeter