Wilson Mar bio photo

Wilson Mar

Hello!

Calendar YouTube Github

LinkedIn

Azure Cloud

The #2 public cloud is a leader as well

US (English)   Norsk (Norwegian)   Español (Spanish)   Français (French)   Deutsch (German)   Italiano   Português   Estonian   اَلْعَرَبِيَّةُ (Egypt Arabic)   Napali   中文 (简体) Chinese (Simplified)   日本語 Japanese   한국어 Korean

Overview

This article contains higher and practical level details about Microsoft Azure, but with less confusing grandiose marketing generalizations.

Cloud Computing Terms Dictionary

  • “Data estate” refers to all the data an organization owns, regardless of where it is stored.

Naming conventions for Azure resources

Microsoft’s Azure cloud was first announced in 2008 and released in 2010.

Why?

My Cloud Comparison article defines the why:

  • Money Cost (OpEx vs. CapEx)
  • Time - Speed (Quick provisioning)
  • Global scale
  • Productivity
  • Performance (reduced network latency and greater economies of scale)
  • Reliability (data backup, disaster recovery, and business continuity easier and less expensive because data can be mirrored at multiple redundant sites on the cloud provider’s network)

Architectural components

azure compute platform 650x252-c60.jpg

End-Users buy SaaS (Software as a Service) online with only an internet browser (and a credit card):

  • Office 365
  • Skype
  • Dynamics CRM
  • Salesforce
  • Lucid Charts to draw diagrams

Developers interact with a platforms as a service (PaaS) for “Rapid Development”:

  • Service Fabric apps
  • Power (BI) apps
  • Web apps

  • Mobile apps (Xamarin)

  • Media Services
  • Stream Analytics

Operations people interact with Infrastructure as a service (IaaS) components for “High Control”:

  • Azure Service Fabric
  • Azure Batch
  • Define Virtual Machines
  • Define VM Scale Sets
  • VM Extensions
  • Azure Container Service that uses Docker Swarm
  • Cloud Foundry
  • Open Shift
  • Kubernetes
  • Apprenda
  • Jelastic

Azure Stack runs Azure runs within a private data center.

Web services

The big picture of Azure services: azure-big-picture-1923x1083-160564.jpg Click diagram for full-frame pop-up

This is missing some new services such as DevOps.

https://docs.microsoft.com/en-us/azure/architecture/icons/

Visual Studio Dev Essentials provides a list of tools and ecosystem.

Cloud Operating Model (COM)

Overview: Microsoft’s Cloud Operating Model covers business, people, and technology strategies to identify where an organization is in the digital transformation journey, identify triggers and opportunities for cloud migration, and recognize these components needed to develop a digital transformation strategy.

A. Meet business requirements
B. Assess organization maturity
C. Strategize business impact
D. Upgrade business processes
E. Identify skills gap
F. Identify migration portfolio
G. Perform migration
H. Modernize the business

Cloud Adoption Framework (CAF)

Microsoft’s CAF is used by Technical Leadership (CISO, CIO) to accelerate each stage of their cloud adoption journey. CAF organizes a set of tools, templates, guidance, and narratives - starting from why cloud, what to move, where to move, how to move, how to manage & operate in cloud.

References:

MCRA

VIDEO: aka.ms/MCRA => Microsoft Cybersecurity Reference Architectures (MCRA) PPTX by Mark Simos [GitHub]

MCSB

v1 of the Microsoft Cloud Security Benchmark was released 12/07/2022 to provide prescriptive best practices and recommendations to help improve the security of workloads, data, and services on Azure and multi-cloud environments.

https://learn.microsoft.com/en-us/security/benchmark/azure/overview

MCSB succeeds Microsoft’s Azure Security Benchmark (ASB) rebranded in October 2022. “Benchmark” is borrowed from the Center for Internet Security (CIS) Benchmarks

The MCSB XLSX (Excel) file is organized into 12 control domains:

  1. NS-10 (Network security) - secure and protect networks, including securing virtual networks, establishing private connections, preventing, and mitigating external attacks, and securing DNS.

  2. DP-8 (Data Protection) - at rest, in transit, and via authorized access mechanisms, including discover, classify, protect, and monitor sensitive data assets using access control, encryption, key management and certificate management.

  3. IM-9 (Identity Management) - establish a secure identity and access controls using identity and access management systems, including the use of single sign-on, strong authentications, managed identities (and service principals) for applications, conditional access, and account anomalies monitoring.

  4. PA-8 (Privileged Access) - protect privileged access to tenant and resources, including a range of controls to protect your administrative model, administrative accounts, and privileged access workstations against deliberate and inadvertent risk.

  5. PV-7 (Posture and Vulnerability Management) - assessing and improving cloud security posture, including vulnerability scanning, penetration testing and remediation, as well as security configuration tracking, reporting, and correction in cloud resources.

  6. LT-7 (Logging and Threat Detection) - detecting threats on cloud, and enabling, collecting, and storing audit logs for cloud services, including enabling detection, investigation, and remediation processes with controls to generate high-quality alerts with native threat detection in cloud services; it also includes collecting logs with a cloud monitoring service, centralizing security analysis with a SIEM, time synchronization, and log retention.

  7. AM-5 (Asset Management) - ensure security visibility and governance over your resources, including recommendations on permissions for security personnel, security access to asset inventory, and managing approvals for services and resources (inventory, track, and correct).

  8. ES-3 (Endpoint Security) - detection and response, including use of endpoint detection and response (EDR) and anti-malware service for endpoints in cloud environments.

  9. BR-4 (Backup and Recovery) - data and configuration backups at the different service tiers are performed, validated, and protected.

  10. IR-7 (Incident Response) - preparation, detection and analysis, containment, and post-incident activities, including using Azure services (such as Microsoft Defender for Cloud and Sentinel) and/or other cloud services to automate the incident response process.

  11. DS-7 (DevOps Security) - security engineering and operations in the DevOps processes, including deployment of critical security checks (such as static application security testing, vulnerability management) prior to the deployment phase to ensure the security throughout the DevOps process; it also includes common topics such as threat modeling and software supply security.

  12. GS-10 (Governance and Strategy) - ensuring a coherent security strategy and documented governance approach to guide and sustain security assurance, including establishing roles and responsibilities for the different cloud security functions, unified technical strategy, and supporting policies and standards.

ASVS (Application Security Verfication System)

Microsoft Defender for Cloud

https://azure.microsoft.com/services/defender-for-cloud

Microsoft Secure Score

Microsoft Secure Score

aka.ms/StoppingRealAttacks

Puma Scan

VIDEO: PumaScan [GitHub, Config] - $300 Pro for Visual Studio & VSCode (vsix), $5000 server, $6000 devops. Just 55 security rules.

Roslyn API - .NET Compiler Platform (aka “Roslyn”) exposes a set of code analysis APIs capable of querying the source code

https://github.com/OWASP/ASVS Controls:

  1. Architecture, Design, and Threat Modeling
  2. Authentication
  3. Session Management
  4. Access Control
  5. Validation, Sanitization, and Encoding
  6. Stored Cryptography
  7. Error Handling and Logging
  8. Data Protection
  9. Communication
  10. Malicious Code
  11. Business Logic
  12. Files and Resources
  13. API and Web Services
  14. Configuration

Cloud Security Roles

https://aka.ms/securityroles

Cloud security functions

Terraform

https://blog.devgenius.io/how-to-implement-azure-landing-zone-using-caf-terraform-part-2-4c5a06127d05

https://registry.terraform.io/modules/aztfmod/caf/azurerm/latest https://github.com/aztfmod/terraform-azurerm-caf Terraform supermodule for the CAF Terraform landing zones part of Microsoft Cloud Adoption Framework for Azure. It includes the list of all Azure resources definitions you can create within an Azure Landing Zone. Variables are used as configuration input, and the deployment is done accordingly. caf_azurerm also utilizes another module, azurecaf_name, to ensure that the resources all follow the same naming convention. In addition, caf_azurerm has a framework in place for tag inheritance to guarantee that all resources are appropriately marked.

https://github.com/Azure/terraform-azurerm-caf-enterprise-scale handle Azure at the organizational level. Management groups, subscriptions, access restrictions, policies, policy assignments, roles, and role assignments are all included. Use this module to arrange your Azure cloud resources and permissions and make sure that the necessary security is in place to stop malicious individuals from infiltrating your company.

Microsoft Learning Account

After getting a Learning account:

Microsoft’s Azure fundamentals class provides a learning path of 12 modules prepares you to pass the $99 for 50 questions over 60-minute AZ900 Microsoft Azure Fundamentals Exam taken at a testing center or at home with a video camera. (see LinuxAcademy video course released May 2019, include the “Book of Basics” interactive diagrams with tabs associated with major sections of the exam:

  • cloud concepts (15-20%)
    • Cloud Services: Benefits and Considerations
    • Infrastructure as a Service (IaaS),
    • Platform as a Service (PaaS),
    • Software as a Service (SaaS) - Salesforce & Office 365
    • Cloud Models: Public, Private, and Hybrid
  • core Azure services (30-35%)
    • Azure Architecture
    • Azure Products and Services
    • Azure Solutions
    • Azure Management Tools
  • security, privacy, compliance, and trust (20-35%)
    • Network Security in Azure
    • Azure Identity Services
    • Azure Security Tools and Features
    • Azure Governance
    • Monitoring and Reporting in Azure
    • Azure Privacy, Compliance, and Data Protection Standards
  • Azure pricing and support (20-35%)
    • Subscriptions
    • Planning and Managing Azure Costs
    • Support Options
    • Service Level Agreements (SLAs)
    • The Azure Service Lifecycle

Learn the business value of Microsoft Azure learning path

Azure Security Center is available in free and paid tiers. The Free subscription assesses Azure resources only. The “Standard” tier provides a full suite of security-related services including continuous monitoring, threat detection, just-in-time access control for ports, and more. After a free 60-day free trial, it’s $15 per node per month.

XP (Experience Points) levels in learning profiles grow by around 145% per level: LEVEL 1 - 1,799 XP

LEVEL 8 - 72,099 XP +145%
LEVEL 9 - 106,299 XP +147%
LEVEL 10 - 155,899 XP +147%
LEVEL 11 - 172,099 XP +110%
LEVEL 12 - 249,544 XP +145%
LEVEL 13 - 483,199 XP +145%
LEVEL 14 - 702,399 XP +145%
LEVEL 15 - 1,020,199 XP +145%
LEVEL 16 - 1,481,099 XP +145%
LEVEL 17 - 2,149,099 XP +145%
LEVEL 18 - 3,114,011 XP +145%

Architecting

Architect great solutions in Azure consists of 5 pillars, similar to the “Well Architected” series from AWS:

  • Design for security

  • Design for performance and scalability: Azure SQL Data Sync between regions. Azure SQL Database geo-replication allows for read-replicas. Azure Cosmos DB globally distributes NOSQL datab for reads and writes regardless of region. Azure Cache for Redis to minimize high-latency calls to remote databases to read frequently accessed data. Polyglot persistence to use different storage technologies for different data.

  • Design for efficiency and operations

  • Design for availability and recoverability

Based on: Pillars of a great Azure architecture

Cloud scale analytics:

azure-dataw-648x239-10988

TODO: Incorporate into Azure IAM page:

Azure account and dashboard

ARM Create instance

  1. At the Azure portal:

    https://portal.azure.com

    Regions & Affinity Groups

  2. Select Resource group physical and logical network-isolated instances of Azure / Regions)

    • In the Americas: westus2, centralus, southcentralus, eastus, brazilsouth
    • In Europe: westeurope (there’s also France Central and North Europe)
    • In Asia Pacific: southeastasia, japaneast, australiasoutheast, centralindia
    • In Middle East and Africa

    PROTIP: Bolded are the only regions that support Availability Zones: Central US, North Europe, and SouthEast Asia.

    NOTE: Some services or virtual machine features are only available in certain regions, such as specific virtual machine sizes or storage types.

    Additionally, Azure has specialized regions for compliance or legal purposes:

    • US DoD Central, US Gov Virginia, US Gov Iowa, and more are for US government agencies and partners. These datacenters are operated by screened US persons and include additional compliance certifications.

    • China East, China North and more: These regions are available through a unique partnership between Microsoft and 21Vianet, whereby Microsoft does not directly maintain the datacenters.

    Each region is paired with another region (West US paired with East US, and SouthEast Asia paired with East Asia, etc.). Such Region pairs are at least 300 miles apart.

    A regional Affinity Group is defined to create a virtual network to define the data center (region). All services within an affinity group are located in the same data center. Azure groups services use Affinity Groups to optimize performance.

    WARNING: Affinity groups in Azure is a higher-level concept of data centers than the facility of the same name within AWS, which refers to affinity between servers on the same subnet.

    Availability Zones are specified for VMs, managed disks, load balancers, and SQL databases. AZs are physically separate datacenters within an Azure region. Each Availability Zone is made up of one or more datacenters equipped with power, cooling, and networking independent of other AZs so that each is set up to be an isolation boundary. If one zone goes down, the other continues working. Availability Zones are connected through high-speed, private fiber-optic networks.

    Network Security Groups (NSGs) inside a virtual network (VNet) are defined for communication between virtual machines to restrict unnecessary communication.

  3. Options include the classic ASM (Azure Service Manager) and newer ARM (Azure Resource Manager):

    • Apps Services
    • Virtual machines (classic)
    • Virtual machines
    • SQL databases
    • Cloud services (classic)

    • Security Center

    • Active Directory
    • Storage
    • Messaging
    • Networking
    • Management

    Each drill-down into ARM creates an additional pane? to the right.

  • Create a virtual machine (for 60 minutes). PROTIP: Use Firefox browser. Don’t use Brave browser.

    azure-cloud-shell-364x199-11637

    PROTIP: naming conventions:

    • Size
    • Region
    • Network
    • Resource groups

  • Get VM information with queries

    az vm show \
--resource-group 7f3943f2-f179-42ba-9823-ba71c7ba7824 \
--name myVM \
--query "hardwareProfile" \
--output tsv
  • Set environment variables from CLI output
  • Creating a new VM on the existing subnet
  • Cleanup

https://docs.microsoft.com/en-us/learn/modules/design-for-efficiency-and-operations-in-azure/2-maximize-efficiency-of-cloud-spend

Module: Store Data in Azure

An example of imperative declaration:

az group create --name storage-resource-group \
        --location eastus
az storage account create --name mystorageaccount \
        --resource-group storage-resource-group \
        --kind BlobStorage \
        --access-tier hot

Declarative automation is done using Azure Resource Manager templates such as this:

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "name": {
            "type": "string"
        },
        "location": {
            "type": "string"
        },
        "accountType": {
            "type": "string",
            "defaultValue": "Standard_RAGRS"
        },
        "kind": {
            "type": "string"
        },
        "accessTier": {
            "type": "string"
        },
        "httpsTrafficOnlyEnabled": {
            "type": "bool",
            "defaultValue": true
        }
    },
    "variables": {
    },
    "resources": [
        {
            "apiVersion": "2018-02-01",
            "name": "[parameters('name')]",
            "location": "[parameters('location')]",
            "type": "Microsoft.Storage/storageAccounts",
            "sku": {
                "name": "[parameters('accountType')]"
            },
            "kind": "[parameters('kind')]",
            "properties": {
                "supportsHttpsTrafficOnly": "[parameters('httpsTrafficOnlyEnabled')]",
                "accessTier": "[parameters('accessTier')]",
                "encryption": {
                    "services": {
                        "blob": {
                            "enabled": true
                        },
                        "file": {
                            "enabled": true
                        }
                    },
                    "keySource": "Microsoft.Storage"
                }
            },
            "dependsOn": []
        }
    ],
    "outputs": {
        "storageAccountName": {
            "type": "string",
            "value": "[parameters('name')]"
        }
    }
}

Module: Deploy a website to Azure with Azure App Service

azure-loadbal-615x424-31664

Azure Traffic Manager provides global DNS load balancing among DNS endpoints within or across Azure regions. Traffic manager also detects and removes failed endpoints.

Azure Application Gateway (AppGW) provides Layer 7 (URL-based) load-balancing such as round-robin distribution of incoming traffic, cookie-based session affinity, URL path-based routing, and the ability to host multiple websites behind a single application gateway. Application Gateway monitors the health of resources in its back-end pool and automatically removes any resource considered unhealthy from the pool. Health probes continue until instances are healthy again and added back.

Azure Load Balancer is a layer 4 load balancer. TCP and HTTP health-probing options to manage service availability are optional.

“Availability sets”

https://docs.microsoft.com/en-us/learn/modules/explore-azure-infrastructure/ Core Cloud Services - Azure architecture and service guarantees

From Learn Path: Administer containers in Azure

QUESTION: How to use https://raw.githubusercontent.com/wilsonmar/Dockerfiles/master/azure-node/Dockerfile

https://docs.microsoft.com/en-us/learn/modules/run-docker-with-azure-container-instances/ Azure Container Instances (ACI).

Container restart policies:

  • Always restart for long-running tasks such as a web server, so it’s the default.
  • Never for run one-time only.
  • OnFailure only when the process short-lived tasks terminates with a nonzero exit code.

Installers

Commands

https://docs.microsoft.com/en-us/learn/modules/welcome-to-azure/4-create-a-vm?pivots=linux-cloud

Install Commands

Each resource group defines scope access control for administrative actions.

Tags are used for all other organization of resources.

Azure PowerShell Login

Login-AzureRmAccount

Type your credentials and press OK.

A sample response:

   Environment           : AzureCloud
   Account               : ???@hotmail.com
   TenantId              : ????????-5f96-4d36-a89b-5ea0f7614e72
   SubscriptionId        : ????????-cf54-443f-b0f1-bcc5e78e9c27
   CurrentStorageAccount :

Azure Resource Groups

Every resource is in only one group, listed here by stack:

  • Web Apps
  • SQL
  • Storage
  • VMs
  • NICs
  • Virtual Networks

A resource group can contain resources residing in different regions.

Get-AzureRmResourceProvider

Azure Container Service (ACS)

Microsoft created and maintains the Azure Container Service with Mesosphere.com

with standard Docker tooling and API.

Streamlined provisioning of DC/OS Clusters

and Docker Swarm support

Mesos-DNS for service discovery and registration (no health checks)

DC/OS Marathon load balancer support of dcos cli commands needs to be installed. Backed up as a HA Proxy.

“Minuteman” provides virtual IPs stored in IP tables synced across the cluster.

Azure Service Fabric


Azure Service Fabric enables you to talk to a cluster of machines as if they were one.

An Azure Service Fabric agent runs on each machine – in Amazon or private cloud as well.

  • One call to manage capacity (add and remove nodes at will)
  • Service endpoint discovery
  • Create (immutable) containers

  • Deploy software to containers

  • health reporting
  • Monitoring based on queue length
  • Dynamic resource balancing based on actual resource usage (queue length)

  • Move resources from one node to another

  • coordinate upgrades (select what node to upgrade)
  • Diagnostics in F5

Different services can run on the same machine.

Azure Service Fabric offers a substitute for external storage via its Reliable Collections programming model accessing dictionary entries.

Data Import/Export Jobs Service

VIDEO

Azure Import/Export service can uses physical drives to import into Azure Blog Storage or Azure Files.

  1. WaImportExportV2.exe for files (v1 for blobs), BitLocker encrypt
  • Data Box Gateway virtual appliance
  • Data Box Edge to Azure IoT Edge
  • Data Box Offline (Robocopy)
    • Data Box Disk - 8 TB SSD x 5 packs (128 AES encrypted)
    • Data Box - 100 TB AES 256
    • Data Box Heavy - 1TB ruggedized

Azure Jobs

CDN Endpoints

Resources

Overview Videos

Extentions Marketplace

VIDEO: Windows Azure Marketplace by Joe Kunk Intermediate Dec 19, 2013 1h 56m

https://marketplace.visualstudio.com/azuredevops?noPrompt=true

  1. If you get a pop-up:

  2. Click to be brought to
    https://marketplace.visualstudio.com/items?itemName=ms.vss-code-search

  3. Click the green “Get it free” button.
  4. Select the organization and click “Install”.
  5. Click “Proceed to organization”.

QUESTION: How to automate the above installation on an org?

  1. https://docs.microsoft.com/en-us/azure/devops/project/search/overview?view=azure-devops
  2. https://docs.microsoft.com/en-us/azure/devops/project/search/work-item-search?view=azure-devops
  3. https://docs.microsoft.com/en-us/azure/devops/project/wiki/search-wiki?view=azure-devops

Operations:

Dev

Data:

Mobile:

ACTUAL MONTHLY UPTIME % SERVICE CREDIT PERCENTAGE
< 99.9 10
99 25
95 100

More on DevOps

This is one of a series on DevOps:

  1. DevOps_2.0
  2. ci-cd (Continuous Integration and Continuous Delivery)
  3. User Stories for DevOps

  4. Enterprise Software)

  5. Git and GitHub vs File Archival
  6. Git Commands and Statuses
  7. Git Commit, Tag, Push
  8. Git Utilities
  9. Data Security GitHub
  10. GitHub API

  11. TFS vs. GitHub

  12. Choices for DevOps Technologies
  13. Pulumi Infrastructure as Code (IaC)
  14. Java DevOps Workflow

  15. Okta for SSO & MFA

  16. AWS DevOps (CodeCommit, CodePipeline, CodeDeploy)
  17. AWS server deployment options

  18. AWS Load Balancers

  19. Cloud services comparisons (across vendors)
  20. Cloud regions (across vendors)

  21. AWS Virtual Private Cloud

  22. Azure Cloud Onramp (Subscriptions, Portal GUI, CLI)
  23. Azure Certifications

  24. Azure Cloud

  25. Azure Cloud Powershell
  26. Bash Windows using Microsoft’s WSL (Windows Subsystem for Linux)

  27. Azure KSQL (Kusto Query Language) for Azure Monitor, etc.

  28. Azure Networking
  29. Azure Storage
  30. Azure Compute

  31. Azure Monitoring

  32. Digital Ocean

  33. Cloud Foundry

  34. Packer automation to build Vagrant images
  35. Terraform multi-cloud provisioning automation

  36. Hashicorp Vault and Consul to generate and hold secrets

  37. Powershell Ecosystem
  38. Powershell on MacOS

  39. Powershell Desired System Configuration

  40. Jenkins Server Setup
  41. Jenkins Plug-ins
  42. Jenkins Freestyle jobs

  43. Jenkins2 Pipeline jobs using Groovy code in Jenkinsfile

  44. Docker (Glossary, Ecosystem, Certification)
  45. Make Makefile for Docker
  46. Docker Setup and run Bash shell script
  47. Bash coding
  48. Docker Setup
  49. Dockerize apps

  50. Docker Registry

  51. Maven on MacOSX

  52. Ansible
  53. Kubernetes Operators

  54. OPA (Open Policy Agent) in Rego language

  55. MySQL Setup

  56. Threat Modeling

  57. SonarQube & SonarSource static code scan

  58. API Management Microsoft

  59. API Management Amazon

  60. Scenarios for load
  61. Chaos Engineering

Azure Cloud was published on .