Wilson Mar bio photo

Wilson Mar

Hello. Hire me!

Email me Calendar Skype call 310 320-7878

LinkedIn Twitter Gitter Instagram Youtube

Github Stackoverflow Pinterest

Take my cloud … please

Español (Spanish)   Français (French)   Deutsch (German)   Italiano   Português   Cyrillic Russian   中文 (简体) Chinese (Simplified)   日本語 Japanese   한국어 Korean

Overview

Why? Advantages of Cloud

Evaluations can be based on the advantages of cloud:

  1. Stop guessing about future capacity when building and maintaining on-premise data centers with idle over-provisioning, which can be expensive.

  2. Tap into more capacity pooled among many customers,to benefit from economies of scale.

  3. Paying for public cloud use as you go conserves cash up-front which trades variable expense (OpEx) for capital expense (CapEx).

  4. Faster to obtain capacity and achieve agility. Cloud vendors can buy newer, faster equipment.

  5. Get expert specialists at facilities, security, scalability, etc. by hiring them through cloud vendors who share their innovations.

  6. Cloud customers’ workers achieve higher productivity from more efficient up-to-date methods and by off-loading tasks to cloud specialists.

  7. Greater reliability to make data backup, disaster recovery, and business continuity easier and less expensive, with data being mirrored at multiple redundant sites on the cloud provider’s network.

  8. Go global - make use of compute and network infrastructure established around the world (time consuming, expensive, and dangerous to do on one’s own).

PROTIP: Memorize the above list for exams.

Amazon’s “Well Architected” notes the constraints removed by being in the cloud:

  • Test production at scale
  • Make experimentation easier (overcome fear of change)
  • Allow architecture to evolve (rather than being frozen in time)

Despite all the good reasons, organization may still “drag their heels” until …

Triggers to go cloud

The need to:

  • Rehost (migration) (“lift-and-shift”) to rehost with little code changes, to minimize application downtime.

  • Refactor (modernization) - transform existing apps with minimal code change to use Docker and Kubernetes, robust logging, “elastic” auto-scaling in & out, etc.

  • Rearchitect (modernization) - isolate application functionalities into separate microservices that can be independently deployed, tested, scaled up or down, and managed.

  • Rebuild (modernization) - reconstruct existing app on a new PaaS platform.

  • Replace (modernization) - retire existing on-premises application and rewriting functionality under a new architecture such as a Serverless or SaaS (“cloud native”) application.

Individuals must pick a single cloud?

Organizations are going multi-cloud. But an individual has a limited amount of time to keep up with changes. So many individual technologists need to select the cloud vendor which provides the best career prospects. Here’s my logic:

  • If you code in C# on .NET and want to continue leveraging that experience, Azure is the natural choice. (But other clouds are trying to support Windows.)

  • Azure now provides free learning instances

  • If you’re looking for the fastest and easiest path to get a job in cloud, go for AWS because of its current market share and pace of innovation. AWS has the most sophisticated authentication and database services.

  • Due to competitive reasons, Target, Best Buy, eBay, and Sony Music use Google. WalMart uses Azure. Since Amazon is moving into agriculture (building urban farms to provide Whole Foods and Amazong Go stores) plus health care, established competitors in those industries would not want to use AWS.

  • If you’re running really large loads on each server, Google offers fast fiber networks that don’t limit what large machines can achieve. Plus Google is ahead in Machine Learning with its popular Tensorflow in Python.

  • Google’s SSD drives are expensive, though needed for speed.

  • IBM has bare-metal machines, if that’s your thing. But since 2017 AWS provides them as well.

  • Increasingly, companies doing business in a particular country are required to keep data within a cloud data center within that country (such as Google in Belgium).

Cloud vendor comparisons

cloud platforms black icons 300x330-300-58kb.jpg

This article compares the largest cloud services platforms: Amazon AWS vs. Microsoft Azure vs. Google Cloud Platform vs. IBM Softlayer vs. Rackspace vs. Alibaba Cloud vs. Huawei Cloud.

Major multi-cloud vendors include Oracle, SalesForce’s Heroku, SkyTap, etc.

Software for private clouds include Red Hat OpenStack and VMWare.

Software for hybrid private/public clouds include CloudFoundry, Hashicorp Terraform, and others. NetApp, or EMC

EDITOR’s NOTE: This is not a complete treatment. Additional information will be added over time.

cloud-compare-364x172 http://cloudcomparison.rightscale.com presents all the details well. DOCS So here I’m adding additional commentary here.

Additionally…

Factor AWS Azure
Company Amazon Microsoft
Initial appearance 2006 2010
Granularity of billing per hour per minute
Entrepreneurial Discount - BizSpark
# Data centers 24 100
# zones 33 100
Largest server x1.32xlarge G:
Largest # cores 128 32
Largest RAM GB 1,952 GB 448 GB

PROTIP: Moving up to double the RAM or cores usually does not yield a doubling of capacity due to overhead and limits in shared components such as networking.

Pricing

Even though cloud vendors provide a great deal of transparency to how they price their services, there are so many variables to what affects the final bill that the only accurate way is to actually run services.

  • AWS has matched Azure pricing, and also per-minute billing.

Cowan analysis May 2016 (using 1.0 as average among vendors) rated Amazon 1.37 (above avarge) in prices (most expensive). Microsoft was rated the leader in price and APIs, but the lowest in support.

QUESTION: Is the comparison based on spot instances in AWS, which are of low cost because they can be taken away at any moment by someone who outbids.

Market share

Amazon had an early lead in 2006 and has been maintaing its lead even though Microsoft and Google are growing fast as well with 50%+ annual growth* aws-onboarding-cloud-growth-441x272.jpg

At the end of 2018, corporate filings reavealed this:
cloud-run-rate-2018.png

Earlier in 2015:

Operating Systems aliances

As of May 2016, Microsoft Azure works with SUSE and Oracle Java, but NOT Red Hat Enterprise Linux (RHEL).

Amazon provides software to host whatever OS is put into images. But its free server images are based on CentOS, derived from Red Hat as both use the yum package manager. Amazon is working on “Linux2” based on Red Hat.

Network infrastructure

Google has laid its own fiber-optic wires around the world:

google-cables-1533x861.png

As of this writing, latency between the EU and Asia locations is very high because Google Compute Engine does not have a direct link between them.

PROTIP: If you only operate out of just one region in the world, use a Central US region, which reaches West to Asia and East to Europe.

Network speed

Under ideal conditions, latency over networks are limited by speed of light traveling within fiber-optic cables, which is roughly 202562 km/s or or 125866 miles/s. (Actual reachable speed is still a bit lower than that).

To estimate distance between terrestial locations, use Distance Calculator website

To compare network speeds:

  • Amazon’s own EC2 Reachability Test does not provide ping speeds to specific IP addresses within each availability zone within each region, but a green icon when it can be currently reached at any speed.

  • http://cloudping.info measures current ping speed to various regions on Amazon’s EC2 cloud. Scroll down to click “HTTP Ping” to begin collection to a running line graph:

    cloud-services-amazon-392x647-65305

  • https://ping.varunagw.com/aws.htm provides green, yellow, red colors with Mean, Median, Min, and Max statistics based several measures of the speed to each EC2 region.

  • http://http://www.azurespeed.com measures current ping speed to various regions on Microsoft’s Azure cloud. To stop collection, click “uncheck all” or individual regions of interest:

    cloud-service-azure-checks-603x261-56530

  • http://www.gcping.com measures current ping speed to regions on Google’s cloud.

    cloud-services-gcping-507x224-14208

    To stop collection, press the dot with the arrow.

    Qwiklabs Network Performance Testing provides step-by-step instructions for obtaining timings between regions. It sets up one network with five subnetworks in different regions and a VM in each subnetwork. Utilities ping, iperf. Alternately, the open-sourced Linux/Unix utility mtr (My Traceroute, originally called Matt’s traceroute) combines Ping and Traceroute functions* brew install mtr is available for macOS to sudo mtr.

Outage tracking

https://status.aws.amazon.com is Amazon’s AWS Service Health Dashboard applicable to all users. Under the tab for each continent is a list of each service plus region combination.

PROTIP: Most new services begin in the N. Virginia region “US-EAST-1”, as do the most famous outages. So if you are running a production load, try to use a different region than that. Nevertheless, that’s where one must provision AWS Cloud Front CDN for worldwide distribution.

https://phd.aws.amazon.com is your Personal Health Dashboard for your account.

http://downdetector.com/status/aws-amazon-web-services provides 3rd-party crowd-source status.

https://istheservicedown.com/problems/amazon-web-services-aws/history is a private-party site reporting the history of downtimes on AWS. For example:

cloud-services-aws-down-608x190-19517

For a list of outages further back: https://outage.report/aws-amazon-web-services

Categories of cloud services:

Service brand names

Here are the names of brand names, some with links to marketing or documentation pages:

Feature:Microsoft AzureAmazon Web Services (AWS)Google
Data center geography Azure Regions global infrastructure -
Government ServicesAzure GovernmentGovCloud -
Security:
Administration & SecurityAzure Active Directory DirectoryService
IAM (Identity and Access Management)
-
Threat Monitoring
Azure Security Center AWS Config -
Multi-Factor Authentication
MFA MFA -
Encryption Key Mgmt.Azure Key Vault KMS (Key Management Service) -
CAAzure Cloud Services Certificate Manager -
Hardware Security ModuleAzure Trust Center CloudHSM -
DDoS protectionAzure DDoS Protection Shield Advanced -
Share data securityInformation Protection ? -
Networks:
Content Delivery Network (CDN )Azure CDNCloudFront -
Load BalancingLoad Balancing for Azure (how to)ELB (Elastic Load Balancing) -
Region routing by latency or on failAzure Traffic ManagerAmazon Route 53 -
Networking OptionsAzure Virtual Network VPC -

Azure ExpressRouteAWS Direct Connect -
VPN GatewayAzure Virtual Network Virtual Private Cloud & Gateway (VPG) -
Management:
What services?Azure Managed Applications AWS Service Catalog -
BillingAzure Cost ManagementBilling -
Monitoring, Logging, AlertingAzure Application Insights CloudWatch -

Azure Event Hubs None
-
Audit logs
Azure Operational Insights CloudTrail -
Zipkin tracingAzure Application Monitor InsightsAWS X-Ray -
-Ops. Mgmt. Suite (OMS)Systems Manager -
Compute:
App/Desktop ServicesAzure RemoteAppWorkSpaces
Amazon AppStream
-
VPSLinux VMsAmazon Lightsail
-
Compute ServicesVirtual Machines (VMs) EC2 (Elastic Compute Cloud) -
DockerHub Azure Container Registry AWS Lambda -
Triggers Logic Apps Elasticc Container Registry (ECR) -
Container SupportDocker Virtual Machine Extension (how to)EC2 Container Service (ECS) Kubernetes
Kubernetes scalingAzure Kubernetes Service (AKS)ECS for Kubernetes (EKS) -
Distributed appsAzure Service Fabric- -
Scaling serviceAzure Container InstancesAWS Fargate -
Integration:
AnalyticsAzure Stream Analytics Kinesis -
MessagingAzure Notification Hubs SNS (Simple Notification Service) -
Email ServicesAzure BizTalk Services SES (Simple Email Services) -
API GatewayAzure API Management Amazon API Gateway Apigee

Azure Service Bus SQS (Simple Queue Service) -

Azure Batch Amazon Batcch -
CalendarAzure Scheduler SWF (Simple Workflow) -
DevOps:
Cloud IDEVisual Studio App Center Cloud9 -

Cloud Services
Azure Websites and Apps
Elastic Beanstalk -
Dev. toolsAzure Visual Studio OnlineNone -
Management Services & OptionsAzure Resource Manager CloudFormation -
Chef Automation- OpsWorks -
GitHubAzure Repos CodeCommit -
JenkinsAzure Code Pipelines CodeBuild -

?
CodeDeploy -
Media ServicesAzure Media Services ElasticTranscoder
MobileAnalytics
Cognitor
-
Persistance:
Storage OptionsAzure Storage (Blobs, Tables, Queues, Files)S3 (SimpleStorage) -
Block StorageAzure Storage Disk(how to)Elastic Block Storage (EBS) -
Hybrid Cloud StorageStorSimpleNone -
Shared File StorageAzure FilesElastic File System (EFS) -
Backup OptionsAzure BackupGlacier Coldline
Storage ServicesAzure Import Export (how to)ImportExport -

Azure File Storage (how to)StorageGateway -

Azure Site Recovery? -
Database OptionsAzure SQL Database RDS (Relational Database Service)
Redshift
-
NoSQL Database OptionsAzure DocumentDB DynamoDB BigTable

Azure Managed Cache (Redis Cache) ElasticCache -
Scaling OptionsAzure Autoscaling (how to) Machine Scale SetsAWS AutoScaling Group -
Data OrchestrationAzure Data Factory DataPipeline -
Analytics:
VisualizationPowerBIQuickSight -
Data DiscoveryData Catalog? -
Analytics/Hadoop OptionsHDInsight (Hadoop)Elastic MapReduce (EMR) -
ElasticSearchin Marketplace ElasticSearch -
SearchAzure Search CloudSearch -
Machine Learning (ML)Azure Machine Learning Machine-Learning SageMaker Tensor Flow
TranslationTranslator Text Amazon Translate Google Translate
Voice RecognitionAzure Speaker Recognition Rekognition image and video -
Computer VisionComputer VisionFace Rekognition -
Video camera? $249 DeepLens -

Credit: TomsITPro and Microsoft’s comparison


Resources

IBM Bluemix

“Understanding the Difference Between Microsoft Azure and Amazon AWS” by veteran Pluralsight video presenter Elias Khanser, a Microsoft MVP

https://cloudonaut.io/my-mental-model-of-aws/

More on DevOps

This is one of a series on DevOps:

  1. DevOps_2.0
  2. ci-cd (Continuous Integration and Continuous Delivery)
  3. User Stories for DevOps

  4. Git and GitHub vs File Archival
  5. Git Commands and Statuses
  6. Git Commit, Tag, Push
  7. Git Utilities
  8. Data Security GitHub
  9. GitHub API
  10. TFS vs. GitHub

  11. Choices for DevOps Technologies
  12. Java DevOps Workflow
  13. AWS DevOps (CodeCommit, CodePipeline, CodeDeploy)
  14. AWS server deployment options

  15. Cloud services comparisons (across vendors)
  16. Cloud regions (across vendors)
  17. AWS Virtual Private Cloud

  18. Azure Cloud Onramp
  19. Azure Cloud
  20. Azure Cloud Powershell
  21. Bash Windows using Microsoft’s WSL (Windows Subystem for Linux)

  22. Digital Ocean
  23. Cloud Foundry

  24. Packer automation to build Vagrant images
  25. Terraform multi-cloud provisioning automation
  26. Hashicorp Vault and Consul to generate and hold secrets

  27. Powershell Ecosystem
  28. Powershell on MacOS
  29. Powershell Desired System Configuration

  30. Jenkins Server Setup
  31. Jenkins Plug-ins
  32. Jenkins Freestyle jobs
  33. Jenkins2 Pipeline jobs using Groovy code in Jenkinsfile

  34. Docker (Glossary, Ecosystem, Certification)
  35. Docker Setup
  36. Dockerize apps

  37. Maven on MacOSX

  38. Ansible

  39. MySQL Setup

  40. SonarQube static code scan

  41. API Management Microsoft
  42. API Management Amazon

  43. Scenarios for load

More on cloud

This is one of a series on cloud computing: