Dynamically audit AWS resource configurations under organization units using OPA Rego, Lambda alerts using SNS through EventBridge, then automatically remediate (via SSM)
Overview
- Global Admins using Control Tower on Organizations
- AWS Services
- Setup AWS Config
- AWS Config Timeline GUI
- Managed Rules for AWS Config
- Remove unattached Security Groups automatically
- Custom Rules for AWS Config
- Simple Notification Service
- RDK (AWS Config Rule Development Kit)
- Chef Inspec Checks
- Troubleshooting
- Social
- References
AWS Config provides a detailed view of the configuration of AWS resources in your AWS account. This includes how the resources are related to one another and how they were configured in the past to show how configurations and relationships change over time.
NOTE: Content here are my personal opinions, and not intended to represent any employer (past or present). “PROTIP:” here highlight information I haven’t seen elsewhere on the internet because it is hard-won, little-know but significant facts based on my personal research and experience.
Global Admins using Control Tower on Organizations
“AWS Control Tower” is a part of AWS Organizations managed by corporate-level Global Administrators. Those administrators use the AWS Control Tower’s centralized dashboard to view AWS Config rule compliance status across Organization Units used to manage AWS accounts.
VIDEO: AWS Control Tower Overview and Landing Zone Hands-On
PROTIP: The more aspects of Well-Architected cloud usage covered to benefit, the more likely people will buy into the work here.
References:
AWS Services
AWS Config has both AWS-managed and custom rules focusing on Cost Optimization as well as Security.
“AWS Security Hub” use only AWS-managed rules and focus only on Security issues.
“AWS Audit Manager” uses compliance evaluation data produced by AWS Config to simplify the process of collecting audit evidence.
While AWS Config focuses on changes in the configuration of resources,
AWS GuardDuty SaaS (released 2017) dynamically audits activities on running resources to continuously detect (at scale) threats using anomaly detection algorithms generated by AWS Machine Learning.
Amazon Detective (new in 2020) VIDEO: helps to manually triage potential security issues identified by AWS GuardDuty and AWS Security Hub. Use it for “Threat Hunting” – for the “Security Graph” to visualize Virtual Private Cloud (VPC) Flow Logs, AWS CloudTrail (much like how SOC uses competitor Splunk).
Setup AWS Config
Resources:
- VIDEO DEMO of setup
- BLOG: 1
2
To enable AWS Config, this CloudFormation template creates components: KMS encryption key, an encrypted S3 bucket, etc.
-
AWS Config can be configured and managed using the AWS Management Console GUI or AWS CLI.
When the AWS Config recorder is enabled for an account, it detects each change in AWS resource configurations and maintains a historical record of them. Behind the scenes, it invokes the Describe or List API calls for each resource in each account.
For example, removing an egress rule from a VPC security group causes AWS Config to invoke a Describe API call on the security group. AWS Config then invokes a Describe API call on all of the instances associated with the security group.
To automated removal of Security Groups:
- https://www.youtube.com/watch?v=eNXk9VA00_s video on
- https://github.com/miztiik/serverless-janitor-for-security-groups
-
In the AWS Management Console GUI, sign-on to an AWS account.
-
Search for “Config” service.
AWS Config is enabled for each by default. ???
-
Customize the resource types AWS Config tracks.
And AWS Config may not cover all AWS services.
The updated configurations of the security group (the resource) and of each instance (the related resources) are recorded as configuration items and delivered in a configuration stream to an Amazon Simple Storage Service (Amazon S3) bucket.
DEFINITION: All CI’s for a given resource is called a “configuration history” in S3.
-
Define the S3 bucket.
- In the Amazon S3 management console, choose Create bucket.
- Enter a name for the bucket and select an AWS Region.
- In Block public access settings for this bucket, make sure Block all public access is selected to protect the data from being exposed to the internet.
- In Default encryption, choose Enable to encrypt the data on the bucket. You can leave the default option of Amazon S3 key (SSE-S3) It is a best practice to encrypt data on S3 buckets.
- Choose Create bucket.
-
Enable recording
AWS Config also examines resource configurations periodically to generate configuration items for the configurations that have changed.
When several configuration changes are made to the same resource too quickly, AWS Config will only record the latest configuration of that resource; this represents the cumulative impact of that entire set of changes.
-
Set retention period.
Changes triggers a Lambda function to evaluate AWS resource configurations based on rules defining the evaluation logic for the rule and desired settings.
Each rule determines whether the resource as COMPLIANT or NON_COMPLIANT in a JSON file returned in a Policy S3 bucket.
When the compliance status of a resource changes, AWS Config sends a notification to your Amazon SNS topic.
-
Set SNS topic for notifications.
-
Set Config role: “Grant AWS Config read-only access to your AWS resources so that it can record configuration information, and grant it permission to send the information to Amazon S3 and Amazon SNS”: Create AWS Config service-linked role.
AWS Config Timeline GUI
AWS Config gives point-in-time and historical states and allows user to see changes visually in a timeline
Managed Rules for AWS Config
By default AWS Config provides managed rules that address the most common use cases for evaluating compliance. Over 200 such rules are described at https://docs.aws.amazon.com/config/latest/developerguide/managed-rules-by-aws-config.html (QUESTION: Where is the code? case 9177818611)
People at Lending Tree found these AWS-manged Config rules helpful in detection:
-
ec2-security-group-attached-to-eni returns NON_COMPLIANT for security groups not associated with an Amazon EC2 instance or an elastic network interface.
-
restricted-common-ports returns COMPLIANT when IP addresses of incoming SSH traffic in the security group are restricted to specified ports (restricts incoming TCP traffic to specified ports).
-
restricted-ssh returns COMPLIANT when IP addresses of the incoming SSH traffic in the security groups are restricted.
-
vpc-default-security-group-closed returns COMPLIANT if the default security group of any Amazon Virtual Private Cloud (VPC) has one or more inbound or outbound traffic rules (). NOT_APPLICABLE is returned if the security group is not default.
Notice AWS’s managed rules are mostly about security, and less about cost optimization, Reliability, and Performance.
STAR: Trend Micro’s product “Conformity” provides 750+ rules as part of their “CloudOne” Security Platform:
Cost Optimization:
- Remove Elastic IP (EIP) addresses not associated with EC2 (which AWS charges for).
-
Remove unused Amazon Machine Images (AMIs) to reduce storage costs.
-
Measure justification for premium cost when using a high-priced region.
-
Ensure AWS servers use the most price-performant server type (such as Graviton)
- Stop or terminate idle AWS EC2 instances
-
Identify underutilized EC2 instances and downsize them
- Ensure EC2 Reserved Instance purchases are regularly reviewed for actual usage.
- Regularaly review EC2 dedicated instances are regularly reviewed for actual usage.
Security:
-
Restrict unrestricted access to RPC, RDP, SMTP, Telnet, Memcached, Redis, SG, CIFS, DNS, FTP, HTTP, ICMP, NetBIOS, PostgreSQL, MongoDB, MySQL, Oracle (in addition to AWS-managed SSH access)
-
Ensure no EC2 security group allows inbound traffic from RFC-1918 CIDRs in order to follow AWS security best practices.
- Enforce AMI naming conventions and using only allowed AMIs not on Blacklisted AMIs.
- Ensure that existing AMIs are encrypted to meet security and compliance requirements.
- Ensure custom Amazon Machine Images (AMIs) are not accessible to public AWS accounts.
Reliability:
- Ensure that the account does not reach the limit set by AWS for the number of Elastic IPs.
- Upgrade overutilized EC2 instances to optimize application response time.
- Ensure that Hibernation feature is enabled for EBS-backed EC2 instances to retain memory state across instance stop/start cycles.
-
Ensure every EC2 instance is launched inside an Auto Scaling Group (ASG) in order to follow AWS reliability and security best practices.
- Check for running AWS EC2 instances older than 180 days available within your AWS account.
- Check for any AMIs older than 180 days available within your AWS account.
Operational Excellence:
- Ensure AWS EC2 security group rules have descriptive text for organization and documentation.
- Ensure that detailed monitoring is enabled for the AWS EC2 instances that you need to monitor closely.
- Ensure that no AWS EC2 Reserved Instance purchases are pending or failed.
Remove unattached Security Groups automatically
VIDEO: To automatically remove Security Groups not attached to AWS resources (EC2, etc.) using Lambda and CloudFormation https://github.com/miztiik/dev-sec-ops”> AWS Advanced Security: SecOps Automation for the cloud
Custom Rules for AWS Config
But there are possibilities to save money, improve performance, etc.
AWS Config also supports custom rules that allow you to define your own logic by using AWS Lambda and one of the programming languages supported by AWS Lambda.
- Cost Optimization: Identify when two EC2 nodes within the same AZ communicate with each other via a public port, because network charges would accrue, unlike when EC2 nodes communicate via private lines.
AWS Config Conformance packs
Since AWS Config is a SaaS service, it can centralize the collection of compliance data across a whole organization.
A collection of AWS Config rules and remediation actions can be packaged into a single entity (known as a “Conformance Pack”) for organization-wide deployment. Thus, Conformance Packs are integrated with AWS Organizations. This is particularly useful to quickly establish a common baseline for resource configuration policies and best practices across multiple accounts in your organization in a scalable and efficient way.
“A conformance pack is a collection of AWS Config rules and remediation actions that can be easily deployed as a single entity in an account and a Region or across an organization in AWS Organizations.”
The sample Conformance Pack yaml file at https://github.com/aws-samples/aws-management-and-governance-samples/blob/master/AWSConfig/ConformancePacks/CP-Prerequisites.yaml is a quick start cloudformation template for AWS Config Conformance packs. It creates resources to start using Conformance Packs in one AWS Account.
https://docs.aws.amazon.com/config/latest/developerguide/conformance-packs.html
The sample conformace pack yaml file at https://github.com/aws-samples/aws-management-and-governance-samples/blob/master/AWSConfig/ConformancePacks/CP-IAMBestPractices.yaml
- Parameters:
- Under Resources are defined Properties referencing Parameters
BLOG: Sample config rules in Conformace Packs in GitHub
Simple Notification Service
VIDEO: AWS SQS vs SNS vs EventBridge - When to Use What?
RDK (AWS Config Rule Development Kit)
The AWS Config Rule Development Kit (RDK) enables rapid deployment of custom AWS Config rules within a single AWS account.
RDK uses Python with the AWS Boto3 library to make calls to AWS. STAR: See https://rdk.readthedocs.io/en/latest/getting_started.html
Multi-region Aggregators
- Get More Out of AWS Config by Using Multi-Account, Multi-Region Advanced Queries May 20, 2020 by Archana Sridhar, AWS Config PM
VIDEO DEMO: For no additional charge, an Aggregator aggregates (combines) rules and configuration history across regions in order to present an aggregated view.
AWS Config RDK: Multi-account and multi-Region shows deployment at scale across multiple AWS accounts and Regions. With multiple regions, also needed is a centralized dashboard to view AWS Config rule compliance status across various AWS accounts.
RDK
AWS Config with RDK (Rule Development Kit) within AWS Organizations Control Tower.
- https://github.com/awslabs/aws-config-rdk
Chef Inspec Checks
https://aws.amazon.com/blogs/mt/setting-up-custom-aws-config-rule-that-checks-the-os-cis-compliance/
Config Snapshot
A configuration snapshot is a collection of the configuration items for the supported resources in each AWS account.
Troubleshooting
When actions fail, triage using the AWS SSM service console GUI -> Automation -> executed tasks, or this AWS CLI command:
aws configservice describe-remediation-execution-status \ -config-rule-name "$MY_CONFIG_RULE_NAME" -region "$MY_REGION""
Social
Announcements are publishes in the Discussion Forum about AWS Config at:
https://forums.aws.amazon.com/forum.jspa?forumID=184
References
how-to-use-aws-config-to-monitor-for-and-respond-to-amazon-s3-buckets-allowing-public-access
Easily Transform Compliance to Code Using AWS Config, Config Rules, and the Rules Development Kit
Remediate Non-Compliance Using AWS Config Rules, AWS CloudWatch Events, & AWS Lambda Functions
AWS Config - Rules, Resources & Timelines with DEMO | IMPORTANT Security Service
https://aliceandbob.company/2020/07/16/aws-config-what-why-and-how/
Notification on security group changes using AWS Config May 3, 2020
AWS Config | Assess, Audit, and Evaluate AWS Resources | Concept and Configuration Demo by “Let’s Go Tech In a Cloudy Way”
STAR: AWS Config - Rules, Resources & Timelines with DEMO | IMPORTANT Security Service May 2, 2017
New! AWS Config Rules: Improve Governance Over Configuration Changes at AWS re:Invent 2015 | (SEC314-R) Oct 19, 2015
Achieving Continuous Compliance using AWS Config Oct 18, 2017
What is AWS Config? Mar 26, 2021 by CloudEnthusiasts
I added this to my deep-dive hands-on tutorial of AWS Config for compliance-as-code at https://wilsonmar.github.io/aws-config
https://www.youtube.com/watch?v=oBuLtfGHETY Manage and Track Application and Infrastructure Configuration Changes using AWS Config May 18, 2021