Wilson Mar bio photo

Wilson Mar

Hello!

Email me Calendar Skype call

LinkedIn Twitter Gitter Instagram Youtube

Github Stackoverflow Pinterest

Federate authentication of authentication through API Gateway via Python Lambda

US (English)   Español (Spanish)   Français (French)   Deutsch (German)   Italiano   Português   Estonian   اَلْعَرَبِيَّةُ (Egypt Arabic)   中文 (简体) Chinese (Simplified)   日本語 Japanese   한국어 Korean

Overview

NOTE: There is a biological Cognito, which is not this topic.

High-level short summary of Cognito: VIDEO course: “Using Amazon Cognito to Manage Authentication & Authorization to your Mobile and Web Apps”

NOTE: Content here are my personal opinions, and not intended to represent any employer (past or present). “PROTIP:” here highlight information I haven’t seen elsewhere on the internet because it is hard-won, little-know but significant facts based on my personal research and experience.

PROTIP: This article presents configurations using both the AWS GUI Management Console and Terraform coding to provide repeatability and ease of reconfiguration (moving from test to prod on several regions).

The Amazon Cognito Documentation menu lists 3 products:

  • Amazon Cognito User Pools
  • Amazon Cognito Identity Pools (Federated Identities)
  • Amazon Cognito Sync

Federation

aws-cognito-feds-398x1168 Amazon Cognito is an authentication and user management service that enables federated authentication from Amazon user store accounts, Apple, Facebook, Google, Twitter/Digits, OpenID, SAML to GitHub or Microsoft Active Directory, and custom mechanisms.

Hosting on AWS means that the SaaS service can scale to a lot users around the world.

“AWS Identity Federation Course: What AWS Identity Federation is, Types & Demos” by Tom Lynch

VIDEO: “Using AWS Identity Federation to Simplify Access at Scale” compares the various approaches:

  • AWS IAM allows you to configure different OpenID or SAML Identity providers for each of your AWS accounts.
  • AWS SSO (Single Sign-On – using a single identity provider for all) with SAML 2.0 for user access to AWS Simple ID, and Microsoft AD and LDAP.
  • AWS Cognito enables secure authentication to your web or mobile applications using both SAML 2.0 and web identity federation.

PROTIP: A developer accounts needs to be setup with each third-party (Facebook) for Cognito to interact with.

Cognito usage workflows

There are several options, from simple to more complex/full featured:

Cognito in ReactJs Serverless in AWS Amplify

AWS workshop: “Build a Serverless Web Application: with AWS Lambda, Amazon API Gateway, AWS Amplify, Amazon DynamoDB, and Amazon Cognito”
aws-cognito-serverless-974x424

Code for the above is at github.com/jspruance/aws-cognito-tutorial-complete

Modules:

  1. HOST A STATIC WEBSITE (ReactJs with AWS Amplify hosting)
  2. MANAGE USERS (using Cognito User Pools)
  3. BUILD A SERVERLESS BACKEND
  4. DEPLOY A RESTFUL API
  5. TERMINATE RESOURCES

Starter ReactJS UI for the “Create a Serverless App” tutorial series by J Spurance https://github.com/jspruance/aws-cognito-tutorial-starter uses https://github.com/facebook/create-react-app to Create React apps with no build configuration.

https://aws.amazon.com/getting-started/hands-on/build-react-app-amplify-graphql/?trk=gs_card

AWS Amplify API Console provides hosting of CSS, libraries, etc. to JavaScript clients.

Cognito to GitHub Pages

A simple example is to access GitHub Pages, explained by Hands-on 1hr “Manage Authentication with Amazon Cognito”
aws-cognito-flow-899x474

Cognito Workflow using AWS API Gateway

References:

aws-cognito-thru-api-631x518

1) Users from a web browser typically invoke JavaScript which addresses an Amazon API Gateway, which controls access. 2) If authenticated, the API Gateway invokes a JWT Authorizer which 3) connects Amazon Cognito. 4) Cognito returns to JWT Authorizer 5) back to API Gateway, which 6) invokes a Lambda that 7) returns to the user’s browser.

https://github.com/ghdna/cognito-express Authenticates API requests on a Node application by verifying the JWT signature of AccessToken or IDToken generated

Cognito with S3 buckets & DynamoDB

A more complex example using S3 buckets is the Hands-on “Serverless Web Development with Python for AWS”
aws-cognito-api-mgmt-605x342

VIDEO: “Fine-grained Access Control with Amazon Cognito Identity Pools” aws-cognito-s3-1890x673 shows use of using Attribute-based access controls to pass claims from token as principal tags.

Cognito with S3, CloudFront, Kinesis Firehose streaming

Add streaming to Amazon Firehose by Hands-on 2h “Deploy a Highly Available Serverless Application Using AWS Services” Athena database and AWS Glacier to archive history.
aws-cognito-with-firehose-802x900

iOS application using Cognito

David Tucker’s sample iOS application that uses Cognito User Pools is at https://github.com/davidtucker/CognitoSampleApplication

github.com/davidtucker/ps-serverless-app is explained in his Pluralsight video series “Building Serverless Applications on AWS”.

Each high-level folder is a workspace defined in package.json:

A. The infrastructure folder

  • AWS CDK> leveraging pre-configured app components in TypeScript
  • Cloudwatch for Observability, Logging, metrics, and alarms;
  • For Communication: event bus and app messaging EventBridge, SQS, SNS
  • XRay for tracing

B. The services folder

  • Backend Serverless microservices built in JavaScript
  • For Continous Delivery: CodeBuild, CodePipeline, CloudFormation

C. The webapp folder

  • Frontend ReactJs app built in JavaScript


To make it all happen:

  • Cognito menu for reference: aws-cognito-menu-221x534

      A. Create Amazon Cognito user pool

  1. Create user pool
  2. Create app client without client secret
  3. Create domain name
  4. Create resource server with custom scopes
  5. Configure App client Identity Provider (IdP) and other settings
  6. Add users manually or import several at a time

    B. Set up an authorization endpoint

  7. Create the authorization Lambda function
  8. Write the function code
  9. Grant Cognito access to the function
  10. Test the function
  11. Create an authorization endpoint

    C. Set up AWS API Gateway Authorization

  12. Create an API authorizer
  13. Setup endpoint authorization
  14. Deploy to Stage

    D. Test the Secure API Gateway

  15. Test the authorization endpoint
  16. Test the secure endpoint
  17. View JWT in CloudWatch Logs

    E. Delete Cognito Users

  18. Delete Cognito User

Cognito Terraform

This provides Terraform that creates an ALB listener rule configured for Cognito authentication using a local user pool. It can also be used with a supplied Cognito user pool allowing for greater customizability. This module is meant to be a better solution when you need to protect web assets and don’t want to use server-side HTTP basic authentication to keep the general public out of a staging site.

Among other benefits, this means your backend configuration does not have to change to restrict access and also means that users can have individual usernames/passwords that they can perform account resets on.

Cognito stores user information in a Resource Server which manages user data in User Pools.

Create Cognito User Pool

See https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-identity-pools.html

Cognito service landing page

  1. Cognito Management Console GUI:

    Version 1 of the Amazon GUI for Cognito at
    https://us-west-2.console.aws.amazon.com/cognito/welcome?region=us-west-2
    enables you to choose between “Manage User Pools” (the directory of users in Amazon Cognito) or “Manage Identity Pools” for Authorization of temporary AWS user credentials.

    Version 2 of the Amazon GUI for Cognito at
    https://us-west-2.console.aws.amazon.com/cognito/v2/home?region=us-west-2#
    has a drop down to “Grant access to AWS services” before clicking the orange “Create indentity pool”.

    It prompts you to create the user pool as the first step.

  2. In Version 1 URL:

    Cognito User Pool Setup

    Cognito User Pools (CUPs) are referenced during sign-up and sign-in operations.

    Cognito normalizes secrets as CUP (Cognito User Pool) tokens (pronounced “cup tokens”) for use with AWS API Gateway and Lambda.

    But to authenticate S3 and DynamoDB, the CUP token is sent to an Amazon Cognito Identity Pool.

    Terraform to setup user pool with SMS and software token MFA, and account recovery.

    When creating a user pool:

  • (Pool) Name - QUESTION: What is the naming convention?

  • (User) Attributes - cClick “Also sign-in with verified email address” because email addresses are uniqu, but requires users to have one). Cognito manages attribute values for each user, and ensures that required attributes are obtained:

    aws-cognito-attribs-1898x778

  • Policies - password minimum length. Selecting “Only allow administrators to create users” requires more toil by administrators.
  • MFA and verifications - recovery; attributes to verify
  • Message customizations
  • Tags
  • Devices
  • App clients
  • Triggers

User Pool App Clients Config

  1. Which app clients will have access to each user pool?

    aws-cognito-client-config-547x392

    QUESTION: What are the recommendations for days and minutes:

    • Refresh token expiration (60 - 3560 days)
    • Access token expiration (5 minutes - 1 day)
    • ID token expiration (5 minutes - 1 day)

    Cognito Client SRP Auth

  2. The “Generate client secret” or generate_secret Boolean true parameter in Terraform for client file is what defines use of SRP auth details (at the bottom of this page).

  3. Optionally with AWS Pinpoint for multichannel marketing communication analytics and to send SMS messages to phones from an originator ID.

    See https://github.com/capless/warrant - a Python library for using AWS Cognito with support for SRP.

    OAuth 2.0 app settings

  4. Define Sign-in Callback and sign-out URLs:

  5. Define OAuth 2.0

    aws-cognito-oauth2-config-577x271

    Create Cognito Domain

  6. Click on “Domain name” on the Configuration menu.

  7. Define the unique internet domain name (in each region) that Amazon Cognito uses to host sign-up and sign-in pages for each User Pool:

    https://___.auth.us-east-1.amazoncognito.com

  8. After association, a DNS alias record needs to be added to the domain’s hosted zone.

  9. Generate a certificate in the ACM (AWS Certificate Manager).

    Create Users and Groups

  10. Terraform to setup user groups

  11. In the GUI “Users and groups”, users can also be imported from a CSV file.

    Alternately, users can also be added manually.

    API Gateway Authorizer config

    Terraform for AWS API Gateway is at:

    • https://registry.terraform.io/modules/terraform-aws-modules/apigateway-v2/aws/latest
    • https://github.com/terraform-aws-modules/terraform-aws-apigateway-v2

    aws-api-gatewy-menu-204x698

  12. VIDEO: Copy the Cognito Pool Id and ARN within its “General Settings” menu (such as us-east-1.p40lmEB0d) to construct Cognito’s Issuer URL for your region:

    https://cognito-idp.us-east-1.amazonaws.com/us-east-1.p40lmEB0d

    PROTIP: Flexibility (and less toil) in specifying different regions is why automation IaC is useful – specify the region in one place and it gets automatically applied without mistakes every time.

    VIDEO: Lambda authorizer

    Set up endpoint authorization

    VIDEO: “Enter client IDs that are registered with identity providers or any arbitrary string in the JWT audience claim that the authorizer must verify.”

  13. Click “Add audience”.
  14. Switch to the Cognito menu item “App clients” to highlight and copy the App client Id (such as 2iqdo52jkqg0q2cqbe76qah08).
  15. Switch back to paste it.
  16. Click “Create and Attach”.

    NOTE: https://github.com/serverless/examples/tree/master/aws-node-auth0-cognito-custom-authorizers-api

    • AWS CLI2
    • Python Boto3
    • AWS Cognito web
    • etc.

    Set up authorization scope

    VIDEO:

  17. Click “Add scope”.
  18. Switch to the Cognito menu item “App client settings” to highlight and copy the Allowed Custom Scopes (such as test/test.read).
  19. Switch back to paste it.
  20. Click “Create and Attach”.
  21. Click “Save”.

    Deploy

  22. VIDEO: Click “Deploy”.
  23. Select version. aws-api-deploy-563x334

    identity_pool_roles_attachment

    TODO:

    View JWT in CloudWatch Logs

    VIDEO:

  24. In API Gateway, click on menu “Routes”.
  25. Click on GET for “/test”.
  26. Click on “Configure” for Integration backend resource.
  27. Click on the Lambda function link to open it.
  28. Scroll to click “Monitor” on the bar.
  29. View logs in CloudWatch.
  30. Click on a Log stream ID.
  31. Click on a Log events entry to expand it.

    VIDEO: JWT Structure dynamically illustrated


Use AWS Directory

  • http://aws.amazon.com/directoryservice/faqs/

The AWS Directory service connects existing on-premises Microsoft Active Directory to the AWS cloud.

  1. Enable Kerberos
  2. Create an AD Connector to connect to your on-premise Microsoft Active Directory domain using AWS applications such as Amazon WorkSpaces, Amazon WorkDocs, or Amazon WorkMail using their corporate credentials. https://docs.microsoft.com/en-us/azure/active-directory/hybrid/whatis-azure-ad-connect

  3. Configure an Amazon Virtual Private Cloud (VPC) with a hardware VPN connection to your on-premises environment, or provision a dedicated connection with AWS Direct Connect.

  4. Establish a limited privilege account used by AD Connector to authenticate and connect to one of the domain controllers and proxy various authentication, domain join, and look-up requests – by providing the name of your on-premises Microsoft Active Directory, DNS servers to discover Microsoft Active Directory, and an account name and password pre-created in your Microsoft Active Directory.

  5. Configure auto scaling.

Cognito Client Coding in various languages

Python:

  • https://github.com/JinlianWang/aws-lambda-authentication-python
  • https://www.educative.io/edpresso/what-is-the-python-code-for-aws-cognito
  • https://medium.com/@houzier.saurav/aws-cognito-with-python-6a2867dd02c6
  • https://gist.github.com/Integralist/07d62f6a55ba42481b23458c15c00e27 require you to be using Pipenv for handling you Python dev environment.

https://docs.aws.amazon.com/lambda/latest/dg/services-cognito.html Example Amazon Cognito message event

{
  "datasetName": "datasetName",
  "eventType": "SyncTrigger",
  "region": "us-east-1",
  "identityId": "identityId",
  "datasetRecords": {
    "SampleKey2": {
      "newValue": "newValue2",
      "oldValue": "oldValue2",
      "op": "replace"
    },
    "SampleKey1": {
      "newValue": "newValue1",
      "oldValue": "oldValue1",
      "op": "replace"
    }
  },
  "identityPoolId": "identityPoolId",
  "version": 2
}

Authentication providers

VIDEO:

When an InitiateAuth operation is successful, Cognito responds with either a token or (another) challenge.

SAML protocol

The Security Assertion Markup Language (SAML) enables Single-Sign-On (SSO) by users to access various systems from a web browser without repeatedly entering user credentials. It is widely used for Software-as-a-Service (SaaS) solutions.

It works like getting an armband for VIP chairs at a music concert. VIDEO:

SAML is a mature technology dating back to 2002. The current SAML 2.0 standard was developed in 2005. That’s why it uses XML language as its identity data format and simple HTTP and SOAP for its data transport mechanisms. SAML provides communication between identity providers and service providers using encrypted, digitally signed XML-based certificates. As an XML-based protocol, SAML is a feature-rich, versatile standard that can be used on nearly every platform. Thus, SAML is widely used in enterprise and government settings.

aws-cog-SAML2-flow-650x280
Image from Microsoft

1) A new user (called the “Principal”) uses a web browser containing JavaScript which 2) makes a SSO request to the Web App which 3) generates a SAML Authorization request to 4) query the “Identity” service (like at a ticket booth). Identity services include AWS Cogito or Azure AD. Alternately, JavaScript on the browser can directly call the Identity service.

Either way, 4) the Identity service (such as ADSF2) authenticates the user by referencing its store of Identities (such as Windows Active Directory) and 5) creates a SAML security token (“Security Assertion” which can include attributes such as name, phone number, email address, etc.) in a callback to the user’s web browser. (an XHTML form so the user can create a to the Provider (like showing the armband at the gate).

6) The web browser redirects tokens (posts SAML assertion) to the Web App Service Provider which 7) validates the SAML response with token before 8) returning the secure page to user (seats in the VIP section), access to AWS Management Console, etc.

The Provider then creates a “Security Context” and redirects to the request resource. This can be an AWS Lambda trigger to send an email and log the event so the SOC (Security Operations Center) can monitor security-relevant activities.

The customizable UI provides an OAuth 2.0 compliant auth server.

The amount of time access lasts is determined by the Identity Provider.

OIDC = OpenID Connect protocol

A competitor to SAML is OpenID Connect.

First published in 2014, OIDC is a simple identity layer on top of the OAuth 2.0 authorization framework managed by the OpenID Foundation.

OpenID is designed to only allow a service provider (Google) to initiate the identification process.

The protocol uses RESTful API communication to transmit JSON web tokens (JWT) between the identity provider and service provider. Each token contains common claims such as the user’s name, email address, birth date, picture, and other personal data. The tokens are digitally signed and encrypted.

OIDC is easy to implement with lightweight data processing requirements, which makes it the preferred authentication standard for mobile games, social media integrations, and other mobile applications.

OIDC is easy to integrate with simple apps, but also provides security options that adhere to rigorous enterprise requirements. OIDC’s easy-to-consume tokens support a broad spectrum of signature and encryption algorithms.

Additional back-end processes include mass upload from a CSV file for first-time setup or for disaster recovery.

Cognito Resource Server and clients

Terraform for resource server for a scope.

Cognito Data Sources

data, domain, signing-cert, data, clients data UI customization

Cognito Identity Pools

  • https://aws.amazon.com/blogs/mobile/customizing-your-user-pool-authentication-flow/

Cognito Identity Pools define identities which have (or have not) been authenticated by 3rd-party providers (Facebook, etc.).

“Federated identities” provide temporary access to AWS credentials by working in tandom with Amazon Cognito User Pools to allow users to operate and access specific features from AWS.

The Identity Pool creates a token using STS (Security Token Service) used to access S3, DynamoDB, etc.

The credentials are linked to an AWS Role the admin. assigned to users within the Identity Pool.

It may allow for unauthenticated (guest) identities to view open public information such as summary dashboards.

Terraform for Identity pool “my-saml-provider”

identity_pool_roles_attachment assume_role_policy

provider principal tag provides an AWS Cognito Identity Principal Mapping.


Amazon Cognito Sync

WARNING: Amazon is moving users to AWS AppSync which enables multiple users to access the same data for collaboration in real time on shared data.

Amazon Cognito Sync ensures resiliancy by storing (for distribution) application state, profile info, previously viewed content, location tracking, etc. This include management of caches on mobile devices, to enable offline use.

Sync info is saved and retrieved by a key-value pair (dictionary) saved within a dataset.

CAUTION: Currently, each dataset is limited to 1MB because the entire dataset is sync’d at once. Each identity can have a maximum of 20 datasets.


More technical details

SRP auth details

The SRP (Secure Remote Password) protocol is an augmented password-authenticated key exchange (PAKE) protocol, designed so an attacker who steals server data would not be able to masquerade as the client (unless they first perform a brute force search for the password). A Wiki entry says it was specifically designed to work around existing patents.

SRP is a zero-knowledge proof protocol, where the server doesn’t have to store password equivalently information (hashed version) in a database. Thus, an eavesdropper or man-in-the-middle cannot obtain any meaningful information to perform an attack.

During registration on the browser, a verifier posted to the server instead of sending the password entered by the user:

  • client.generateRandomSalt(); by a KDF (Key Derivation Function) to derive a very large number eg: PBKDF etc.

  • client.generateVerifier(salt,email,password); using the derived PBKDF and an SRP group, which consists of one large prime number and a generator. Admins can choose between several groups eg: 1024 bit, 2048 bit, etc

In addition to Go code, there is JavaScript code for the above is at Simon Massey’s https://github.com/simbo1905/thinbus-srp-npm, which provides this diagram of authentication using SRP:

SRP

To prove that the user knows their password, client and server exchange non-sensitive information to generate a key independently for mutual verification. It generates using SRP group a one-time ephemeral (private) a value and its (public) counterpart A, where private a is kept in-memory and its public value A is sent to the server.

Ramesh Lingappan shows how to install his code for a demo.

HTTP API

  • https://www.youtube.com/watch?v=yCAlJv6zfn4

PROTIP: AWS HTTP API costs 30% less than REST API calls.

HTTP APIs can be edge-optimized, private, and use AWS WAF with resource policies and certificates for backend authentication.

HTTP APIs can have execution logs and use X-Ray tracing via Access logs to Amazon Kinesis Data Firehose (REST APIs can’t)

Cognito User Pool Backup/Restore

https://github.com/rahulpsd18/cognito-backup-restore


References