Wilson Mar bio photo

Wilson Mar

Hello. Join me!

Email me Calendar Skype call 310 320-7878

LinkedIn Twitter Gitter Google+ Youtube

Github Stackoverflow Pinterest

Invisible ubiquitious server clouds across the land


This is a hands-on narrated tour on how to end up with a serverless app running in a server cloud.

The sentiment about servers is reflected in the wi-fi password attendees of #ServelessConf type in:
serverless hateservers-452x183-70kb

Serverless = FaaS

“Serverless” refer to an architectural style called FaaS (Function as a Service) where programming code are independently deployed and run on a cloud system rather than on-premises servers. The developer simply uploads the source code, then leave it to pros at the cloud provider to take care of security, monitoring, disk space management, log management, scaling, redundancy, backup, crash reporting, etc..

Zero system administration by the developer.

It’s a fast route to get apps in production.

PROTIP: It’s still up to developers to do testing and performance measuring and tuning. Use of multi-tenancy makes for response-time variation. So do sythentic transactions outside the cloud vendor to monitor user experience, and to keep your app in cache to avoid process start-up after sleeping.

Mike Roberts at http://martinfowler.com/articles/serverless.html wrote “Depending on the circumstances, such systems can significantly reduce operational cost and complexity at a cost of vendor dependencies and (at the moment) immaturity of supporting services.”

Database idle costs money!

“You never pay for idle” Austen says in an interview with by CloudAcademy Introduction to the Serverless Paradigm [23:50]

PROTIP: But this is not true within AWS if you use DynamoDB, which Amazon touts as the default database. While Lambda does not incur charges while idle, DynamoDB instances do incur charges for data stored even though no data is read or written to it.

PROTIP: On AWS use SimpleDB instead of DynamoDB for true no-cost idle.

FaaS Providers

There is plenty of competition (to keep prices low).

PROTIP: The future of FaaS vendors isn’t the front-end but the back-end services that include API Gateways and Artificial Intelligence features such as image recognition, text sentiment analysis, natural language process (NLP), and Machine Learning.

AWS Lambda

The AWS Serverless Application Model (SAM) at https://github.com/awslabs/serverless-application-model was announced Nov 2016 to define the SAM format to build Cloud Formation templates to access Amazon API Gateway APIs, AWS Lambda functions, and Amazon DynamoDB tables needed by serverless applications.

Two AWS evangelists created pre-built templates with manual instructions, which they walked though in 30 minutes on Twitch.tv:

  1. Get an AWS IAM User with AWSCodeStarFullAccess.
  2. Login the AWS Console to select region N. Virginia.
  3. In AWS CodeStar. Start a project. Node.js. create role.
  4. Choose a project template:
    • Under Application Category select Web Application.
    • Under Programming Languages select Node.js.
    • Pick “Node.JS Web Application AWS Lambda (running serverless)”.

Azure Functions

Marketing information is at https://azure.microsoft.com/services/functions

  1. Create an Azure Storage account.
  2. Search for “Function App” in the Azure Marketplace.
  3. Click Create.
  4. Provide an app name that’s globally unique among all others public.


  5. Create or use a Resource Group to manage your similar functions as a group.
  6. Select a Location closest to you.

    PROTIP: Include the location as a suffix in your app name.

  7. Selection of Memory Allocation can be adjusted once you figure out how much after running a few times.
  8. Create.
  9. Click Refresh or the “bell” icon to see messages related.

  10. After deployment, click the Resource Group, then the app name.
  11. Click “Webhood + API” and JavaScript language (or C#).
  12. Copy the Function URL and paste it in a browser.
  13. Click Logs to see the history tracking.
  14. Add a Parameter.

IBM Bluemix OpenWhisk


The advantage of IBM’s hybrid-cloud approach is that one can use IBM’s proprietary Bluemix UI and then use command-line with OpenWhisk (which is open sourced).


Google Cloud Functions



Google Firebase

Iron.io and other on-premises

Iron.io has their Gesalt Framework

Usage in the wild

Compelling Science Fiction.com uses Simple Email to read emails and sends it to S3. Lambda emails notifications. Stories are saved in DynamoDB. Python pulls data.

Eric Jonas does hyperperameter sweeps, Monte Carlo simulations. Spark requires dedicated servers and is not very elastic. map of map reduce.

His Simultaneous “Big Lambda” talk


The other side of freedom from server hassles is that developers also give away control.

Having one’s data in another company’s cloud requires trust in that company’s ability to keep data secure, redundant, etc.

PROTIP: Going with a particular vendor’s API means that you need to keep up with changes in APIs that can occur frequently, even though they may not apply to your own operation.

There is the danger vendor lock-in. But luckily, there is a way to alieviate that:

Serverless Coding Frameworks

Apex is from Terraform, a more feature-rich server product.


Sparta for AWS Lambda, as a Golang app, is baked into deliverable binary (unlike Node).

  • https://gospart.io
  • https://github.com/mweagle/Sparta

Gordon is written in Python

Zappa is written in Python for Flask apps


Serverless the company

The name “serverless” has been co-opted by enprepreneur Austen Collins @austencollins who built the Serverless company around its open-source Serverless Framework on GitHub, a combination of command-line utilities and conventions.

His initial Serverless presentation at AWS:Invent 2015

Social media:

The company hosts on August 17 in San Francisco an Emit Conference for “event-driven architectures”.

Install Serverless framework

  1. Install Node.js as a pre-requisite since the framework is written in Node.js.

  2. Install the serverless-framework

    npm install -g serverless
  3. Verify version:

    serverless -v

    NOTE: Serverless was in Beta version 0.5.6 as of June 2016, with v1.0 announced 24 June 2016.

    Update Serverless

    PROTIP: I subscribed to get notifications of changes, and I can see a lot of refactoring is happening very quickly, so I suggest that you update frequently.

    npm update -g serverless

    Nothing returns if you’re up-to-date.

    Look around

  4. Get summary of commands using the abbreviated command:


    the response:

           * Serverless documentation: http://docs.serverless.com
           * You can run commands with "serverless" or the shortcut "sls"
           * Pass "--help" after any  for contextual help
    config ........................ Configure Serverless
    config credentials ............ Configures a new provider profile for the Serverless Framework
    create ........................ Create new Serverless service
    install ....................... Install a Serverless service from GitHub
    package ....................... Packages a Serverless service
    package function .............. undefined
    deploy ........................ Deploy a Serverless service
    deploy function ............... Deploy a single function from the service
    deploy list ................... List deployed version of your Serverless Service
    invoke ........................ Invoke a deployed function
    invoke local .................. Invoke function locally
    info .......................... Display information about the service
    logs .......................... Output the logs of a deployed function
    metrics ....................... Show metrics for a specific function
    remove ........................ Remove Serverless service and all resources
    rollback ...................... Rollback the Serverless service to a specific deployment
    slstats ....................... Enable or disable stats
    AwsConfigCredentials, Config, Create, Deploy, Info, Install, Invoke, Logs, Metrics, Package, Remove, Rollback, SlStats

    Look how far they’ve come since the first versions:

     _______                             __
    |   _   .-----.----.--.--.-----.----|  .-----.-----.-----.
    |   |___|  -__|   _|  |  |  -__|   _|  |  -__|__ --|__ --|
    |____   |_____|__|  \___/|_____|__| |__|_____|_____|_____|
    |   |   |             The Serverless Application Framework
    |       |                           serverless.com, v0.5.6
           * Serverless documentation: http://docs.serverless.com
           * You can run commands with "serverless" or the shortcut "sls"
           * Pass "--help" after any <context> <action> for contextual help
           * Add "--debug" to any command for extra useful logs
    project ........ create, init, install, remove
    function ....... create, deploy, logs, remove, rollback, run
    endpoint ....... deploy, remove
    event .......... deploy, remove
    dash ........... deploy, summary
    stage .......... create, remove
    region ......... create, remove
    resources ...... deploy, diff, remove
    plugin ......... create
    variables ...... list, set, unset
  5. Verify:

    command -v serverless

    The response:

  6. Navigate to framework folders and files:

    cd ~/.npm-packages/lib/node_modules/serverless
    ls -al

    In the serverless folder

  7. View the README.md file using a Markdown reader:

    atom README.md

    Instead of the Atom text editor (from GitHub), alt-click on the file to select Markdown Preview.

    The file contains a list of projects, plugins, and consultants who provide services.


README.md, CONTRIBUTING.md, LICENSE.txt are standard GitHub files.

.eslintrc.js contains rules for how lint programs identify issues with code formatting.

.jsbeautifyrc contains settings for JavaScript code beautify program.


.npmignore defines files and folders for NPM to ignore.

.travis.yml is used by the Travis task runner.

Dockerfile defines how to load the program in Docker.



.idea contains settings for use by the IDEA IDE.








#### New #

  1. Create a serverless.yml file containing the infrastructure:

    sls create -t aws-node.js
  2. Define API Gateway

       handler: handler.hello
          - http:
              path: hello
              mention: get


React Serverless app running in Azure?

https://github.com/99xt/serverless-dependency-install Serverless plugin to manage dependencies

with architectural best practices using NPM, Angular, Gulp.



  • Handlers compress or transform objects while being uploaded to Amazon S3.


Sample Projects

  1. Create a folder to hold serverless projects. For example:


  2. Create a blank project.

    serverless create project


    Pick a sample project to implement:

    For example, load a sample project from above:

    serverless project install serverless-graphql

    The response:

    _______                             __
    |   _   .-----.----.--.--.-----.----|  .-----.-----.-----.
    |   |___|  -__|   _|  |  |  -__|   _|  |  -__|__ --|__ --|
    |____   |_____|__|  \___/|_____|__| |__|_____|_____|_____|
    |   |   |             The Serverless Application Framework
    |       |                           serverless.com, v0.5.6
    Serverless: Installing Serverless Project "serverless-graphql"...  
    Serverless: Downloading project and installing dependencies...  
    Serverless: Initializing Serverless Project...  
    Serverless: Enter a new stage name for this project:  (dev)
  3. Press Enter to accept the default

  4. Keyboard up and down to select:

    Serverless: For the "dev" stage, do you want to use an existing Amazon Web Services profile or create a new one?
      > Existing Profile
     Create A New Profile
    Serverless: Select a profile for your project:
      > GitHubPublisher1
    Serverless: Creating stage "dev"...  
    Serverless: Select a new region for your stage:
      > us-west-2
    Serverless: Creating region "us-west-2" in stage "dev"...  
    Serverless: Deploying resources to stage "dev" in region "us-west-2" via Cloudformation (~3 minutes)...

If you see these error messages, let me know because I’m stuck!!!!

        fn = function () { throw arg; };

ServerlessError: ServerlessError: The security token included in the request is invalid.
  at new ServerlessError (/Users/mac/.npm-packages/lib/node_modules/serverless/lib/Error.js:17:11)
  at ResourcesDeploy.<anonymous> (/Users/mac/.npm-packages/lib/node_modules/serverless/lib/actions/ResourcesDeploy.js:241:25)
  at ResourcesDeploy.tryCatcher (/Users/mac/.npm-packages/lib/node_modules/serverless/node_modules/bluebird/js/release/util.js:16:23)
  at Promise._settlePromiseFromHandler (/Users/mac/.npm-packages/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:502:31)
  at Promise._settlePromise (/Users/mac/.npm-packages/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:559:18)
  at Promise._settlePromise0 (/Users/mac/.npm-packages/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:604:10)
  at Promise._settlePromises (/Users/mac/.npm-packages/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:679:18)
  at Async._drainQueue (/Users/mac/.npm-packages/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:138:16)
  at Async._drainQueues (/Users/mac/.npm-packages/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:148:10)
  at Immediate.Async.drainQueues [as _onImmediate] (/Users/mac/.npm-packages/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:17:14)
  at tryOnImmediate (timers.js:543:15)
  at processImmediate [as _immediateCallback] (timers.js:523:5)

Structure of Folders and Files

PROTIP: The serverless framework save developers’ time by standardizing the structure of folders and files.

s-project.json       // Project file (JSON or YAML)
s-resources-cf.json  // CloudFormation template
s-templates.json     // Config templates and variables
admin.env            // AWS Profiles (gitignored)
_meta                // Metadata (gitignored)
function1            // A custom function

This set of project file is what developers work with.

The serverless framework programming which read and process project files is written in Node JavaScript.

The s-project.json lists plugins:

  "name": "notes",
  "custom": {},
  "plugins": [

A key differentiator with the Serverless Framework is that infrastructure specs are defined as code in one project. s-resources-cf.json is a AWS CloudFormation template specifying security (IAM) roles, SNS email topics, DynamoDB tables, Queues, ARNs.

Components are deployed per stage (dev, prod, etc.), allowing developers to easily deploy separate prod, test and dev environments.

Within the folder for each function is a s-function.json file containing metadata such as plugins installed with the project.

When the framework is created for a project, a .gitignore file is created with other files to specify private and temporary work files that should not be publicly uploaded to GitHub.

Project metadata in file _meta outputs and user variables used in function configurations. But due to the sensitive nature of these variables, the file needs to be gitignore’d by default. A workaround is to use the Serverless Meta Sync plugin which stores project metadata in S3.


Serverless Command-line

Lambda functions can be defined from a command-line using the Serverless framework.

AWS May 2016 Webinar Series - Deep Dive on Serverless Web Applications


Keeping Secrets

BLAH: AWS Lambda doesn’t allow setting and reference to operating system environment variables.

Lambda functions have a deploy.env file in combination with the --configFile flag to set values which will be prepended to your compiled Lambda function as process.env environment variables before it gets uploaded to S3.

Secrets such as DB connection string or encryption key are secure values that should not be checked into version control (specified in a .gitignore file).

Local Runs for testing

For a “test-first” approach to achieve code maturity, doing test runs locally before committing to a team branch is important.

Let’s examine the choices to emulate AWS Lambda locally:

  1. Install the “Command line tool for locally running and remotely deploying your node.js applications to Amazon Lambda.” from github.com/motdotla/node-lambda

    npm install -g node-lambda

  2. Describe tests in a JSON file.


The heart of Serverless are its Plugins. Several plugins come with the Framework.

PROTIP: Plugins need to be installed for each project that uses each.

  1. List plugins installed.

    ls _meta/

  2. Navigate your active directory to the root of your project.
  3. Plugins are downloaded from GitHub by npm:

Functions of the same component can use the lib folder to share common code.

Get Permissions


  • http://stackoverflow.com/questions/37779324/how-to-troubleshoot-serverless-iam-permissions


Phillip Muens (@pmmuens, github.com/pmuens) from Germany

Matthew Fuller

Jake Knowles

  • AWS Lambda: Serverless Microservices Guide with Simple Instructions

John McKim @johncmckim blogs on Medium:


More on Clouds

This is one of a series on Cloud computing