Scala ecosystem

The language, not the opera house in Milan

Here is my hands-on approach to introduce the ecosystem around the Scala programming language.

The sequence of information and examples here has been hand-crafted based on various other tutorials. What’s new here is the sequence and hands-on approach of how information is introduced.

(This one page will be separated into separate ones)

History

• 2016 Feb 22 Typesafe changes name to
• 2014 version 2.10.4
• 2011 TypeSafe formed to provide commercial support for a Reactive Platform.
• 2003 Martin Odersky’s group releases Version 1.0 of Scala after a year of work.

Features of each version are detailed on the Wikipedia page

What is Reactive?

On 2013, several people associated with Typesafe published a website named http://www.reactivemanifesto.org/ which asked readers to sign the manifesto as a commitment to build “Reactive” system with a more “coherant” appoach to systems that have these characteristics:

• Responsive - responds in a timely manner
• Resilient - responsive in the face of failure
• Elastic - stays responsive under varying workload
• Message Driven - rely on asynchronous message-passing that ensures loose coupling, isolation, location transparency, and provides the means to delegate errors as messages.

Terms used in the Manifesto are further defined in a Glossary, and reflect the blog article by @jboner named Reactive Manifesto 2.0. dated September 16, 2014.

Tweets about this topic are at @reactivemanifesto.

Curiously, the Manifesto doesn’t mention HOW it achieves all that wonderfullness. It doesn’t ever mention the word “Scala” at all.

What Is Scala?

The name Scala is a portmanteau of “scalable” and “language”. Thus, the emphasis on stateless and non-blocking features in the language.

The official repository is at github.com/scala/scala

• docs.scala-lang.org/tutorials/

• Martin Odersky’s Coursera Course

• Scala for the impatient

Prominant companies using Scala are listed on this Wikipedia page

Scala is said to power The Guardian (UK), Walmart, Sony, Huffington Post, etc.

The extent of Twitter’s adoption of Scala are:

• twitter.github.io/scala_school started as lectures at Twitter.

• twitter.github.io/effectivescala presents “best practices” for Scala at Twitter. Useful for understanding idioms in Twitter’s code.

Agencies/consultants working with Scala:

• softwaremill.com

Basics of the Scala language is presented during hands-on activities to set it up on your machine, below.

Installation

1. Identify the latest version
   http://www.scala-lang.org/download

   PROTIP: On a Mac, use Homebrew instead of downloading from http://scala-lang.org/download (as many tutorials suggest).

   brew install scala --with-docs
   

   which installs to:

   /usr/local/Cellar/scala/...
   

   Alternately, on Ubunto Linux:

   sudo apt-get install scala
   

2. Verify the version:

   scala -version
   

   The response (assuming an installer from March 10, 2016):

   Scala code runner version 2.11.8 -- Copyright 2002-2016, LAMP/EPFL
   

NOTE: The “EPFL” is for École Polytechnique Fédérale de Lausanne in Switzerland where Martin Odersky works, in their Programming Methods Laboratory.

The staircase at EPFL, pictured at right, was the inspiration for the Scala logo:

Installation

1. Where is scala installed?

   which scala
   

   The response on a Mac:

   /usr/local/bin/scala
   

2. Find the various scala files on the whole drive:

   sudo find / -name scala-*
   

Interactive Command Line REPL

1. In a Terminal command line window, invoke the Scala run-time console REPL (Read Evaluate Print Loop) Scala’s interactive shell

   scala
   

   The response:

   Welcome to Scala 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_65).
   Type in expressions for evaluation. Or try :help.
   
   scala> _
   

NOTE: Scala runs on the Java platform (Java Virtual Machine).

1. Get a list of REPL commands:

   :help
   

   REMEMBER: The colon precedes commands.

2. Clear the screen using REPL :keybindings on the keyboard. On a Mac, press control + L. On Windows press Ctrl + L.

3. Press keyboard up arrow to retrieve history.

4. Quit back to bash console (like in the vim editor):

   :q
   

   Alternately, press control+C on a Mac to exit the process running Scala.

5. PROTIP: Navigate to the folder you want before entering scala REPL.

Print Line functions and semicolons

1. Enter the scala REPL again at the command line.

2. Print text without line breaks:

   print("Hello, World");print(10)
   

   NOTE: Semicolons (“ugly cockroaches”) are used only with multiple verbs on same line.

   This is an example of how Scala has more syntactic flexibility than Java coding.

3. Print text within REPL:

   println("Hello, World!")
   

   NOTE: Everything returns something (all Scala code is “expression based”).

   More at Scala coding.

4. Exit REPL again.

Clone the GitHub repository for this tutorial

At your operating system shell:

1. PROTIP: Create a workspace folder (such as “gits”) to hold organizations within GitHub and other clouds.

2. PROTIP: Create a folder for each organization, account, or other grouping of repositories (such as “wilsonmar”).

3. At a folder of your choosing, clone the repository referenced by this tutorial:

   git clone https://github.com/wilsonmar/scala.git
   

4. cd within the folder just created:

   cd scala
   

   There are several folders.

5. cd into the folder relevant to the next section of this tutorial:

   cd HelloWorld
   

Shell commands within REPL

1. Open a Scala interactive command window (as explained above) if you’re not already in one.

2. Enable shell processing within REPL by importing the system library:

   import sys.process._
   

3. Issue a bash command to how much disk space is available. On a Mac:

   "df -k" !
   

4. Issue a command to list files in the present directory. On a Mac:

   "ls -al" !
   

We need to ignore stuff around the response, such as:

warning: there was one feature warning; re-run with -feature for details
total 8
drwxr-xr-x   3 mac  staff  102 Mar 11 05:31 .
drwxr-xr-x+ 18 mac  staff  612 Mar 11 05:30 ..
-rw-r--r--   1 mac  staff   63 Mar 11 04:33 HelloWorld.scala
res2: Int = 0

The Int = 0 returned is expected. But the “res2” is a name that changes with each invocation.

1. Alternately, send output into a variable named result:

   val result = "ls -al" !!
   println(result)
   

NOTE: Rembember the double !!.

Notice there is only one file: HelloWorld.scala because files created during compilation are not stored in git, but in another repository such as Artifactory.

1. Confirm your present working directory. On a Mac:

   "pwd" !
   

If your user name is “mac”, then:

   /Users/mac/gits/wilsonmar/scala/HelloWorld

Alternately, you could create the HelloWorld program by following these instructions:

Scala Program HelloWorld.scala

1. Create a folder to hold files for your Scala program source and files created around it.

   mkdir HelloWorld && cd HelloWorld
   

2. Typically this would be under Git version control.

   git init
   

3. Create a text file:

   copy con HelloWorld.scala
   

4. Open a text editor (such as Sublime Text).

5. Copy the code below and paste it in the editor.

   object HelloWorld extends App {
       println("Hello, World!")
   }
   

This program right now only uses built-in println function, so no libraries need to be imported yet.

NOTE: Rather than coding Java classes, Scala programmers code singleton objects. The compiler converts Java classes with only one Java object.

NOTE: Default visibility in Scala is public.

1. Save the file as name HelloWorld.scala.

NOTE: The text editor applies text coloring for Scala.

1. Navigate to the file you just created.

Compile Scala from command line:

Regardless of how you got the HelloWorld.scala program source:

1. In a bash command line at the source folder:

2. Compile it:

   scalac HelloWorld.scala
   

No response is a good response when it comes to this.

Troubleshoot compilation errors

However, if you get this:

<console>:1: error: ';' expected but '.' found.

This is because you’re still in the scala REPL. Exit out to the operating system command line to compile.

View results of compilation

1. List files created during compilation:

   ls -al

   The response:

   • HelloWorld.class
   • HelloWorld$.class
   • HelloWorld$delayedInit$body.class

   NOTE: Compilation creates class files containing Java byte code.

Run scala class file

1. Run the compiled Scala class file so on completion lands within the interactive Scala shell (instead of exiting):

   scala -i HelloWorld.scala
   

   If you see the text specified in the println() function, congratulations.

   NOTE: This is to run Scala script to prepare Scala before presenting the scala prompt.

2. Alternately run by specifying a path:

   scala -i -classpath . HelloWorld
   

3. Run the compiled Scala class file so on completion an exit occurs back to the OS shell after execution.

   scala HelloWorld
   

   NOTE: Such commands are used within shell scripts.

Compile and run a package

1. In a command line window, navigate to the folder containing file named pkg.scala.

2. Edit the file.

NOTE: This file contains Scala code to define a package of functions.

   package com.xyz.app1 {

      object Hello {
         def main(args:Array[String]) {
         println("Hello")
        }
      }
   }
   

1. Exit the editor to compile it:

   scalac pkg.scala
   

   No response is a good response when it comes to this.

2. List files created during compilation:

   ls -al
   

WARNING: Unlike individual source files created in the same folder as the scala code, a new folder “com” has been created.

1. Drill into the folder com. Then into folder “xyz”. Also folder “app1”. Finally two files:

   • Hello.class
   • Hello$.class

2. To run the compiled classes, specify the full path, including an object defined in the code:

   scala com.xyz.app1.Hello
   

If you see the text specified in the println() function, congratulations.

NOTE: on completion an exit occurs back to the OS shell after execution.

SBT (Simple Build Tool)

SBT comes with Scala core (based on Maven).

Docs on it is at:

• http://www.scala-sbt.org/0.13.1/docs/Howto/

1. Install it using Homebrew:

   brew install sbt
   

   The response:

   xcode-select: error: unable to get active developer directory, 
     use `xcode-select --switch` to set one (or see `man xcode-select`)
   ==> Downloading https://homebrew.bintray.com/bottles/sbt-0.13.11.el_capitan.bott
   ######################################################################## 100.0%
   ==> Pouring sbt-0.13.11.el_capitan.bottle.tar.gz
   ==> Caveats
   You can use $SBT_OPTS to pass additional JVM options to SBT:
   SBT_OPTS="-XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=256M"
   This formula is now using the standard typesafe sbt launcher script.
   Project specific options should be placed in .sbtopts in the root of your project.
   Global settings should be placed in /usr/local/etc/sbtopts
   ==> Summary
   🍺  /usr/local/Cellar/sbt/0.13.11: 5 files, 1.2M
   

2. Confirm:

   sbt -version
   

3. Build it

   sbt
   

The command prompt

   Java HotSpot(TM) 64-Bit Server VM warning: ignore option MaxPermSize=256m; 
   port was removed in 8.0

NOTE: The Scala compiler (scalac) was written by an author of the Java compiler.

1. Use the console task:

   console
   

   The response:

   Welcome to Scala version 2.10.3
   

   (or whatever version)

NOTE: Scala compiles into Java. As Android applications are typically supplied as Java bytecode to be translated upon installation.

NOTE: Scala also can compile to JavaScript, making it possible to write Scala programs that can run in the browser.[21]

Akka

Most new computers are being built with multi-core processors for concurrent execution, which is why Scala is built for concurrency.

Scala’s Actors are concurrent isolated units of processing that run in parallel without concern for threads and locking.

The various units communicate by exchanging messages. Actors can also be seen as a form of active objects where invoking a method corresponds to sending a message.

Actors may communicate using futures which handles requests asynchronously, but return a representation (the future) that allows await of the reply.

The Scala Actors library provides both asynchronous and synchronous message sending. (the latter are implemented by exchanging several asynchronous messages).

Resources explaining Akka:

• http://akka.io/ for asynch and background processing

• http://spray.io/ for REST/HTTP servlet container Akka actors.

• Up, Up, and Out: Scaling Software with Akka:

Lightbend provides courses on Akka.

• Advanced Akka focuses on clustering and sharding.

Logback for logging

Logging in Scala can use the Logback framework.

Play! 2 Framework

• http://www.lightbend.com/community/core-projects/play-framework

• https://www.playframework.com/

BTW, version 1 of Play was first published in 2008 by Zenexity headed by Guillame Bort. Play 2 is a fundamentally different than Play! 1. Play! 2 is developed in Scala under Reactive principles (self hosted, stateless, horizontally scalable, async non-blocking).

Play! 2 for Scala 1 hr 37m video course (14 Apr 2014) by James Hughes (http://yobriefca.se and james@yobriefca.se) requires a Pluralsight subscription. The course shows how to build a simple Contacts app using IntelliJ IDEA and command-line commands.

The app’s data is obtained from a model accessing a SQL database using ANORM abstraction.

Play 2 comes with Specs2 and Selenium support.

“Introduction to Play Framework for Java developers” by Lightbend

Introduction the the Play Framework by James Ward (Heroku):

• Asychronous and non-blocking (Google Docs, Trello)
• From vertical scalability to scale horizontally with stateless framework
• API first

The Play Framework at LinkedIn: Productivity and Performance at Scale:

“Node.js v.s. Play Framework” by Yevgeny(Jim) Brikman at ScalaMatsuri 2014

Lightbend/Typesafe Activator

Activator is a server app that aims to be a friendly one-stop-shop to bootstrap Scala, Akka, and Play development.

Such friendliness comes at a price of licensing.

Activator can be used as a wrapper script that launches into traditional command line sbt, but it also includes a template and tutorial system, and an optional GUI for getting started.

Activator Installation

1. On a Mac, instead of downloading http://www.lightbend.com/activator/download

   brew install typesafe-activator
   

2. Create a folder where a new project is created:

   activator new
   

   This can take several minutes, ending with:

   Choose from these featured templates or enter a template name:
   1) minimal-akka-java-seed
   2) minimal-akka-scala-seed
   3) minimal-java
   4) minimal-scala
   5) play-java
   6) play-scala
   

BLAH: This got stuck for me.

1. Create:

   activator ui
   

   BLAH: This did not create assets in my pwd.

   The server is live when you see:

   [info] play - Listening for HTTP on /127.0.0.1:8888

   To stop the server, press control+C.

2. Open another Terminal Shell to enter commands while the server runs.

   which activator
   

   The response:

   /usr/local/bin/activator
   

3. In a browser, use the default port 8888:

   http://127.0.0.1:8888
   

   The Activator UI enables you to switch quickly among code, compile, test, run, and app windows.

   

4. The Tutorial Hello Scala provide apps using basic Scala features.

5. Click Create app for the template to be cloned (downloaded).

Select a template.

1. Specify a folder location.

2. Activate.

lightbend.com/community/core-tools/activator-and-sbt is based on assets at github.com/typesafehub/activator

Play! 2 folders

The default folder structure is similar to .NET MVC with Razor view engine:

• app

  • controllers
    • Application1.scala
  • models
    • Application1.scala
  • Views
    • Global
    • index.scala.html
    • layout.scala.html
    • main.scala.html

• conf contains non-source files for configuration

  • application.conf
  • routes (no file extension)

• project

  • build.properties
  • plugins.sbt

• build.sbt file written in Scala-based DSL.

• public holds static assets:

  • images
  • javascripts
  • stylesheets

• test

  • ApplicationSpec.scala
  • IntegrationSpec.scala

• build.sbt

Layout

   @(message: String, names : List[String] )
   @layout {
      <h1>@html( @message )</h1>
      @nameList(names)
   }
   

The @html processes HTML tags.

Views

The first line of html files contain a block to pass in values:

   @(message: String, names : List[String] )

   <!doctype html>
   <h1>@message</h1>
   

Controller Application.scala

The string is specified in this example:

   package controller

   import play.api._
   import play.api.mvc._

   object Application extends Controller {
      def index = Action {
         val names = List("One","Two","Three")
         Ok( views.html.index("Hello", names ))
      }

      // Establish session cookie to block injection attacks:
      def another = Action { implicit request ==> 
         val initialValue = session.get("counter").map(_.toInit).getOrElse(0)
         Ok(views.html.another()).withSession(
            "counter" -> initialValue + 1).toString
            )
      }
   }
   

Play! uses cookies as default session and flash notification mechanisms.

Define build files

1. In bash command-line console within the custom program folder:

   touch build.sbt
   start build.sbt
   

2. Create a project folder.

3. Edit build.properties file:

   name := "Hello"
   version := "0.1"
   scalaVersion := "2.10.4"
   

   NOTE: The “:=” is Scala’s transformation operator to set a new value or replace existing value.

4. Edit plugins.sbt for use with Eclipse IDE:

   addSbtPlugin("com.typsafe.sbteclipse" % "sbteclipse-plugin" % "2.4.0")
   

5. Invoke to create scaffolding:

   sbt eclipse
   

6. In Eclipse, specify the workspace:

   PROTIP: Use folder ws or something short to differentiate among projects.

7. Create a Scala Worksheet. ???

ScalaTest Styles

http://www.scalatest.org/

Test classes extends a test style class:

• For xUnit, there is FunSuite.
• For BDD, FlatSpec.
• For RSpec lovers, there is FlatSuite.
• For Acceptace testers, FeatureSuite.

After selection, test packages should appear among Refer

Performance Micro-Benchmarking

scalameter.github.io/ by Aleksandar Prokopec of Switzerland ( @alexprokopec, axel22) and clone https://github.com/scalameter/scalameter.git and https://github.com/scalameter/scalameter-examples

Among Alex’s videos is this one showing JVM GC profile:

Profilers and metrics

• YourKit Profiler

• Oracle’s VisualVM is free and often good enough

• The Dropwizard Metrics library collects metrics from the running production systems.

• Metrics are push to Graphite,

• Google Caliper for benchmarking Java code.

• https://www.codacy.com/

Front-end Scala.JS

Scala.js, the Scala to JavaScript compiler. by Sébastien Doeraene in Switzerland (@sjrdoeraene)

Scala.js: Next generation front end development in Scala:

• https://github.com/sbt/sbt-ghpages generates a XSBT project website and pushes to ghpages on GitHub.com.

Mobile

Due to its Java roots, Scala can be used to create Android apps using the Assembla IDE

Google App Engine

Scala works smoothly on Google App Engine.

Spark Big Data

Spark is written in Scala.

How Scala Conquered the Big Data World:

Web Frameworks

Among the votes forTop Web Frameworks for the JVM at InfoQ, Play was identified as being more important and adoption-ready than Lift:

• http://liftweb.net was created by David Pollak.

The open source Play framework was created in 2007 by Guillaume Bort, who sought to bring a fresh web development experience inspired by modern web frameworks like Ruby on Rails to the long-suffering Java web development community.

Play 1.x had scala support via modules.

Play 2 is written completely in Scala with full java support.

Play follows a familiar stateless model-view-controller architectural pattern, with a philosophy of convention over configuration and an emphasis on developer productivity. Unlike traditional Java web frameworks with their tedious compile-package-deploy-restart cycles, updates to Play applications are instantly visible with a simple browser refresh.

• https://www.quora.com/Why-did-Typesafe-select-Play-for-their-stack-instead-of-Lift

Libraries

Scalaz (from this Github) provides purely functional data structures to complement those from the Scala standard library. It defines a set of foundational type classes (e.g. Functor, Monad) and corresponding instances for a large number of data structures.

Scalaz consists of three parts:

• New datatypes (Validation, NonEmptyList, etc)
• Extensions to standard classes (OptionOps, ListOps, etc)

• Implementation of every single general functions you need (ad-hoc polymorphism, traits + implicits)

• Slick

• http://scalatra.org/ web micro-framework for creating REST APIs.

• Logback logging library (16% vs. 5% for Log4j)

• http://liftweb.net/ Lift framework is popular

Among the top 100 libraries on GitHub: http://blog.takipi.com/the-top-100-scala-libraries-in-2015-based-on-64562-github-libraries/

• Spark blows Hadoop out of the water
• https://github.com/twitter/scalding for cascading Hadopp MapReduce jobs.
• https://github.com/adamw/veripacks to verify package specifications
• https://github.com/adamw/elasticmq for message-based queuing
• https://github.com/adamw/macwire for dependency injection.

The ranking named H2 as the most popular database at #21. As a “very fast open source small footprint JBDC API database”, H2 outranked MySQL (#33), and PostgreSQL (#50). MongoDB didn’t make the list for Scala.

Full courses

• https://www.eduonix.com/courses/Software-Development/Learn-Scala-Programming-Language-from-Scratch

Learning Resources for Introduction

In addition to resources noted above,

http://scala-lang.org/contribute/hacker-guide.html The Scala Hacker Guide “covers the entire process, from the conception of your idea or bugfix to the point where it is merged into Scala. Throughout, we will use a running example of an idea or bugfix one might wish to contribute.

http://amzn.to/1JfA1bV

http://www.atomicscala.com/free-sample $25 self-published ebook released March 2015 on Gumroad by Bruce Teckel is about Scala 2.11

http://joelabrahamsson.com/learning-scala/ gives an intro blog to Scala

Pluralsight published several video courses:

• https://app.pluralsight.com/player?course=scala-getting-started by Justin Pihony (JustinPihony.blogspot.com) shows the building of the fileSearcher package.

https://va1.scalacourses.com/ offers several self-paced video courses:

• $495 Introduction to Scala
• $495 Intermediate
• $275 Intro to Play Framework
• $249 Java 7/Scala 2.10 Object-Oriented Interop

http://www.amazon.com/Learning-Scala-Practical-Functional-Programming/dp/1449367933 Learning Scala: Practical Functional Programming for the JVM (1st Edition April 2014) by Jason Swartz

Published by O’Reilly, the book’s page is at: http://shop.oreilly.com/product/0636920030287.do

http://it-ebooks.org/tag/scala

Scala Rock Stars

Here are the most well-known people who are putting stuff out about Scala:

Dick Wall, CTO of Escalatesoft.com and international champion of Scala:

• Video tutorials at https://www.parleys.com/channel/dick-walls-channel is the most thorough series I’ve seen so far.

• Authored book “Programming in Scala”

Martin Odersky (@odersky from Switzerland)

• Scala with Style:

• Scala - the Simple Parts

• Martin conducted on Coursera COURSE: Functional Programming Principles in Scala which is not available now.

• However, still available from May 2015 is Martin’s COURSE: Principles of Reactive Programming

• Book: Programming in Scala: http://www.artima.com/shop/programming_in_scala Second Edition Published December 13, 2010 Errata: http://booksites.artima.com/programming_in_scala

Josh Suereth (@jsuereth)

• Scalawags YouTube channel hangouts

• Google+ https://plus.google.com/u/0/112145465018184674652/posts

• Scala In Depth Developer book

• SBT in action book

• basementcoders.com podcast

• https://github.com/ctataryn/LearningScala contains the PDF of slidedeck

Jonas Bonér (@jboner) is Founder & CTO of Lightbend (Typesafe) and inventor of Akka.

• http://jonasboner.com/

Duncan K. DeVore (@ironfish, VP of Engineering at a power company)

• VIDEO: The Autobiography of Building a Reactive Application at ScalaDays 2014 Berlin

Shadaj Laddad

• shadaj.me
• https://www.parleys.com/tutorial/scala-power-versatility

Alvin Alexander

• OReilly book “Scala Cookbook”
• http://alvinalexander.com/

Viktor Klang, one of the main contributors to Akka and co-author of Scala Future)

• Defends Scala vs. C# https://medium.com/@viktorklang/hi-eef4acf316a8#.ez6xx6gxe

Notifications

• Follow Lightbend on SoundCloud.com for podcasts such as https://soundcloud.com/lightbend/scala-days-2014-git-going-faster-with-scala-roberto-tyley

• Videos at http%3A%2F%2Fwww.parleys.com

• http://reactconf.com/ occured November 20, 2014 in San Francisco. See http://lanyrd.com/2014/reactsf/nov-20/

• ScalaDays (2012, 2013, 2014, 2015)

• Meetup - Bay Area Scala Enthusiast group

• http://www.brianstorti.com/the-actor-model/ The actor model in 10 minutes

• http://fruzenshtein.com/site-map/