Neo4j graph database introduction

Don’t use this if you’re fond of SQL joins and static schemas

Overview

Neo4j and its Cypher language
- Developer Certification
Visual diagrams and ASCII art
Sample graph and code in legacy Console
Use cases in new Sandbox
- Bloom visualization
- Parent/Child in Sample Recommendations
- Prior Cyber Console Online
- Other Use cases in sandbox
Docker Install
Install on Mac using Homebrew
- Pre-requsite Java
- To avoid Desktop database creation error
- Install server
- Cask install Desktop on Mac
- Enable offline mode
- Define Graph database
- Browser
- Install Enterprise Desktop GUI
- Install server CLI on Mac using Homebrew
- Install utilities
- Manual download back version
- Manual install
- Invoke
- Restart server
- Stop server
- Edit Config for upgrade
- Connect to Neo4j
- Start Browser Console
- Ports (within Docker)
- Bolt Network Protocol
- Browser Commands
- Dark theme color for commands
- Play intro
- Monitoring limits
- Manage users and roles
- Change password
Database creation commands
- Movie sample creation
- Northwind sample creation
Load data
- Stop Browser
- Backup default database
- Download sample DrWho database
- Check db consistency
- Use cases
- Graph Gists
- Mental Illnesses
Cypher Query
- Direction
Fast?
Importing data
- Import your own data
- Polyglot
Optimizing Cypher Queries
Programming
Language Extensions
- Python
- Java coded queries
Awesome Procedures
Social
- Live in person
- Rockstars
References and tutorials
Triplestore
- Aggregate functions
- Timings / State
More info
- https://github.com/neo4j-examples
More about Python

NOTICE: Below are my notes on Neo4j I have abandoned because I found the product to be too difficult to figure out for me, with not enough support nor documentation. The killer for me is not being able to get pass the password on initial use.

NOTE: Content here are my personal opinions, and not intended to represent any employer (past or present). “PROTIP:” here highlight information I haven’t seen elsewhere on the internet because it is hard-won, little-know but significant facts based on my personal research and experience.

This tutorial is a hands-on introduction to install Neo4J, configure, create a database from commands,load data, etc. Rather than innudating you with facts and conceptual words to remember, commentary here is provided after you take some action, step-by-step. Like a guided walking tour. So you learn by doing. “PROTIP” flags unique or important information.

The contribution of this article is a maticulously sequenced presentation that curates a concise yet deep tidbits from the many resources about this topic.

Before reading this, read my notes on graph databases.

The great thing about Neo4j is that it can provide future flexibility with multi-platform data access without being locked in. This means if you want to use a different language (such as C#) next year, you can access the same database (using a plugin for that language). If your data volume scales massively, you can easily switch to a cloud-based Neo4j database. That’s not the case with Python data dictionaries. If data expands, you would need to either need to get a larger machine or rewrite to access databases.

There are several ways to use Neo4j:

Online Sandbox showcasing Bloom visualization and other use cases
Install Docker image of server
Install on a Mac using Homebrew:
- Install Desktop using Homebrew
- Install Neo4j server locally using Homebrew
Neo4j’s own Aura cloud offering runs on GCP (Google Cloud Platform).
Install Neo4j server in an EC2 instance

If you’re on Windows, read this.

Neo4j and its Cypher language

Neo4j is a product from Neo Technology, Inc. since 2007.

neo4j.com started as neo4j.org but now Neo4j installers include a Enterprise edition which requires buying a license (unless the application built on top of it is also open-sourced) to unlock limitations, allowing for LDAP role-based and subgraph access control, lock manager, clustering, hot backups of Causal Clustering using Raft protocol, and monitoring. See https://neo4j.com/startup-program/

Flagship customers include* Walmart, eBay, Cisco, Citibank, ING, UBS, HP, CenturyLink, Telenor, TomTom, Telia, Comcast, Scripps Interactive Networks, The National Geographic Society, Airbus, Orange, AT&T, Verizon, DHS, US Army, Pitney Bowes, Vanguard, Microsoft, IBM, Thomson Reuters, Amadeus Travel, Caterpillar, Volvo, and many more.

Neo4j open-sourced (at github.com/neo4j/neo4j) their “Cypher” OpenCypher query language in 2015. Among their docs is a “Refcard” of commands that are declarative (the database figures out how to do the fetching).

Developer Certification

Neo currently sends you a certificate for free if you answer 80% of 80-questions in their 60 minute on-line exam (by classmarker.com). It covers these 105 points by subject category:

44 - Cypher syntax, including creation and querying of data
11 - Developer - Developing with Neo4j
35 - Intro
15 - Modeling (The property graph model)

After going through materials below, sign up and take the exam at neo4j.com/graphacademy/neo4j-certification. At this time, you can take as many tries as you want, so use it to prompt your research.

What about the blue t-shirt?

Visual diagrams and ASCII art

Neo4j databases can be defined as both visual diagrams and by ASCII art.

[23:50] into VIDEO:

Each node can be labeled (such as entity type “Person”), have an identifier (id), and contain properties (attributes).

(graphs)-[:ARE]->(everywhere)

Its relationships from the movie “Matrix”. [13:26]

MATCH path = (:Person)-[:ACTED_IN]->(:Movie)
   RETURN path

MATCH and RETURN are Cypher keywords that perform some action. Ironically, these are in all-caps but can be like “maTch” or “return” (are case-insensitive). However, others are case sensitive.

path is a variable.
:Movie is a node label.
:ACTED_IN is a relationship type.
Parentheses define nodes.
Square brackets provide details such as an action verb.

”->” arrows define one-way relationships between nodes, with Patterns drawn by connecting nodes and relationships.

In VIDEO: What are Graph Databases and Why should I care?, Dave Bechberger (@bechbd of Expero Labs and VIDEO: A Skeptic’s Guide to Graph Databases) explains that with Neo4j, one can easilly “travese” a graph with arbitrary hops such as “similar” to identify a recommandation:

neo4j-lunchbox-hops-866x365-48889

With Neo4j, one can “travese” a graph with arbitrary hops such as “similar” without the need to build foreign key joins or bridge tables.

PLUG: Moreover, as more relationships are added in Neo4j, performance is not degraded with joins.

Sample graph and code in legacy Console

http://console.neo4j.org is a legacy UI that displays a sample graph defined by this Cypher code:

create (Neo:Crew {name:'Neo'}), (Morpheus:Crew {name: 'Morpheus'}), (Trinity:Crew {name: 'Trinity'}), (Cypher:Crew:Matrix {name: 'Cypher'}), (Smith:Matrix {name: 'Agent Smith'}), (Architect:Matrix {name:'The Architect'}),
(Neo)-[:KNOWS]->(Morpheus), (Neo)-[:LOVES]->(Trinity), (Morpheus)-[:KNOWS]->(Trinity),
(Morpheus)-[:KNOWS]->(Cypher), (Cypher)-[:KNOWS]->(Smith), (Smith)-[:CODED_BY]->(Architect)

Note the commas separating commands.

Query:

match (n:Crew)-[r:KNOWS*]-(m) where n.name='Neo' return n as Neo,r,m

Note spaces separating commands.

Use cases in new Sandbox

In a Chrome browser, go to neo4jsandbox.com, which provides built-in guides and sample datasets for popular use cases for a limited time:

Bloom visualization
Select the Bloom “springy” UI for Visual Discovery (a good first one to view)

Neo4j’s Bloom visualization is a web GUI that allows “near natural language” filter text to control what is visualized from the database (docs)

VIDEO preview based on the Beers database.

Near Natural Language Search in Bloom

VIDEOS playlist

Different “Perspectives” show different information depending on permissions level.

VIDEO: Neo4j Bloom Data Visualization for everyone

VIDEO: Ontologies in Neo4j: Semantics and Knowledge Graphs

Parent/Child in Sample Recommendations

In VIDEO: Graph Databases Will Change Your Freakin’ Life Nov 28, 2016, Ed Finkler (CTO of Cloud SaaS vendor GraphStory.com) presents this sample graph:
neo4j-child-of-484x177-27830

The arrow in the relationship line points from the “CHILD_OF” in the direction of the parent. A property of this relationship (named “Created”) is when the relationship was established (in “2002”).

Prior Cyber Console Online

Prior to the Sandbox was http://console.neo4j.org, still available.

Other Use cases in sandbox

Also on neo4jsandbox.com

Spreadsheets Grapher (load Google Sheets)
Graph Algorithms
Legis Graph? (models US Congress bills, members, votes)
Recommendation engines
Crime investigation uses a “POLE” model (Persons, Objects, Locations, Events) applicable to most forensics
Sports (Women’s World Cup 2019)
Paradise Papers by ICIJ (offshore account info leaked) VIDEO: “Graph databases: The best kept secret for effective AI”
Russian Twitter Trolls
Twitter (analysis)
Trumpworld (from Buzzfeed)
Conference (GraphConnect 2018 schedule)
Network/IT management
Blank sandbox
Neo4j 3.3
Neo4j 3.4
Neo4j 3.5.3

Additional use cases from VIDEO: “Neo4j Top Use Cases”, etc:

Geography (Euler’s 7 bridges of Kunisberg)
Internet of Things(IoT)
fraud detection,
Artificial Intelligence
Knowledge graph
Centrality and Clustering
Master data

Episodes of Dr. Who, mentioned by Ian Robertson, Neo4j Lead Engineer.

Docker Install

PROTIP: Alternately, rather than doing the above by hand, I recommend that you create and store in GitHub a shell script that does the above, then execute it a single command.

Create or navigate to an account folder, such as:
```
~/gits/$GITHUB_USER
```

Clone the whole repo, which creates the repo’s folder,

git clone https://github.com/wilsonmar/pyneo.git
cd pyneo

Set permissions, then run neo4j-docker.sh cloned from GitHub:
```
chmod +x neo4j-docker.sh
./neo4j-docker.sh
```
Set permissions, then run it:
```
chmod +x neo4j-docker.sh
./neo4j-docker.sh -a -v
```
-a actually runs the docker run command.
When you see this, you’re within the Docker container:
```
root@db39751a2f11:/var/lib/neo4j#
```
Optionally, you can exit the script here by typing exit.
Set permissions, then run Neo4j’s Cyhper shell:
```
cypher-shell -u neo4j -p test
```
Type in Neo4j Cypher commands as described below.

Optionally, you can exit here by typing :exit, the exit the Docker container with exit.

Alternately, there is a Docker image at https://hub.docker.com/_/neo4j

docker pull neo4j

Install on Mac using Homebrew

PROTIP: Its better to install on you local laptop, which does not expire, as Sandbox instances do after 10 days.

PROTIP: If you’re installing on your laptop, I recommend use of Homebrew for its ease-of-use, even though its version can be behind the official website.

In a Terminal:

Pre-requsite Java

Install pre-requisite AdoptOpenJDK 8 with Homebrew Cask:

PROTIP: Neo4j is multi-platform because it is written in Java.

brew install --cask adoptopenjdk8

Response:

==> Tapping homebrew/cask-versions
Cloning into '/usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask-versions'...
remote: Enumerating objects: 191, done.
remote: Counting objects: 100% (191/191), done.
remote: Compressing objects: 100% (188/188), done.
remote: Total 191 (delta 13), reused 39 (delta 0), pack-reused 0
Receiving objects: 100% (191/191), 82.30 KiB | 4.84 MiB/s, done.
Resolving deltas: 100% (13/13), done.
Tapped 160 casks (208 files, 321.3KB).
==> Downloading https://github.com/AdoptOpenJDK/openjdk8-binaries/releases/downl
==> Downloading from https://github-production-release-asset-2e65be.s3.amazonaws
######################################################################## 100.0%
==> Verifying SHA-256 checksum for Cask 'adoptopenjdk8'.
==> Installing Cask adoptopenjdk8
==> Running installer for adoptopenjdk8; your password may be necessary.
==> Package installers may write to any location; options such as --appdir are i
Password:
installer: Package name is AdoptOpenJDK
installer: Installing at base path /
installer: The install was successful.
🍺  adoptopenjdk8 was successfully installed!

Verify:

java -version

Troubleshoot if you don’t see:

openjdk version "1.8.0_232"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.232-b09, mixed mode)

To avoid Desktop database creation error

Using Mac Finder:

Navigate to the directory where the jre is installed, in my case it was in: C:\Program Files (x86)\Java\jre1.8.0_201\bin 2 - Within this directory create a folder called server 3 - Still in the bin directory, go to the client directory 4 - Copy all content from the client folder into the sever folder 5 - Restart Neo4J Desktop as administrator 6 - Try again to create the database

Install server
From any folder:
```
brew search neo4j
```
The response shows that there is both a cask (UI) and command-line neo4j avialable:
```
==> Formulae
neo4j
==> Casks
neo4j
```
Cask install Desktop on Mac

From any folder:

brew cask info neo4j

neo4j: 1.2.3
https://neo4j.com/download/
Not installed
From: https://github.com/Homebrew/homebrew-cask/blob/master/Casks/neo4j.rb
==> Name
Neo4j Desktop
==> Artifacts
Neo4j Desktop.app (App)

Install the latest version:

brew install --cask neo4j

Updating Homebrew...
==> Downloading https://neo4j.com/artifact.php?name=neo4j-desktop-offline-1.2.3.
######################################################################## 100.0%
==> Verifying SHA-256 checksum for Cask 'neo4j'.
==> Installing Cask neo4j
==> Purging files for version 1.2.3 of Cask neo4j

In the Mac’s Finder app, navigate to your /Applications folder. Scroll to see “Neo4j Desktop”.

PROTIP: Here is where you’ll remove the program by right-clicking and selecting “Move to Trash”.

Alternately, pinch 4 fingers together on the Mac’s Touchpad to swipe until you see the Neo4j logo.
Click on the logo. You should see this:

Enable offline mode
Click the “gear” icon on the left menu.
Check “Enable offline mode.

Define Graph database
Click the “Graph” panel.
Click “Create a Local Graph”.
Click “Graph” next to the database “sandwich” icon.
Replace “Graph” with “Dr.Who” or other name for sample database.
Click in the Set Pasword field and type a password (“password”).
Click “Create”.

Browser
Click “Neo4j Browser” for a pop-up dialog:
Click the documentation icon for a list of Documents to read:
Next, work with databases.

Install Enterprise Desktop GUI

Alternately:

https://neo4j.com/download
Select Save file such as neo4j-desktop-offline-1.2.3.dmg.
From Finder, invoke the installer.
Authorize the Desktop App by copying and pasting the Desktop key and pasting it in the app’s Authorization field.
Create a database per instructions.

Install server CLI on Mac using Homebrew

If you want programs making calls to a Neo4j server:

From any folder, get information about it:

brew info neo4j

PROTIP: Notice in the response “Required: java = 1.8 ✔” that a JVM is a pre-requisite for Neo4j server:

neo4j: stable 3.5.12
Robust (fully ACID) transactional property graph database
https://neo4j.com/
/usr/local/Cellar/neo4j/3.5.12 (141 files, 163.6MB) *
  Built from source on 2019-11-28 at 00:27:37
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/neo4j.rb
==> Requirements
Required: java = 1.8 ✔
==> Caveats
To have launchd start neo4j now and restart at login:
  brew services start neo4j
Or, if you don't want/need a background service you can just run:
  neo4j start
==> Analytics
install: 1,953 (30 days), 5,351 (90 days), 18,320 (365 days)
install-on-request: 1,736 (30 days), 4,684 (90 days), 15,963 (365 days)
build-error: 0 (30 days)

Install the latest version:

brew install neo4j

The response presents tips for starting the server:

==> Downloading https://neo4j.com/artifact.php?name=neo4j-community-3.5.12-unix.
######################################################################## 100.0%
==> Caveats
To have launchd start neo4j now and restart at login:
  brew services start neo4j
Or, if you don't want/need a background service you can just run:
  neo4j start
==> Summary
🍺  /usr/local/Cellar/neo4j/3.5.12: 138 files, 163.6MB, built in 24 seconds

Remember the path above for the forthcoming step:
```
/usr/local/Cellar/neo4j/3.5.12
```

Verify version:

neo4j version</stro ng></pre>

The response at time of this writing was:

neo4j 3.5.12




### Configure Environment Variable

PROTIP: The Summary response provides a hint of where Neo4j's binary is located.

Create an environment in ~/.bash_profile to hold the curerent version:
```
export NEO4J_VER="3.5"
echo $NEO4J_VER
```
Create an environment in ~/.bash_profile to hold a path to the goodies:
```
export NEO4J_HOME="/usr/local/Cellar/neo4j/3.5.12/libexec/"
echo $NEO4J_HOME
ls $NEO4J_HOME
```
The response should be:
```
LICENSES.txt	bin		data		lib		plugins
UPGRADE.txt	conf		import		logs		run
```
Install utilities

Install the Apoc neo4j-contrib library of community-contributed code:
Identify the latest release in community-contributed jar from https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases
Navigate to the plugins folder temporarily to obtain the url to the latest release to download, such as:
```
pushd plugins
curl -O https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases/download/3.5.0.6/apoc-3.5.0.6-all.jar
popd
```
VIDEO: Neo4j APOC Utility Library HowTo Series
Edit $NEO4J_HOME/conf/neo4j.conf
Whitelist only specific procedures and functions (rather than all) to be loaded:
```
dbms.security.procedures.whitelist=apoc.coll.*,apoc.load.*
```
Save and exit the text editor.

Streaming Graph Loading with Neo4j and APOC Triggers by David Allen

APOC Articles by Mark Needham

See https://medium.com/neo4j/efficient-neo4j-data-import-using-cypher-scripts-7d1268b0747

Now skip to the Invoke section.

Alternately:

Manual download back version

You may want to install a back version if you want to follow along an older tutorial: “Introduction to Graph Databases and Neo4j 2h Pluralsight video course by Microsoft MVP Roland Guijt (@RolandGuijt, rmgsolutions.nl) released February 5, 2015 while using Neo4j version 2.1.3 on Windows. So the UI has changed.
In an internet browser, go to Neo4j’s Other Releases to download:

https://neo4j.com/download/other-releases/
Find the link for the version you want to use.
Click the “Download” button for sign-ups
Get the OReilly book.

Manual install

neo4j-desktop-?.?.?.dmg
Click “Open” to “Are you sure you want to open it?”.
Click “I Agree”.
Login using your email or through social media.

NOTE: It says Java 8 is downloaded if it doesn’t exist, but I got errors.
Set the $NEO4J environment variable to point to where Neo4j is installed.

Invoke
For a menu, invoke the neo4j CLI executable without any parameters:
```
neo4j
```
The help response lists the sub-commands:
```
Usage: neo4j { console | start | stop | restart | status | version }   
```

Initially we don’t want/need a background service, so:

neo4j start

If you had installed using Homebrew:

Active database: graph.db
Directories in use:
  home:         /usr/local/Cellar/neo4j/3.5.12/libexec
  config:       /usr/local/Cellar/neo4j/3.5.12/libexec/conf
  logs:         /usr/local/var/log/neo4j
  plugins:      /usr/local/Cellar/neo4j/3.5.12/libexec/plugins
  import:       /usr/local/Cellar/neo4j/3.5.12/libexec/import
  data:         /usr/local/var/neo4j/data
  certificates: /usr/local/Cellar/neo4j/3.5.12/libexec/certificates
  run:          /usr/local/Cellar/neo4j/3.5.12/libexec/run
Starting Neo4j.
Started neo4j (pid 3336). It is available at http://localhost:7474/
There may be a short delay until the server is ready.
See /usr/local/var/log/neo4j/neo4j.log for current status.

Alternately, to start neo4j now and automatically restart at login:

brew services start neo4j

Add sudo before the command if you get this error:

Error: Permission denied @ rb_sysopen - /Users/wilson_mar/Library/LaunchAgents/homebrew.mxcl.neo4j.plist

The expected response (at time of writing):

Password:
Warning: Taking root:admin ownership of some neo4j paths:
  /usr/local/Cellar/neo4j/3.5.12/bin
  /usr/local/Cellar/neo4j/3.5.12/bin/neo4j
  /usr/local/opt/neo4j
  /usr/local/opt/neo4j/bin
  /usr/local/var/homebrew/linked/neo4j
This will require manual removal of these paths using `sudo rm` on
brew upgrade/reinstall/uninstall.
==> Successfully started `neo4j` (label: homebrew.mxcl.neo4j)

Verify:

brew services list

You should see:

Name    Status  User Plist
neo4j   started root /Library/LaunchDaemons/homebrew.mxcl.neo4j.plist

Open in the default internet browser:

open http://localhost:7474/browser/

Restart server

bash ./bin/neo4j restart

Neo4j not running
Starting Neo4j.
Started neo4j (pid 29227). It is available at http://localhost:7474/
There may be a short delay until the server is ready.
See /usr/local/var/log/neo4j/neo4j.log for current status.

Stop server

To get the PID for the Neo4j process, widen the Terminal screen, then:

ps -al

Remember the PID on the line containing this CMD:

/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/bin/java -cp /usr/local/Cellar/neo4j/3.5.12/libexec/plugins:/usr/l

Stop it:
```
brew services stop neo4j
```
Edit Config for upgrade
Within a Terminal, edit the configuration file using “subl” or substituting “vi”, “nano”, or your preferred text editor (Visual Studio Code, which is invoked by the code command):
```
echo "NEO4J_HOME=$NEO4J_HOME"
code $NEO4J_HOME/conf/neo4j.conf
```
See a description of the dozens of keys at: https://neo4j.com/docs/operations-manual/3.5/reference/configuration-settings
Find text “dbms.allow_upgrade=” within the file.
Remove the # comment so your database upgrades automatically in case its built version is older than your current Neo4j version:
```
dbms.allow_upgrade=true
```
Set logging specifications: TODO:
```
???=true
```
Change the password:
```
call dbms.security.changePassword('Pa$$word1');
```
Connect to Neo4j

Database access not available. Please use to establish connection. There’s a graph waiting for you.
Type the default username and password of “neo4j” and “neo4j”.
Click “Connect”.

BLAH

Graph database can store any kind of data using these concepts:

Nodes - graph data records
Relationships - connect nodes
Properties - named data values

https://diseaseknowledgebase.etriks.org/metabolic/browser/

### Query Logging

Query logging must be enabled by setting the parameter to:
```
dbms.logs.query.enabled=true
```
To set all queries to be logged:
```
dbms.logs.query.threshold=0
```
Alternately, set a threshold for the number of seconds before logging, such as 7:

There are additional logging parameters that are false by default:

dbms.logs.query.parameter_logging_enabled=true
dbms.logs.query.time_logging_enabled=true
dbms.logs.query.allocation_logging_enabled=true
dbms.logs.query.page_logging_enabled=true

Save the file and exit the editor.

Start Browser Console

When your Neo4j instance is ready:
Open the Neo4j Browser client:

PROTIP: So you can quickly switch among the two using Command+Tab, open a browser different than the one used to display this tutorial (perhaps Firefox instead of Chrome), then go to the default URL:

http://localhost:7474

From https://neo4j.com/developer/docker-run-neo4j/
```
docker run \
 --name testneo4j \
 -p7474:7474 -p7687:7687 \
 -d \
 -v $HOME/neo4j/data:/data \
 -v $HOME/neo4j/logs:/logs \
 -v $HOME/neo4j/import:/var/lib/neo4j/import \
 -v $HOME/neo4j/plugins:/plugins \
 --env NEO4J_AUTH=neo4j/test \
 neo4j:latest
```
Alternately, when running within Docker Machine, open the Neo4j Browser client using your default browser. On a Mac:
```
open http://$(docker-machine ip default):7474
```
Ports (within Docker)

PROTIP: By default the Docker image for Neo4j exposes three ports for remote access:
```
docker run \
 --publish=7474:7474 --publish=7687:7687 \
 --volume=$HOME/neo4j/data:/data \
 --volume=$HOME/neo4j/logs:/logs \
 neo4j:3.2
 
```
- 7474 for HTTP
- 7473 for HTTPS
- 7687 for Bolt
Bolt Network Protocol

PROTIP: Although Neo4j as a transactional http/https versioned endpoint, the Neo4j database (since v3) by default communicates using the Bolt binary protocol (for multi-cluster routing) based on Packetstream. Much like JDBC for relational databases, Bolt operates over a TCP connection or WebSocket. Built-in Auth and TLS. It’s defined at Bolt network protocol. Connection credentials are stored in your web browser, such as:
```
from py2neo import Database
db = Database("bolt://ws-10-0-1-64-32989.neo4jsandbox.com:443")
```
To use the default bolt:
```
default_db = Database()
```
Confirm:
```
default_db
```
returns:
```
<Database uri='bolt://localhost:7687'.
```
Currently, Neo4j only supports one Graph per Database.

Browser Commands

Commands can be entered in the Editor field at the top which begins with a dollar sign in gray.
Click the top command entry field to the right of the dollar sign, type the first letter of commands, a colon (:), for auto-completion list of most common commands:
```
Click image for larger image pop-up.
 
```
Dark theme color for commands
Click the gear icon near the lower-left corner among menu icons.
```
Click image for larger image pop-up.
```
Click Theme Dark.
Click the icon again to dismiss menu contents. (it’s a toggle)

Play intro
Click the icon that looks like a book (previously this was an i icon for information).
Click “Getting Started”.

Notice that these commands appear in the command field:
```
:play intro
```
Notice that the commands now have different colors.
To submit the command, press Enter or click the arrow icon at the upper-right corner.

The page says for multi-line commands to press Shift+Enter to enter multi-line mode, then press Ctrl+Enter instead of the arrow icon.
Scroll down the page to see that the new response is added above the previous frame, as in a stack as a stream.
Click the gray X to dismiss a content frame.
Click “Operations Manual” under Useful Resources to pop up a new browser tab to the version installed:

https://neo4j.com/docs/operations-manual.

Monitoring limits
Type command :sysinfo and press Enter for:

PROTIP: Execute this on a schedule to ensure that more space is allocated before need.

Manage users and roles

PROTIP: Passwords can be changed in the command line. See: https://neo4j.com/docs/operations-manual/3.2/reference/user-management-community-edition/#userauth-list-all-users-ce

https://neo4j.com/docs/operations-manual/3.2/tutorial/role-based-access-control/

The Enterprise edition of ???
```
CALL dbms.security.listUsers()
```
Change password

Per documentation:

neo4j-admin set-initial-password h6u4%kr

http://boopathi.me/blog/reset-neo4j-graph-database-password/
Notice in the sample console image that the password is blank.
In the Password field, type the default “neo4j” (lower case).
Type your own password. Twice.

Note in the response your user name is “neo4j”.

After desktop install:

Database failed to create: Error: Could not change password https://github.com/neo4j/neo4j/issues/11429

Database creation commands

Neo4j comes with several databases.

Click “Write Code” and press Enter to invoke :play write code.

Movie sample creation
See instructions to build a relationship graph about among actors and directors:
```
:play movie graph
```
The “Create” pane appears with code that begins with command “CREATE”.
Click the code for it to be posted in the command line.
Click the full screen icon.
BLAH: Drag and drop items to arrange the graph to your asthetic taste.

This is like ER (Entity-Relation) diagrams for SQL databases.

PROTIP: Neo4J data is stored the same way as illustrated by the data model, whereas with SQL data is stored in separate tables joined together.
The gray controls appear after you click on an object (the red colored dot labeled “The Polar Express”).
Click the “X” in the wheel to delete the node.

PLUG: Thus, Neo4j is naturally adaptive. Entities can be added dynamically, without schema migrations required in SQL databases.
Click the database icon at the top of the left menu to view a list of
- Node Labels
- Relationship Types
- Property Keys

Create an entry:

CREATE (m:Movie {title:'Mystic River', released:2003})
RETURN m

On Create, time stamp: [40:40]

MERGE (p:Person {name: 'Your Name'})
ON CREATE p.created = timestamp(), p.updated = 0
ON MATCH SET p.updated = p.updated + 1
RETURN p.created, p.updated

Modify an existing entry with a tagline: [37:11]

MATCH (m:Movie {title:'Mystic River'})
   SET m.tagline = 'We bury our sins here, Dave. We wash them clean.'
   RETURN m

Create a relationship: [37:31]
```
MATCH (m:Movie {title:'Mystic River'})
MATCH (p:person {name: 'Kevin Bacon'})
CREATE (p-[r:ACTED_IN {roles: ['Sean']}])->(m)
RETURN p, r, m
```
Northwind sample creation

VIDEO:

PROTIP: Relations are not “first-class citizens” in a relational database. But they are in graph databases.

https://github.com/neo4j-contrib/developer-resources/tree/gh-pages/data/northwind

Open the “Northwind” sample database which Microsoft provides with its SQL database.

:play northwind graph

This is a more complex database with data common to business accounting.

LOAD CSV WITH HEADERS FROM "http://data.neo4j.com/northwind/products.csv" AS row
CREATE (n:Product)
SET n = row,
  n.unitPrice = toFloat(row.unitPrice),
  n.unitsInStock = toInteger(row.unitsInStock), n.unitsOnOrder = toInteger(row.unitsOnOrder),
  n.reorderLevel = toInteger(row.reorderLevel), n.discontinued = (row.discontinued <> "0")

</pre>

LOAD CSV WITH HEADERS FROM "http://data.neo4j.com/northwind/categories.csv" AS row
CREATE (n:Category)
SET n = row

CREATE INDEX ON :Product(productID)
CREATE INDEX ON :Category(categoryID)
CREATE INDEX ON :Supplier(supplierID)

### Indexes

Indexes are used to find the starting point for queries to traverse (where by contrast SQL uses indexes to look up rows in tables)

PROTIP: Cypher keywords ENDS, WITH, and CONTAINS are, as of v3, index-backed. Neo4j uses “index read adjacency” to make itself quicker to traverse nodes instead of slower index lookups in SQL.

To improve performance, indexes in Neo4j can be backed by Lucene index type.

To schedule resampling of an index:
```
db.resampleIndex(":Person(name)")
```

Load data

First, stop the service:

Stop Browser
Open a Terminal window to issue command:
```
neo4j stop
```
The response:
```
Stopping Neo4j.. stopped
```
Now you can backup and dump the database.
Switch to or open a Mac/Linux Terminal instance and:
```
cd $NEO4J_HOME/data/Databases
ls -al
cd graph.db
ls -al
```
Notice graph.db.

PROTIP: In the Neo4j world, a physical database consists of files stored under a folder named with a .db suffix. A “graph” references a physical Neo4j database that stores data.

Backup default database

See https://neo4j.com/docs/operations-manual/3.2/tools/dump-load/ for the various attributes to add.

Download sample DrWho database

With the server stopped:
Open a new browser tab to download a zip file containing a sample database from:

https://neo4j.com/developer/example-data
Right-click to Save Link as… (download) the Jim Webber’s Doctor Who Data Set drwho.zip file to your Downloads folder.
Unzip the file to create folder drwho.
Look into the folder index.

PROTIP: Neo4j uses Lucene to index, same as ElasticSearch and others.

Copy the drwho folder within the Neo4j database folder:

cp ~/Downloads/drwho  /usr/local/Cellar/neo4j/3.3.0/libexec/data/Databases
ls

Check db consistency

See https://neo4j.com/docs/operations-manual/3.2/tools/consistency-checker/

Use cases

Recommendations can be made. You like Tom Hanks? Here are his other movies.

More sophisticated versions of such a database are being used to detect fraud. See https://t.co/OMHaHOrYtq

Video of use cases “What SQL had to process in batch can now be processed in real-time with Neo4j.”

https://www.youtube.com/watch?v=lb90EBfAj0o

https://maxdemarzi.com/2017/11/21/mutual-fund-benchmarks-with-neo4j/

Graph Gists

The developer community’s repository of sample data models and queries for a variety of use cases at portal.graphgist.org is showcased on the neo4j.com/GraphGists web page.

Any developer can create a GraphGist. Reach out to devrel@neo4j.com.

Building a graph of your data is fairly simple as the graph structure represents the real world much better than columns and rows of data.

Mental Illnesses

The Open Source Mental Illness Neo4j database is at: https://github.com/OSMIHelp/osmi-survey-graph

VIDEO: datasets are easily accessible using the blue “Write Code” button under the “Jump into Code” section of the guides.

Cypher Query

https://www.opencypher.org/#resources

A big innovation by Neo4j is that it provides programmers with a flexible network structure of nodes and relationships rather than static SQL tables.

Cypher is a language akin to SQL.

Cypher Style Guide (for M08), a 10-page pdf covers Indentation and line breaks, Meta-characters, Casing, Patterns Spacing. The document presents this classical piece by Mark Needham: as a “sane query”:

MATCH (member:Member {name: 'Mark Needham'})
      -[:HAS_MEMBERSHIP]->()-[:OF_GROUP]->(:Group)-[:HAS_TOPIC]->(topic)
WITH member, topic, count(*) AS score
MATCH (topic)<-[:HAS_TOPIC]-(otherGroup:Group)
WHERE NOT (member)-[:HAS_MEMBERSHIP]->(:Membership)-[:OF_GROUP]->(otherGroup)
RETURN otherGroup.name, collect(topic.name), sum(score) AS score
ORDER BY score DESC

PROTIP: Do not use a semicolon at the end of the statement.

execution plan with EXPLAIN and PROFILE.

https://www.youtube.com/watch?v=1TSBXZMv6tc

https://neo4j.com/download-thanks-desktop/docs/developer-manual/current/cypher/#how-do-i-profile-a-query

Neo4j is considered among other “NOSQL” database tech that include Key-Value Stores, Column-Family Stores, and Document Databases. But these others use aggregate data models whereas graph databases such as Neo4j work with simple records and complex interactions.

Instead of SQL union statements. an example of code is:

(graphs)-[:ARE]->(everywhere)

Direction

Neo4j API allow developers to completely ignore relationship direction when querying the graph.

MATCH (boss)-[:MANAGES*0..3]->(sub),
      (sub)-[:MANAGES*1..3]->(report)
   WHERE boss.name = "John Doe"
   RETURN sub.name AS subordinate,
   count(report) AS Total

https://www.youtube.com/watch?v=NH6WoJHN4UA&index=3&list=PL9Hl4pk2FsvWM9GWaguRhlCQ-pa-ERd4U

Fast?

For many applications, Neo4j offers orders of magnitude performance advantage compared to relational DBs. Each index and join adds to the time needed.

Antlr grammar
EBNF (Extened Backus–Naur Form) metasyntax notation is used to make a formal description of a formal (computer programming) language . “Extended” refers to its pedigree from Niklaus Wirth’s BNF syntax which consists of some of the concepts, but with a different syntax and notation. See https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form
Railroad diagrams https://s3.amazonaws.com/artifacts.opencypher.org/M08/railroad/Cypher.html
Grammar specification
TCK (Technology Compatibility Kit) specification

Importing data

Michael Hungar, Neo4j Developer Evangelist, wrote

Michael started the https://github.com/neo4j-contrib “Apoc” add-on library for Neo4j neo4j-apoc-procedures. It provides hundreds of procedures and functions adding a lot of useful functionality. It is available in all Neo4j Sandboxes and in the Neo4j Cloud. It can also be installed with a single click in Neo4j Desktop. It’s described with videos at neo4j.com/labs/apoc and in the Developer Guide which references https://github.com/neo4j-contrib/developer-resources.

We build something similar to apoc.load.json just for YAML. You can create a GH issue there and ask for guidance.

VIDEO: Ambrose Leung at Microsoft Seattle wrote aka.ms/csv2graph program to load CSV files using API calls.

Click the star icon on the menu to reveal the import area to drag files.

Only text files are dropped there, not images of Neo4j databases (.db files).

Import your own data

http://guides.neo4j.com/fundamentals/import.html

Import data from CSV files using Cypher LOAD CSV command in the developer guide [49:12]
```
[USING PERIODIC COMMIT]  // optional transaction batching
LOAD CSV
WITH HEADERS  // first header row as keys in "row" map
FROM "url"
AS row
FIELDTERMINATOR ";"
...
```
Polyglot

Neo4j believes in polyglot persistence (multiple ways to store connected data), with columnar, tabular and document data stored elsewhere. The various types of data integrations possible with Neo4j is at: https://neo4j.com/developer/integration/

For information about importing transactions into the database, see: https://neo4j.com/docs/operations-manual/3.2/tools/import/ https://neo4j.com/docs/operations-manual/3.2/tutorial/import-tool/

The above links are for a specific version, so can becone outdated.

https://neo4j.com/developer/guide-importing-data-and-etl
```
cycli -f import-file.cypher
```
[41:25]

Optimizing Cypher Queries

Optimizing Neo4j Cypher queries

Programming

A “hello world” example of a @userfunction:

WITH ['Hello','World'] as words
   RETURN apoc.test.join(words,' ')

==> “Hello World”

Example of making a call in Cypher:

CALL
   apoc.index.nodes('User','name:Brook*')
   YIELD node, score
   RETURN node.name, node.age, score

Language Extensions

To extend Cypher with user-defined procedures in JavaScript, C# .NET, Python, and Community drivers such as R, see https://neo4j.com/developer/language-guides

Python

pip install neo4j
pip install -r docs_requirements.txt
make -C docs html

git clone https://github.com/neo4j/neo4j-python-driver
Graph Databases in Python from 2012
https://github.com/neo4j/neo4j-python-driver

Install Python driver:

pip install neo4j-driver

The response (at time of writing):

Collecting neo4j-driver
  Downloading https://files.pythonhosted.org/packages/0e/96/bc81664d87975948713f7e4d3d4c3a21a3a6a813d03a161637573a587817/neo4j-driver-1.7.6.tar.gz
Requirement already satisfied: neobolt~=1.7.15 in /usr/local/lib/python3.7/site-packages (from neo4j-driver) (1.7.16)
Requirement already satisfied: neotime~=1.7.1 in /usr/local/lib/python3.7/site-packages (from neo4j-driver) (1.7.4)
Requirement already satisfied: pytz in /usr/local/lib/python3.7/site-packages (from neotime~=1.7.1->neo4j-driver) (2019.1)
Requirement already satisfied: six in ./Library/Python/3.7/lib/python/site-packages (from neotime~=1.7.1->neo4j-driver) (1.13.0)
Building wheels for collected packages: neo4j-driver
  Building wheel for neo4j-driver (setup.py) ... done
  Created wheel for neo4j-driver: filename=neo4j_driver-1.7.6-cp37-none-any.whl size=32641 sha256=d9193b86dcfc42c27af73bdb026be1290521411e6e96678103fb8b3f0a4b8fa1
  Stored in directory: /Users/wilson_mar/Library/Caches/pip/wheels/8b/bf/07/48400007240b3dbb8bd336fdbba1a99c890144a3ba83fdfc38
Successfully built neo4j-driver
Installing collected packages: neo4j-driver
Successfully installed neo4j-driver-1.7.6

Java coded queries

Neo4j provides Native server-side extensions in Java.

https://neo4j.com/blog/efficient-graph-algorithms-neo4j/

For large amounts of data, Cypher run time performance may be slower than Java API coding of traversals and writes. So do massive writes by writing a Java API program to read and query using parameterized Cypher queries.

Get the official drivers for Javascript, Java, .NET, and Python

https://neo4j.com/docs/developer-manual/current/drivers/#driver-get-the-driver

Additionally, the community has built a wide variety of other drivers in languages like PHP, Ruby, Go, Haskell and more.

https://neo4j.com/developer/language-guides/

https://neo4j.com/docs/developer-manual/current/procedures/

Awesome Procedures

APOC (Awesome Procedures on Cypher) are complex implementations that can’t be expressed directly on Cypher. They support data integration, graph algorithms, data conversion.

https://neo4j-contrib.github.io/neo4j-apoc-procedures/index32.html

refers to the code repository at:

https://github.com/neo4j-contrib/neo4j-apoc-procedures

community.neo4j.com provides weekly news and highlights popular community content.
Sign up for events at neo4j.com/events/world/all
Sign up for http://neo4j.com/slack at http://neo4j-users-slack-invite.herokuapp.com/
Join https://groups.google.com/forum/#!forum/neo4j
Subcribe to Neo4j’s YouTube channel and view videos.
Click the icon at the bottom-left corner among menu icons.

Click image for larger image pop-up.
Q&A on http://stackoverflow.com/questions/tagged/neo4j
Visit a Meetup group - https://www.meetup.com/Neo4j-Online-Meetup/
Tweet to https://twitter.com/neo4j - #GraphViz #Neo4j #GraphDatabases
Read https://neo4j.com/blog/
Read https://en.wikipedia.org/wiki/Neo4j

Live in person

https://www.meetup.com/pro/neo4j/

Travel to the https://neo4j.com/GraphTour
http://datadaytexas.com/2018/graphday (@GraphDay) was Saturday, Jan. 27, 2018 in Austin, TX #ddtx18.

Rockstars

Michael Hunger (@mesiri), Neo4j developer evangelist in Dresden, Germany

Jim Webber provides “koan” style tutorial presents a set of databases which have something not right, so students learn to fix things. Brilliant approach and a great learning tool:

https://github.com/jimwebber/neo4j-tutorial

Ryan Boyd (LinkedIn) a SF-based ex-Googler, now Neo4j Head of Developer Relations.

Combining graph analytics with real-time graph query workloads for solving business problems
5 short videos from March 2016 annotated in the Intro to Graph Databases

William Lyon (@lyonwj, lyonwj.com), software developer on the Developer Relations team, he works primarily on integrating Neo4j with other technologies, building demo apps, helping other developers build applications with Neo4j, and writing documentation. Applying an active learning algorithm for entity de-duplication in graph data.

https://www.slideshare.net/neo4j/building-a-graphql-service-backed-by-neo4j
https://github.com/neo4j-graphql/neo4j-graphql
https://grandstack.io (GraphQL, React, Apollo, Neo4j)

Emil Eifrem (@emileifrem, emil@neotechnologies.com), CEO

Mark Needham and Amy E. Hodler

DOWNLOAD O’Reilly BOOK: Graph Algorthms: Practical examples in Apache Spark and Neo4j
https://neo4j.com/docs/graph-algorithms/current/

Tim Ward (@jerrong, tiw@CluedIn.com)

Using Neo4j and Machine Learning to Create a Decision Engine https://www.slideshare.net/neo4j/graphconnect-europe-2017-using-neo4j-and-machine-learning-to-create-a-decision-engine-cluedin

Mats Rydberg living in Sweden

https://github.com/opencypher Mats-SX
https://github.com/opencypher/openCypher/blob/master/tools/tck/README.adoc

Johan Svensson, CTO

Philip Rathle, products VP

Neo4j, Inc. board of directors includes Rod Johnson (founder of the Spring Framework).

@GraphCurmudgeon

Rik van Bruggen

References and tutorials

https://neo4j.com/graphacademy/online-training/introduction-graph-databases

O’Reilly’s Graph Databases 211 page ebook from May 2015 with NeoLoad 2.2:

https://github.com/graphaware/neo4j-nlp Implementation of Microsoft Concept Graph

https://www.experfy.com/training/courses/an-introduction-to-neo4j#description $80 class

neo4j-hierarchy-graph-1250x476 From https://neo4j.com/blog/7-ways-data-is-graph/

quackit.com/neo4j/tutorial contains lots of ads, but is a nicely formatted tutorial if you don’t mind clicking after reading a few lines.

https://buff.ly/2w9PQFy The Top 13 Resources for Understanding Graph Theory & Algorithms

https://r.neo4j.com/2iSaBRi Geocoding #ParadisePapers Addresses in #Neo4j to Build Interactive Geographical Data Visualizations

https://www.slideshare.net/bachmanm/modelling-data-in-neo4j-plus-a-few-tips neo4j-property-graph-pulp-fiction.png

https://www.slideshare.net/KennyBastani/building-a-graph-based-analytics-platform?next_slideshow=1

https://www.youtube.com/watch?v=78r0MgH0u0w
https://www.youtube.com/watch?v=vJcxRjJ982k
https://www.youtube.com/watch?v=NO3C-CWykkY&index=4&list=PL9Hl4pk2FsvWM9GWaguRhlCQ-pa-ERd4U

Triplestore

Triplestore is based on the semantic web W3C RDF (Resource Definition Framework) for inter-connection

It is based on the subject - object - predicate triples where each property is a vertex (node).

An example is airline flight information VIDEO:

Graph databases are used by inference engines to discover relationship insights not easily found using traditional approaches. Thus, it is popular among (crime) forensic scientists.

Aggregate functions

VIDEO

Aggregates in Cypher do not require a “GROUP BY” keyword as in SQL.

A node and relationship are essentially a container of properties.

See the one-page Cyhper Refcard.

@UserAggregationFunctions need to keep state (the aggregates).

Timings / State

neo4j-timings-850x776

More info

https://github.com/elementsinteractive/flask-graphql-neo4j https://medium.com/elements/diving-into-graphql-and-neo4j-with-python-244ec39ddd94 makes use of docker-compose based on https://hub.docker.com/_/neo4j/

https://github.com/wgwz/flask-neo4j-demo by Kyle Lawlor https://github.com/wgwz/flask-py2neo

https://www.youtube.com/watch?v=78r0MgH0u0w Tips and Tricks for Graph Data Modeling Mar 25, 2015

VIDEO: Introduction to Graph Databases, Cypher, and Neo4j October 2018 by Roland Guijt (@RolandGuijt) in the Netherlands.

https://github.com/neo4j-examples

github.com/neo4j-examples contains several

neo4j-movies-template, presented at http://neo4jmovies.herokuapp.com returns an application error.

https://diseaseknowledgebase.etriks.org/metabolic/browser/

http://boopathi.me/blog/category/neo4j/

Neo4j High Performance (Packt, March 2015) by Sonal Raj covers:

Query Neo4j using Cypher, and optimize your data model and queries to improve Cypher’s performance
Migrate from existing SQL stores and data import/export techniques
Explore the data modeling concepts and techniques associated with graph data in Neo4j
Develop applications with Neo4j to handle high volumes of data
Define how to develop an efficient architecture and transactions in a scalable way
Study the in-built graph algorithms for better traversals and discover Spring-Data-Neo4j
Look under the hood of Neo4j, covering concepts from the core classes in the source to the internal storage structure, caching, transactions, and related operations

5-hr VIDEO: Neo4j Course for Beginners with Java Spring Boot and React. by Farhan Chowdhury and Gavin Lon.

https://www.youtube.com/watch?v=7PKiBX_zMeQ

https://www.youtube.com/watch?v=E3eXSdeNYsk

More about Python

This is one of a series about Python:

Wilson Mar

Neo4j graph database introduction

Neo4j and its Cypher language

Developer Certification

Visual diagrams and ASCII art

Sample graph and code in legacy Console

Use cases in new Sandbox

Bloom visualization

Parent/Child in Sample Recommendations

Prior Cyber Console Online

Other Use cases in sandbox

Docker Install

Install on Mac using Homebrew

Pre-requsite Java

To avoid Desktop database creation error

Install server

Cask install Desktop on Mac

Enable offline mode

Define Graph database

Browser

Install Enterprise Desktop GUI

Install server CLI on Mac using Homebrew

Install utilities

Manual download back version

Manual install

Invoke

Restart server

Stop server

Edit Config for upgrade

Connect to Neo4j

Start Browser Console

Ports (within Docker)

Bolt Network Protocol

Browser Commands

Dark theme color for commands

Play intro

Monitoring limits

Manage users and roles

Change password

Database creation commands

Movie sample creation

Northwind sample creation

Load data

Stop Browser

Backup default database

Download sample DrWho database