Expert Data Visualization

Jos Dirksen

• Publisher: Packt Publishing
• Year: 2017
• eBook ISBN: 9781786466624
• Edition: 1
• Topic: Design
• Subtopic: Open Source Programming

Breathe life into your data by learning how to use D3.js V4 to visualize informationAbout This Book• Create complex visualizations powered by D3.js and open data.• Provides an extensive set of visualizations that explore all the functionality provided by D3.js V4.• Shows how to set up an easy–to-use environment to create stunning visualizations.Who This Book Is ForThe typical target audience of this book is JavaScript developers, designers, and visual artists who have some basic JavaScript programming knowledge and who now want to master pro-level techniques to create interactive data visualizations using web standards which work on desktop as well as mobile devices.What You Will Learn• Learn how D3.js works to declaratively define visualizations.• Create charts from scratch by using SVG and the D3.js APIs• See how to prepare data for easy visualization using D3.js.• Visualize hierarchical data using chart types provided by D3.js• Explore the different options provided by D3.js to visualize linked data such as graphs.• Spice up your visualizations by adding interactivity and animations.• Learn how to use D3.js to visualize and interact with Geo- and Gis-related information sources.• Create visualization by streaming data over WebSocketsIn DetailDo you want to make sense of your data? Do you want to create interactive charts, data trees, info-graphics, geospatial charts, and maps efficiently? This book is your ideal choice to master interactive data visualization with D3.js V4.The book includes a number of extensive examples that to help you hone your skills with data visualization. Throughout nine chapters these examples will help you acquire a clear practical understanding of the various techniques, tools and functionality provided by D3.js. You will first setup your D3.JS development environment and learn the basic patterns needed to visualize your data. After that you will learn techniques to optimize different processes such as working with selections; animating data transitions; creating graps and charts, integrating external resources (static as well as streaming); visualizing information on maps; working with colors and scales; utilizing the different D3.js APIs; and much more. The book will also guide you through creating custom graphs and visualizations, and show you how to go from the raw data to beautiful visualizations. The extensive examples will include working with complex and realtime data streams, such as seismic data, geospatial data, scientific data, and more. Towards the end of the book, you will learn to add more functionality on top of D3.js by using it with other external libraries and integrating it with Ecmascript 6 and TypescriptStyle and approachThis book will have a real–world, case-study approach, where you will be given data sets from different domains. These data sets will have different visualization goals; some might need 2D or 3D charts, some might need automated workflows, others might require interactive maps. While you fulfill these goals, you will learn different techniques and best practices, which will enable you to perform data visualization tasks on your own

Visualizing Graphs

So far, we've seen a number of different data structures. We've visualized linear data with bar, pie, and line charts, and have shown how to visualize nested data using trees and treemaps. An important data structure that we haven't looked at yet is the graph. With a graph, we have a number of nodes which are connected to each other to form a network. In this chapter, we'll look at a couple of different approaches you can use to visualize these kinds of structures.

In this chapter, we'll show the following D3 supported visualizations:

• We'll start with the force layout. With a force layout, we can create force-directed graphs. In a force-directed graph, the nodes are connected to each other with links. All the nodes and links can exact forces on one another which will result in visually pleasing network of nodes. We will visualize this using data from the Simpsons, where we show characters' interaction with each other per episode.
• We'll make a bubble chart of the most used words of the main Simpsons characters, with the size equivalent to the number of times, and colored per person: https://bl.ocks.org/ctufts/f38ef0187f98c537d791d24fda4a6ef9.
• An alternative way to visualize a network of relations is by using a chord layout. This visualization results in a circular layout, where various parts of the circle are connected with one another to show the relation between the nodes.
• If you've got a large network you need to visualize, a force layout or a chord layout can quickly become unreadable. A static alternative is a matrix diagram. We will use this diagram to visualize the interactions between a very large number of Simpson characters.

The first layout we'll look at is the force layout.

Force layout

A force layout renders a graph as a set of nodes and interconnected links. It's easiest to visualize by looking at an example. The following graph is a very simple example of what you can create with a force layout:

While this is a very simple example, if you have the time, you can create beautiful layouts. An especially beautiful one is the one created by (amongst others) the creator of D3:

You can interact with this visualization here: http://www.nytimes.com/interactive/2013/02/20/movies/among-the-oscar-contenders-a-host-of-connections.html.

For our first visualization in this chapter, we're going to create a force layout that uses information from the Simpsons. We're going to show which characters appear in a specific location in an episode. The final result looks something like this:

Here we have a force layout (which you can see in DVD3/src/chapter-04/D04-01.html) with the following features:

1. We have two different kinds of node. The small circles in the previous figure represent locations, and the large circles shows the characters.
2. The links between a character and a location means that that character, in the specific episode (we used the first one of season one), appeared in that location.
3. When you hover over a node, it'll highlight all the outgoing connections and show the name of the location at the bottom of the screen.
4. And finally, you can reposition nodes by dragging them.

The first thing we need to do is get the data behind this visualization.

Getting the raw Simpsons data

For the visualizations, we're going to work with data from the Simpsons. There is a number of online resources that offer a lot of information regarding this TV series, but getting and converting that data into an easy-to-use form is hard to do and very time consuming. Luckily, somebody has already done the hard work, and provided a way to get access to most of the data we'll be using in this chapter.

Todd Schneider created a GitHub project (https://github.com/toddwschneider/flim-springfield) in which he provides a tool to gather information from a set of online resources and store it in a PostgreSQL database. You can see some of the things he did with this data at http://toddwschneider.com/posts/the-simpsons-by-the-data/. You can either run this for yourself or use the PostgreSQL dump that's provided in the data directory of this chapter. If you want to do this for yourself you can follow the instructions from the GitHub link, and if you want to restore the PostgreSQL dump, just run the following command:

 psql dbname < <DVD3>/src/chapter4/data/simpsons_db.dmp 

Note that it isn't necessary to actually set up the database to work with the examples from this chapter, since we'll use exports from specific SQL queries as input for our data preparations scripts. If you didn't load the data into PostgreSQL, you can skip the following section and go directly to the section titled Converting the data from the database.

Getting the relevant information from the database

When you've either imported the database dump or run the steps from Todd Schneider's GitHub project yourself, we can now export a couple of tables from the database, which we'll process using a JavaScript file, like we did in the previous chapters. We're interested in the characters table, the locations table, and a selection from the script_lines table:

 COPY(SELECT * FROM characters) TO 'characters.csv' 
 WITH CSV DELIMITER ',' HEADER ;
COPY(SELECT * FROM locations) TO 'locations.csv' 
 WITH CSV DELIMITER ',' HEADER ;
COPY(select episode_id, character_id, location_id, raw_character_text, 
 raw_location_text FROM script_lines s1 
 WHERE s1.character_id IS NOT NULL 
 AND episode_id = 1801) TO 'simpsons-s18e01.csv' 
 WITH CSV DELIMITER ',' HEADER ; 

The preceding SQL commands will export the relevant data to a set of CSV files, which we'll process further using a simple JavaScript file, which we run using Node.js. The content of the files looks something like this:

 # characters.csv 
id,name,normalized_name,gender,created_at,updated_at 
7,Children,children,,2016-09-29 15:24:20.372136,2016-09-29 15:24:20.372136 
12,Mechanical Santa,mechanical santa,,2016-09-29 15:24:20.690947,2016-09-29 15:24:20.690947 
... 
 
# locations.csv 
id,name,normalized_name,created_at,updated_at 
1,Street,street,2016-09-29 15:24:20.189358,2016-09-29 15:24:20.189358 
2,Car,car,2016-09-29 15:24:20.213079,2016-09-29 15:24:20.213079 
 
# simpsons-s18e01.csv 
episode_id,character_id,location_id,raw_character_text,raw_location_text 
1801,1,2,Marge Simpson,Car 
1801,2,2,Homer Simpson,Car 
1801,2,2,Homer Simpson,Car 
1801,1,4,Marge Simpson,Auditorium 
1801,2,4,Homer Simpson,Auditorium 
... 

By analyzing the output from this last file, we can determine which character appeared at a specific location. With the raw content extracted, we need to convert it so that we can easily load it into D3 and use it as a basis for our visualization.

Converting the data from the database

To create the visualization we saw at t...

Table of contents

• Title Page
• Copyright
• Credits
• About the Author
• About the Reviewer
• www.PacktPub.com
• Customer Feedback
• Preface
• Getting Started with D3
• Basic Charts and Shapes
• Working with Hierarchical Data
• Visualizing Graphs
• Working with Geo Data
• Visualizing Streaming Data
• Voronoi Diagrams and Heatmaps
• Custom Shapes and Paths and Using a Brush Selection
• ES6, TypeScript, and External D3.js Libraries