Graph University Data Using D3.

js
Overview
Welcome to TreeHacks 2017! In this hackpack, you will be learning the basics of
D3.js, a popular javascript library used primarily for its effective and powerful
applications to data visualization. Whether you are doing this hackpack as part of
TreeHacks Education Vertical or for other purposes, we hope that you find the
following material educational, interesting, and fun!
After completing this hackpack, you will have used D3 to create a bar chart that
clearly visuallizes average SAT scores for incoming students from various
universities using real data from www.data.gov. Although this may seem like a
superficial use of D3's robust library of data visualization tools, the concepts and
models used in creating this graph underly even the most complex examples of D3
data visualization.
D3 is an important tool used within the education space to visualize important data
collected from various students, schools, school districts, companies, and many
other sources. Before getting started, consider checking out these uses of data
visualization within education that I found to be exemplary, elegant, or just plain
interesting.
http://www.education-inequalities.org/
http://www.humblesoftware.com/demos/trig_d3
http://vallandingham.me/vis/gates/
I got these from D3's examples page on their own website
Also, not necessarily related to education and not exclusively D3, but definitely a
showcase of the power of data visualization, check out one one of my favorite
publications for some truly impressive examples: http://polygraph.cool/

Background Knowledge
Although this hackpack is intended to be accessible to any TreeHacks hacker,
whether they have just taken 106A or already have a few coding internships under
their belt, we have a short list of technologies and programming concepts that we
highly recommend hackers to have a basic understanding of in order to optimally
gain from their experience with this hackpack.

Learn about Javascript

First and foremost, we suggest having a basic understanding of Javascript.
If you've never coded in Javascript before, and you have the time, Codecademy has
an in-depth, easy-to-follow tutorial. It takes about 10 hours
If you're proficcient in several programming languages, or you used to know
Javascript and you just need a refresher on syntax, check out Learn X in Y Minutes.
This tutorial takes about 10 minutes.
If you just want to know the concepts and features of javascript that are used in the
hackpack go to MSN's JavaScript Guide and read the following sections: * Grammar
and Types * Loops and Iteration (were only really using for loops) * Functions *
Working with Objects * Details of the Object Model

What is D3.js?
Check out D3's website to see what they are all about. Dont worry if the specifics
don't make too much sense to you at the moment. However, pay special attention to
the introduction section -- if you don't know what something means (for example
SVG, DOM, or CSS) take a few minutes to look it up. This may help a more complete
picture of D3, along with where it fits within the world of programming, nestle into
your brain.
Its worth noting that D3 has recently released version 4.0. This means that many of
the tutorials and resources online are slightly outdated, with most of the material
I've found being based on version 3.0. Working through these inconsistencies may
initially be cumbersome, but finding the equivalent methods in version 4.0 will help
you understand the library all the better. Here is the D3 4.0 API reference -- you'll
likely use it later in the hackpack.

Getting Started
Download and unzip the starter code.
You should have the following files.
d3-hackpack/
README.md
data
data_for_hackpack.json
images
finessedGraph.png
satScores.png
src

Alternatively, you can use some simple git commands within your terminal to sync
up with our git repository. If you're not familiar with Git, or don't know what the
following lines would do, don't use this method.
from Terminal (or an equivalent command line interface)...

# Clone via HTTPS

$ git clone https://github.com/morhm/d3-hackpack.git
# or SSH
$ git clone git@github.com:morhm/hackpack-chrome-ext.git

The Prologue: Linking up with D3

Before we get into creating our graph, we'll want to establish a connection to D3's
library of tools. To do this, we need to create a simple HTML page that references
the library.
Create a new HTML page within your working directory and paste the following
code into it.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>University Data</title>
</head>
<body>
</body>
</html>

For those of you who aren't familiar with HTML, this is just a skeleton of an HTML
page. If you have never seen HTML before, or are unfamiliar with the concept of a
tag, now is a good time to familiarize yourself with the basics of HTML. A thorough
knowledge of HTML is not necessary for this hackpack -- a working knowledge of
basic concepts will do.
Complete the following steps to link up to D3 --
1. Include a reference to the D3 library
2. create a JS file called school_graph.js and include it in your HTML
file

If done correctly, those two additions will allow you to access D3's library of tools
within your JS file. To test wheter this is true, add the following line of code:
d3.select("body").append("p").text("New paragraph!");

Don't worry about what this means yet, we'll explain it in a second. Open up your
html file in your favorite browswer, and voil! There should be a single line of text
that says "New paragraph". Of course, if this isn't the case, you should open up your
browser's console display using a developer tool (I used inspector in Chrome and
Firebug in Firefox). If an error message is printed there, you've most likely
incorrectly referenced D3's library.
Although you may be able to make out what that line of code did, its important that
you read this page of Scott Murray's D3 guide. Notice that the steps I just had you
complete were identical to those outlined in the guide. Throughout the rest of the
hackpack, this guide will serve as your "text book" that will help you complete tasks
along the way of building your graph.
The final step of the prologue is to become acquainted with the data you will be
using for creating your graph. You may have noticed a directory called data when
you downloaded the source files from Github. Go ahead and navigate to that folder.
Inside you should see a single file called school-data.json. In order to access this
data within your javascript file, you must:
3. Reference the data from your HTML page.

If you accomplished that step, you should now be able to reference that data within
your js file. Go ahead and change the text of your <p> element to something in the
data.
Great! At this point, you are ready to begin using D3's library of data interaction
tools on your very own chunk of school data. Thats it for the prologue!

Part 1: Binding and displaying data the D3 way

Now that we have access to D3 and some data, its time to "stick" the data on to
visible elements (in this case HTML elements). The way that you do this with D3 is
called data binding. Read this section of Scott Murray's guide to learn about D3
data binding.
After finishing that section, you should have a good understanding of D3 selections,
data binding, and how to create data-bound elements. These are some of the most
subtle concepts of D3 -- I personally had trouble understanding what it meant to
"select" elements that don't even exist yet. But the more effort you put into
understanding these concepts, the easier you will find actually implementing the
graph.
Now, you want to go ahead and complete the following steps to bind some data to
the DOM --
1. Within the <body> of your html, make a selection of <p> elements
that are bound to the school data.
2. Set the text of those "p" elements to say whatever you want.

If you correctly completed those tasks, reloading your webpage should now display
a repeating list of a single line of text. However, if you look within your web
inspector and examine each of those lines of text, you will notice that they each
contain a __data__ field with the information stored within its respective section of
your school data.
Of course, this looks nothing like a bar graph and, frankly, is a lame data
visualization by any standard, but the binding of data to elements within the DOM
that you just did is the most powerful feature of D3 by far.
The next step is to actually use the data to express some kind of information
visually. This step shouldn't be too hard once you read the next chapter of Scott
Murray's guide.
After reading this section, you should be familiar with writing your own javascript
anonymous functions, passing parameters to that function, and setting styles and
attributes of databound elements. With these new skills, you should be able to easily
make the following change to your code.
3. Instead of setting the text of those <p> elements to whatever you
choose, instead make them say the name of the school and the average
SAT score of that school using the following template: {name of school}
{average SAT score}.

Great, you've just used your first data source to display information on a webpage
using D3. This isn't a very appealing visualization, however, especially if you're try
to display a lot of data. To make out data visually interesting, we are going to learn
about combining D3 data binding with SVG elements.

Part 2: SVG and D3

The next concept we need to learn in order to create appealing visualizations is SVG.
I found the introduction from W3 School's guide on SVG to be a particularly clear
and succinct explanation of what SVG is:
What is SVG?
SVG stands for Scalable Vector Graphics
SVG is used to define vector-based graphics for the Web
SVG defines the graphics in XML format
SVG graphics do NOT lose any quality if they are zoomed or resized
Every element and every attribute in SVG files can be animated
SVG is a W3C recommendation
SVG integrates with other W3C standards such as the DOM and XSL

The reason why we use SVG elements to generate visuals instead of divs and other
HTML elements is because SVG's library of elements are more robust and easier to
use. Unless you are an SVG expert, go ahead and read Scott Murray's section on
SVG then, when you are done with that, read the following section on using data to
draw SVG elements on your webpage. After reading these sections, I'm sure that you
will have a clearer view of the vast number of possible visualizations that you can
achieve with D3 and SVG.
You will now work on creating your own rudimentary bar chart using SVG elements.
You will no longer be working with <p> elements, but the process of selecting
elements and binding data are the same.
1. Append an SVG element to your body element with a width and height
that can hold your data.
2. Bind your school data to a selection of <rect>'s. Each rect should
be given a class, a height of 30px, a color, and a width that
corresponds to the average SAT score of the university data it is bound
with.
3. Evenly space the <rect>'s apart from each other and have them all
start from the left edge of your SVG canas.
4. Append <text> elements to your SVG canvas to label each bar of the
graph. These text elements should say the name of the university and
the precise value of the average SAT score. Center these labels
vertically within their respective bars.

After this step, your webpage should look something like the following image:
intermediary graph
intermediary graph
Great, now you've got an idea of how to use SVG and data binding to create a bar
chart. However, this visualization wouldn't work very well with larger amounts of
data. Also, its missing an x and y axis -- essential componenets of a bar graph. To
solve this, we will learn about making scales and axes with D3.

Part 3: Making scales and axis

An axis is a visual representation of a scale. To create a scale in D3, we will need to
determine two main things -- the input domain and the output range. Because scales
and axes are so closely related, I found that reading about both of them together was
the best way to solidify my understanding of them -- here is Scott Murray's chapter
on scales, and here is his chapter on axises.
To create your scales and axises for your bar graph, you will need to complete the
following tasks. I won't give you a lot of detail for how to implement them, but I will
provide some helpful tips.
1. Create a scale for the x-axis with a domain whose lower bound is 800
and upper bound is the maximum SAT score of the given data. Then
construct the x axis object and place it at the bottom of all of your
bars.
2. Create a scale for the y-axis with a domain that consists of all of
the school names. Append the y-axis to your svg element with the proper
positioning so that your axis labels will fit.

Tips: * A linear scale won't cut it for the y-axis -- look into D3's other scale types *
Some schools don't release their data on SAT scores -- how will you deal with non-
numeric values? * If you've been using "magic numbers" in your code thus far,
consider which values should become global variables.
Now that you've positioned the axises on the screen, you are ready to use your
scales to map data onto your graph. Remember that your scale object is also a
function!
3. Use your scale functions to correctly set the y position and width
of your bars.

At this point, reloading your page should display what is clearly a bar graph.
However, there are a few final touches that can greatly increase the readability of
this visualization.

Part 4: Final touches

One of the most intersting aspects of data visualization is how clever, technical
manipulation of data is combined with aesthetic choices to present data in an
effective, context-appropriate, and appealing way. The final part of this hackpack is
making small changes in the presentation of the data that will go a long way toward
making the data clearer and more visually appealing.
Here is a picture of my finessed graph.
finessed graph
finessed graph
The improvements that I made are the following --
1. I added a text label to each bar that displays the numerical value
of that data point.
2. I increased the thickness of the axises, the text, and removed the
tick marks from the y-axis

Go ahead an implement those features. You will likely make heavy use of D3's
version 4.0 documentation.
Tips * Centering the bars takes some frustruating arithmetic manipulation -- this is
hard to avoid. However, don't forget to consider the length and height of the text
when centering the text within your bars. * Check out the tickSizeInner method for
axises. * Manipulating colors generally involves messing with RGB values.
Carefully compare your final work to the image of my own from above.
Congratulations! You've completed your first data visualization using D3. The skills
you've acquired to reach this point are a good basis for any other visualizations that
you choose to make with D3. Whether you want to further hone your skills by
polishing this graph, or you would like to move one, we hope that you found this
hackpack a useful exercise along your path to learning skills in the intersection of
education and tech.

Prologue: Wait, there's (a little bit) more!

Obviously, there is a lot more that you can do with D3 that we did not touch upon in
this hackpack. If you want to take your learning further, there are many tutorials out
there, many of which are aggregated on D3's own website.
We hope you found this hackpack useful. Thank you for coming to TreeHacks, we
hope you have a great weekend!
by Mark Orozco and Akshay Ramaswamy