Deep Dive on JSON & Python

Coding 103

Mike Maas, Technical Evangelist, IoT, DevNet

DEVNET-2006
Agenda

Overview
Requirements
Concepts JSON / Python
Requesting Content
Parsing Content
Resources
Requirements
Requirements : Tools
A web browser
Chrome is a good choice : http://getchrome.com
Python 3.5+
https://www.python.org/downloads/release/python-352/
Editor
A capable text editor or Integrated Development Environment (IDE)
Or you can use IDLE installed by Python

Git (Optional)
http://git-scm.com/
Postman (Optional)
Get from the Chrome Web Store or the standalone version at http://www.getpostman.com/

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
Requirements : Postman
Pre-Flight Check: Postman
Postman is used to create, send, and explore HTTP requests
Install Postman if you don't already have it installed
This isn't required but it is a very useful tool
Available at http://www.getpostman.com/
If you are used to a different tool (SoapUI, Chrome DevTools, etc.) Feel free

Browser-based version or Standalone packaged application is fine

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
Requirements : Code
All the samples used are available online
GitHub
Get Git from http://git-scm.com/
Get code from https://github.com/CiscoDevNet/coding-skills-sample-code
coding202-parsing-json

To clone locally run the following command:

git clone https://github.com/CiscoDevNet/coding-skills-sample-code.git

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
Optional : JSON Tools
Other online resources for our foray into JSON parsing
JSON Test
http://www.jsontest.com/

JSON Lint
http://pro.jsonlint.com/
JSON Placeholder
http://jsonplaceholder.typicode.com/

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
Concepts
JSON
Concept : JSON
JavaScript Object Notation (JSON) {
A data-interchange text format
"first_name": "Mike",
Really just
Collections of name/value pairs
"last_name": "Maas"
Object {} }
Ordered list of values
Array []

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
Concept : JSON
JavaScript Object Notation (JSON) [
A data-interchange text format
{
Really just
Collections of name/value pairs
"first_name": "Mike",
Object {} "last_name": "Maas"
Ordered list of values },
Array []
{
"first_name": "Matt",
"last_name": "Denapoli"
}
]

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
{ "id": "0001",
"type": "donut",
"name": "Cake",
"ppu": 0.55,
"batters":
{
"batter":
[
{ "id": "1001", "type": "Regular" },
{ "id": "1002", "type": "Chocolate" },
{ "id": "1003", "type": "Blueberry" },
{ "id": "1004", "type": "Devil's Food" }
]
},
"topping":
[
{ "id": "5001", "type": "None" },
{ "id": "5002", "type": "Glazed" },
{ "id": "5005", "type": "Sugar" },
{ "id": "5007", "type": "Powdered Sugar" },
{ "id": "5006", "type": "Chocolate with Sprinkles" },
{ "id": "5003", "type": "Chocolate" },
{ "id": "5004", "type": "Maple" }
]
} http://json.org

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
Python
Concept : Python Syntax Review
Indentation defines scope
Python code blocks do not have explicit statement scope delimiters
Indentation amount is relative
All the typical things you would find in a programming language
Types
Variables and Assignment
Arrays
Statements
Import
Print
Conditional
Looping

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 15
Concept : Python Types Review
Numeric Sequence
int / float / complex Lists
Types that describe numeric content Typically homogeneous sequences of objects
An mutable, ordered array
Boolean Square Brackets []

bool Tuple
Typically heterogeneous sequences of objects
Types that define true/false relationships
An immutable collection
Logical operators: and / or / not Parenthesis ()

String Mapping
str Dictionaries
Immutable string objects A mutable, unordered, associative array
Curly Brackets {}
Single quotes, 1 or 3 apostrophes

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
 Numeric x = 10

type(x) is int # True

Boolean y = False
z = True
y and z

String x = 'Mike'

List my_list = [10, 20, 30]

Data Structures
Tuple my_tuple = (10, 'World', 4.0)

Dictionary my_dict = {"one":1, "two":2}

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
 import json
Import Statements import urllib2

string_variable = "A String"

Variable Assignment
number_variable = 7

if life > death:

Conditional Statements print('Still Alive!')

Method Invocation jsonContent = json.loads(contents)

for apple in basket:

Looping wash(apple)

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
 opened_url = urllib2.urlopen(request)
content = opened_url.read()

Method Chaining - is functionally equivalent to the following -

content = urllib2.urlopen(request).read()

an_array = ['first element', 'second element',

Array Index 'third element']
element = an_array[1]

Multidimensional Arrays
buildings_array['Building']['Floor']

Compound Statements for entry in jsonContent['Device']['entries']:

if entry == "Device007":
print(entry['macAddress'])

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
Requesting Content
Request and Response
What you want; what you get
Request : What you want
Parameters
Response results can often be controlled via Request elements
URL query parameters
For RESTful interactions, this is your best bet
Example: https://developer.cisco.com/site/cmx-mobility-services/documents/api-reference-manual/#get-maps
Form-based transactions (application/x-www-form-urlencoded, multipart/form-data)
HTTP Headers
Pseudo Extensions (https://stream.twitter.com/1.1/statuses/sample.json)
Response : What you get
Content
Can often return the same data in different formats per request
XML, JSON, ATOM,

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
Requesting Content via Postman
Lets get something to look at
From Postman
In the Request field enter our test URL
http://jsonplaceholder.typicode.com/users
Click Send
From a Browser
Just put http://jsonplaceholder.typicode.com/users in as the URL
We should get some JSON returned

http://jsonplaceholder.typicode.com/users
http://jsonplaceholder.typicode.com/todos

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
Understanding the structure of the Response
Lets look at what we got (Condensed)
[
{
"id" : 1,
"name": "Leanne Graham"
},
{
"id": 2,
"name": "Ervin Howell"
}
]
Let's look at this in JSON Lint and then JSON.org.

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
Understanding the structure of the Response
Lets look at what we got (Condensed)
[ (Array)
{ (Object)
"id" : 1, (Value)
"name": "Leanne Graham" (Value)
},
{ (Object)
"id": 2, (Value)
"name": "Ervin Howell" (Value)
}
]

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
Understanding the structure of the Response
Lets look at something else Cisco CMX
In Postman
In the Request field enter our test URL
https://msesandbox.cisco.com/api/contextaware/v1/maps
Add an Basic Auth Authorization header (learning:learning)
Click Send
Again, we should get some data returned

https://msesandbox.cisco.com/api/contextaware/v1/maps

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
Understanding the structure of the Response
Lets look at what we got from the CMX call (Condensed)
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Maps>
<Campus objectVersion="16" name="DevNetCampus">
<Dimension length="81.9" width="307.0" height="10.0" offsetX="0.0" offsetY="0.0" unit="FEET"/>
<Image imageName="domain_1_1421088051901.png"/>
<Building objectVersion="13" name="DevNetBuilding">
<Dimension length="81.9" width="307.0" height="10.0" offsetX="0.0" offsetY="0.0" unit="FEET"/>
<Floor objectVersion="11" name="DevNetZone" isOutdoor="false" floorNumber="1" >

and so on

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
Understanding the structure of the Response
Lets look at what we got from the CMX call as REST (Condensed)
{
"Maps": {
"Campus": [
{
"objectVersion": 16,
"name": "DevNetCampus",
"Dimension": {
"length": 81.9,
"width": 307,
"height": 10,
"offsetX": 0,
"offsetY": 0,
"unit": "FEET"
},
"Image": {
"imageName": "domain_1_1421088051901.png"

... and so on

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
Activity: Request
Requesting Content via Python
Lets get something to look at in our console
Get Code Via Git Get Code Via Download
Git clone the following command: Download the following file:
git clone https://github.com/CiscoDevNet/coding-
https://github.com/CiscoDevNet/coding- skills-sample-code/archive/master.zip
skills-sample-code.git

When you get a coding-skills-sample-code directory/folder

Launch a terminal and navigate to coding-skills-sample-code
Locate the coding202-parsing-json directory/folder
Pass the get-cmx-json.py file to the Python interpreter
Execute python get-cmx-json.py
You may need to run python3 get-cmx-json.py on OS X

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
Understanding the structure of the Response
Lets look at what we got from Python (Condensed) - python get-cmx-json.py
{ (Object)
"Maps": { (Object)
"Campus": [ (Array)
{ (Object)
"objectVersion": 16,
"name": "DevNetCampus",
"Dimension": { (Object)
"length": 81.9,
"width": 307,
"height": 10,
"offsetX": 0,
"offsetY": 0,
"unit": "FEET"
},
"Image": { (Object)
"imageName": "domain_1_1421088051901.png"

..

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 30
Parsing Content
Parsing Content
Methods of Parsing
Don't Parse
Output is truly human-readable
Do-It-Yourself (DIY) String Parsing
Useful for really, really simple string extraction
Build or Use a Framework
A lot of ready-made libraries and tools are already available

Available Libraries
JSON.org has lists of utilities at http://json.org/

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 32
Parsing and Extraction
Syntactic analysis and processing of the result
Understand
The Content
What it represents A user? A book? A log entry? A data sample? An access point?
Schema What is the order, form, and requirements of the data?
Size How much content will you get back? And will that be a problem?
What you need
Names and Email Addresses, X and Y coordinates, Patient ID numbers
How much you need
Do you need a count of items, a representative sample, or everything?

Be aware that there is sometimes more than one path to the content
Different web libraries (urllib, urllib2, Requests, etc.)
Different JSON parsers (JSON, SimpleJSON, UltraJSON, etc.)

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 33
Activity: Parse and Extract
Activity: Parse and Extract
Extract a single piece of information
Open activity-parse-1.py
Review the Code
This is where it is important to know what the content looks like
Access an Array via Index and Key Name
Output via Print
Let's get the location of "Chelsey Dietrich"

Run the code with python activity-parse-1.py (or use python3)

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
Activity: Parse and Extract
Extract multiple pieces of information
Open activity-parse-2.py
Review the Code
More content, More complications
Looping through Collections with for
Access an Array via Index and Key Name
Output via Print

Run the code with python activity-parse-2.py (or use python3)

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 36
Activity: Parse and Extract
Extract and combine multiple pieces of information
Open activity-parse-3.py
Review the Code
Keep what we want and show it how we want to
Looping through Collections with for
Storing Variables
Formatted output via Print

Run the code with python activity-parse-3.py (or use python3)

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 37
Activity: Parse and Extract
Extract and combine multiple pieces of information forming a new Request
Open activity-parse-4.py
Review the Code
Storing relevant information
Python arrays
Querying a source (same or different)

Run the code with python activity-parse-4.py (or use python3)

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
Resources
Resources
Information to help you on your way
Learning Labs
https://learninglabs.cisco.com
Python : Python.org
Python Documentation : https://docs.python.org
JSON : JSON.org
JSON Test : http://www.jsontest.com/
JSON Lint : http://jsonlint.com/
JSON Placeholder : http://jsonplaceholder.typicode.com/

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
Complete Your Online Session Evaluation
Please complete your Online
Session Evaluations after each
session
Complete 4 Session Evaluations &
the Overall Conference Evaluation
(available from Thursday) to receive
your Cisco Live T-shirt
All surveys can be completed via
the Cisco Live Mobile App or the
Dont forget: Cisco Live sessions will be available
Communication Stations for viewing on-demand after the event at
CiscoLive.com/Online

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 41
Continue Your Education
Demos in the Cisco campus
Walk-in Self-Paced Labs
Lunch & Learn
Meet the Engineer 1:1 meetings
Related sessions

DEVNET-2006 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 42
Thank You