Data Access Web Sites

1. General Comments

The controllers of data have power. Often controllers want to keep the power. For example
those who analyze the data and publish first get the credit. Those that understand the patterns
in the data may profit from their knowledge as long as others are kept in the dark.

There may be many problems with data in terms of quality, documentation and formatting.
Substantial work may be required to prepare some data sets for analysis. For example there
may be categorical variables with multiple words or misspellings for the same category label.
Some students have used OpenRefine as clean up tool. (OpenRefine began as Google tool to
address variety of recurrent clean up tasks. However, I recommend starting with “nice” data
for class projects.

Some data sets my not be safe to have on your computer. I like to think that the federal
government data is relative well-structured and safe.

Federal agencies address congressional mandates. Agencies do what they are funded to do.
Politics play a big role in congressional mandates.

Federal agencies have to address the laws addressing confidentiality.

Federal agencies have limited resources to deal with public questions. Someone may ask if the
three occurrences of brain cancer in their community constitute a hot spot? If so, what is
causing this? Many questions arise due to misunderstanding of the data or what can be
inferred. With less data available there are the fewer questions. Once in a while a pattern is
the result of a problem that needs to be addressed.

2. Recent Microsoft links to multiple data sources

Revolution Analytics blog

More Cited by Microsoft

3. Old data access links that many no longer work.

1) Department of Education
National Assessment of Educational Progress
I have often used this web ties to obtain US state data.

2) NCI and CDC Health Data:

State Cancer Profiles
CDC Wonder
 In the past I have used both sites to obtain data.

3) Department Labor Statistics: has many times series

http://beta.bls.gov/maps/cew/us
Provides access to state and county quarterly Census of Employment and
Wages data. Easy to use.

http://www.bls.gov/data/
I used this site a fair amount in past. Directions I prepared for my students
no longer work, but the site still may be useful. BLS seems to have shifted
emphasis from states to metropolitan statistical areas. This may be an
indicator of the future.

4) Bureau of Census:
Census has lots of data and different path to data. Some paths may be disappointing.

America Fact Finder

Data

Economic Indicators

Census population type flow mapper

Info graphics
C-SPAN's America by the Numbers

5) EPA
Data Finder

Toxic Release Inventory

Finding releases for a given location is easy.
Download the TRI.NET software to access data. There is access to data for
state, county and zip code regions. You can enter your street address and
find release sites in a small region that contains your street address.

6) Justice
National Archive of Crime Justice Data
National Institute of Justice: Accessing Data

7) NSF: National Center for Science and Engineering Statistics

NCSES
Example: Business R&D and Innovation 2011
8) US Geological Services: USGS
The national map viewer

9) US Department of Agriculture: USDA

http://datagateway.nrcs.usda.gov

National Agricultural Statistics Service

Economic Research Services

Food and expenditure, nutrition, obesity and other statistics.

10) The Earth

Goddard Space Flight Center:
http://disc.sci.gsfc.nasa.gov/giovanni/overview/index.html

http://disc.sci.gsfc.nasa.gov/AIRS/data-holdings

Likely not suitable for small projects.

Global atmospheric data. Dr. Ashley (Ph.D. 2013), used Matlab to read the HDF4 files
for L3Q* compressed multivariate summaries. R could could read HDF5 files not HDF4

National Oceanic and Atmospheric Administration

National Climatic Data Center

National Snow and Ice Data Center

Note that some files are in HDF5 format. There is a capability in R to read
such data, but using MatLab may be better. One STAT763 student report
difficulty with getting data into shape using R. There were missing data and
other issues to address.

11) World Health Organization:

World Heath Observer Maps
WHO Data Access

12) World Bank

World Bank Data

13) Machine Learning

UCI Machine learning Repository Many data sets here.
14) War
Correlates of War

15) Global Warming

Skeptical Science
Getting skeptical about global warming skepticism
My quick glance at this website did not lead me immediately to data.
It seems the current hot issues still concern the identifiability of the
warming causes.

16) Federal Data

Data.Gov
I have not found much data here of interest to me. I not check recently.