What is an API and what might the Discovery API mean for services contributing data to Discovery

Aleks Drozdov
Enterprise Architect

Agenda

API and Data Discovery architecture Getting data into Discovery Discovery data structure Discovery API (Scenarios) Next steps

API and Data

Application Programming Interface A language and message format used by an application program to communicate with the operating system or some other control program such as a database management system (DBMS) or communications protocol.

UI

Personal computer

Application

API

Operating System

Web API
When used in the context of web development, an API is typically defined as a set of Hypertext Transfer Protocol (HTTP) request messages. Response messages, which is usually in an Extensible Markup Language (XML) or JavaScript Object Notation (JSON) format. While "Web API" is virtually a synonym for web service, the recent trend (so-called Web 2.0) has been moving away from Simple Object Access Protocol (SOAP) based services towards more direct Representational State Transfer (REST) style communications. Source: Wikipedia

Web browser

Personal computer

Application

API

Discovery System

Data

Explosion in the amount of data It is common to mashup data from a number of sources User contributed data

Since 80s processor speed increas of 360 RAM price from $1000/MB to $25/GB Reduction in RAM size

1956 disk drive

5MB Refrigerator size 250 pounds

Discovery Architecture

Discovery Architecture

System User Interface

Datasets and data transformations

Object Data Store

Getting data into Discovery

Data import - databases

Relational (SQL) databases

Text input EXCEL TXT CSV

Data denormalisation/ Transformation

Nonrelational Object Data Store

XML XML XML XML

Data import user

Discovery data model

Information Asset
Core of the Discovery is an Information Asset object. Information Asset models The National Archives data in the hierarchical way and uses Collections and parent-child relationship. Properties of the Information Asset object dont change across different data sets and hierarchies. Different data sources are using different schemas, which can lead to missing properties in the Information Assets. Still, the structure of the Information Asset stays unchanged. The main challenge in the design Information Asset object is to find minimum common set of properties which will work across all data sources.
Information Asset

Series
Collection<InformationAsset>

Piece

Information Asset

Information Asset

Information Asset

Collection<InformationAsset> Information Asset

Item

Information Asset

Information Asset

Discovery data - Search

Discovery data - Browse

Discovery data - Details

http://discovery.nationalarchives.gov.uk/SearchUI/details?Uri=C10127419

Discovery API

Operations at http://discovery.nationalarchives.gov.uk/DiscoveryAPI
Uri Method Description

informationasset/{id}

GET

Service at http://discovery.nationalarchives.gov.uk/ DiscoveryAPI/[xml][json]informationasset/{ID} Service at http://discovery.nationalarchives.gov.uk/ DiscoveryAPI/[xml][json]children/{ID}/{PAGE} Service at http://discovery.nationalarchives.gov.uk/DiscoveryAPI/[xml][json]totalchildren/{ID}

Service at http://discovery.nationalarchives.gov.uk/ DiscoveryAPI/[xml][json]Parent/{ID} Service at http://discovery.nationalarchives.gov.uk/DiscoveryAPI/[xml][json]siblings/{ID}/{PAGE} Service at http://discovery.nationalarchives.gov.uk/DiscoveryAPI/[xml][json]search/ /{PAGE}/query={QUERY};all={ALLWORDS};exact={EXACTPHRASE};collection={DEPARTME NTS}&startDate={STARTDATE}&endDate={ENDDATE};formerRef={FORMERREFERENCE}

children/{id}/{page}

GET

totalchildren/{id}
parent/{id}

GET
GET

siblings/{id}/{PAGE}

GET

search/{PAGE}?{query}

GET

Documentation: http://discovery.nationalarchives.gov.uk/SearchUI/api.htm

Information Asset C10127419

http://discovery.nationalarchives.gov.uk/DiscoveryAPI/xml/informationasset/C10127419

API XML response

XML
<IAID>C10127419</IAID> <SourceLevelId>7</SourceLevelId> <ParentIAID>C2389021</ParentIAID> <Reference>1</Reference> <CatalogueId>-5596696</CatalogueId> <Language>English</Language> <LegalStatus>Public Record(s)</LegalStatus> <FormerReferenceDep/> <FormerReferencePro/> <Title/> <MapDesignation/> <CoveringDates>1853 October 8</CoveringDates> <PhysicalDescriptionExtent/> <PhysicalDescriptionForm/> <Dimensions/> <MapScaleNumber>0</MapScaleNumber> <PhysicalCondition/> <HeldBy> <HeldBy> <Corporate_Body_Name_Text>The National Archives, Kew</Corporate_Body_Name_Text> </HeldBy> </HeldBy> { "assetView" : { "IAID" : "C10127419", "SourceLevelId" : 7, "ParentIAID" : "C2389021", "Reference" : "1", "CatalogueId" : -5596696, "Language" : "English", "LegalStatus" : "Public Record(s)", "FormerReferenceDep" : "", "FormerReferencePro" : "", "Title" : "", "MapDesignation" : "", "CreatorName" : [ ], "CoveringDates" : "1853 October 8", "PhysicalDescriptionExtent" : "", "PhysicalDescriptionForm" : "", "Dimensions" : "", "MapScaleNumber" : 0, "PhysicalCondition" : "", "HeldBy" : [ { "Corporate_Body_Name_Text" : "The National Archives, Kew" } ],

JSON

Scenarios

Scenario navigating hierarchy

Search for Information Asset http://discovery.nationalarchives.gov.uk/DiscoveryAPI/xml/search/1/query=C%20203
<SearchResultList> <AssetView> <IAID>C3735</IAID> <CitableReference>C 203</CitableReference> <Title>Chancery: Petty Bag Office: Various Certificates</Title> <Description>Chancery: Petty Bag Office: Various Certificates. Miscellaneous returns and certificates in the custody of the Petty Bag Office, including inquisitions and presentments by jurors concerning the killing, sale or eating of meat in Lent; records of the appointment of collectors of subsidies, possibly connected with their own tax liabilities; documents relating to the sequestration of estates of delinquents; certificates confirming the surrender of offices; certificates recording the oath of allegiance, supremacy and abjuration sworn by attornies; certificates naming those who failed to swear the oaths required by the Security of the Sovereign Act 1714; certificates concerning concealed deaths under the Cestui que Vie Act 1707; certificates attesting that members of Parliament met the property qualification required by the Parliament Act 1710.</Description> <Places /> <People /> <Subjects> <string>C10106 Taxation</string> <string>C10039 Food and drink</string> </Subjects> <StartDate>01-01-1558</StartDate> <EndDate>31-12-1839</EndDate> <References /> <URLParams>066/1/C43/C598/C3735</URLParams> <Department>C</Department> </AssetView>

Scenario navigating hierarchy

Get total number of children

http://discovery.nationalarchives.gov.uk/DiscoveryAPI/xml/totalchildren/C3775

109

Scenario navigating hierarchy

Get all children

http://discovery.nationalarchives.gov.uk/DIscoveryAPI/xml/children/C3735/1
<InformationAssetIdentity> <ObjectId>4f24d14055e0f1922313380a</ObjectId> <IAID>C3413406</IAID> <ParentIAID>C3735</ParentIAID> <SourceLevelId>6</SourceLevelId> <ReferenceComponent>1</ReferenceComponent> <Title>Description available at other catalogue level</Title> <PhysicalDescription> </PhysicalDescription> <CoveringDates /> <HasChildren>true</HasChildren> <ChildrenTotal>0</ChildrenTotal> <ContextPath /> </InformationAssetIdentity>

Scenario - search
http://discovery.nationalarchives.gov.uk/DiscoveryAPI/xml/search/1/query=witchcraft

Build API request

Send request to API

Receive list of results

http://discovery.nationalarchives.gov.uk/DiscoveryAPI/xml/search/2/query=witchcraft

<TotalResults>53</TotalResults> <SearchResultList> <AssetView> <IAID>C2341588</IAID> <CitableReference>DO 119/1373</CitableReference> <Title>Ritual murder and &lt;span class="highlight"&gt;witchcraft&lt;/span&gt;; Rex vs Chiefs Bereng and Gabashane and others</Title> <Description>High Commissioner for South Africa, and High Commissioner for Basutoland, the Bechuanaland Protectorate and Swaziland, and UK High Commissioner for the Union of South Africa: Correspondence. Basutoland. Ritual murder and &lt;span class="highlight"&gt;witchcraft&lt;/span&gt;; Rex vs Chiefs Bereng and Gabashane and others.</Description> <Places /> <People /> <Subjects> <string>C10125 Africa</string> <string>C10117 Witchcraft</string> <string>C10023 Crime</string> </Subjects> <StartDate>01-01-1949</StartDate> <EndDate>31-12-1949</EndDate> <References> <string>9657 II</string> </References> <URLParams>066/1/C86/C495/C6128/C51353/0/C2341588</URLParams> <Department>DO</Department> </AssetView>

Page through results

Retrieve Information Asset data

http://discovery.nationalarchives.gov.uk/DiscoveryAPI/xml/informationasset/C2341588

Next steps

External Data

External Data

External Data

Data Import API Discovery Platform Data Export API User participation API

Data sharing and reuse

Thank you!
http://discovery.nationalarchives.gov.uk http://discovery.nationalarchives.gov.uk/DiscoveryAPI

adrozdov@nationalarchives.gov.uk