Audit Trail in Research Data Management

The idea
The idea of the audit trail to monitor the quality of research data management comes from accountants
auditing financial data. One of the tools they use is to choose a transaction and follow it along a trail
through the financial system. For example, they note a cheque has been paid to X for petrol. They will
check back to make sure the payment was approved by the budget office to be made from that account,
that the claim was approved by the supervisor, that a reasonable receipt was produced and that the
travel was authorised and in line with the travel policy. Of course they dont look at all transactions,
just a sample, but that is enough to identify parts of the financial management system, which are not
working well.
When considering research data management the idea is much the same. Processing data involves
many steps between when an observation is recorded in the field and when a result appears, much
transformed and summarised, in a scientific paper or presentation to the World Bank. Following the
trail of this process will reveal places where errors could arise or inefficiencies be removed.
Following an audit trail
There are two directions in which a trail can be followed:
1.

2.

Forwards. You know data has been collected (for example, completing a farmer survey was
an activity planned for the year and carried out). Choose some raw observations from this
data set and find out what has been done with them. They will be transformed, summarised
and combined with other information to contribute eventually to a substantive conclusion. It
should be possible to follow this process in the way it was originally done.
Backwards. You see a result - perhaps the key numbers in a paper, or the claim in a piece of
PA literature you are reviewing. Find the numbers from which the result was derived, the
source of those numbers and so on back to original field measurements.

Using either method you will be looking for the same sort of things content and process. Auditing the
process includes:

the ways data have been checked and errors identified and corrected;
the ease with which analyses can be repeated;
the extent to which repetitive operations have to be done by hand;

Auditing the content includes:

the way in which comments on the data made in the field have been incorporated into the
analysis;
reasons for data selection and omission;
the sources of supplementary data used in the analysis calibrations, discount factors,
population sizes etc.;
Requirements
It will be much easier to follow an audit trail if the data managers and manipulators prepare for it.

Document what is done. Most of the points below are examples of this.
If an error that cannot be corrected is found, do not simply delete the record from the database.
Instead mark it as missing and add a comment.
Add comments to data files describing how they were checked and edited.
Try to link rather than copy data. Copying means the connection between the original data
and that used for analysis is broken, and it is up to the user to remember. Linking means the
source and methods of selecting the data are tied to the program doing the analysis.
Document all decisions about data selection. Why were some farms ignored or only 3 of 6
districts analysed?

Document the source of all addition information. For example, where do prices, population
sizes, assumed adoption rates, moisture contents, come from?
Keep copies of the programs that do statistical or other numerical analyses. Document the
programs.