International Journal of Food Microbiology 77 (2002) 39 53 www.elsevier.

com/locate/ijfoodmicro

A simple, spreadsheet-based, food safety risk assessment tool

Thomas Ross*, John Sumner
Centre for Food Safety and Quality, School of Agricultural Science, University of Tasmania, GPO Box 252-54, Hobart, Tasmania 7001, Australia Received 21 June 2001; received in revised form 13 November 2001; accepted 18 January 2002

Abstract The development and use of a simple tool for food safety risk assessment is described. The tool is in spreadsheet software format and embodies established principles of food safety risk assessment, i.e., the combination of probability of exposure to a food-borne hazard, the magnitude of hazard in a food when present, and the probability and severity of outcomes that might arise from that level and frequency of exposure. The tool requires the user to select from qualitative statements and/or to provide quantitative data concerning factors that that will affect the food safety risk to a specific population, arising from a specific food product and specific hazard, during the steps from harvest to consumption. The spreadsheet converts the qualitative inputs into numerical values and combines them with the quantitative inputs in a series of mathematical and logical steps using standard spreadsheet functions. Those calculations are used to generate indices of the public health risk. Shortcomings of the approach are discussed, including the simplifications and assumptions inherent in the mathematical model, the inadequacy of data currently available, and the lack of consideration of variability and uncertainty in the inputs and outputs of the model. Possible improvements are suggested. The model underpinning the tool is a simplification of the harvest to consumption pathway, but the tool offers a quick and simple means of comparing food-borne risks from diverse products, and has utility for ranking and prioritising risks from diverse sources. It can be used to screen food-borne risks and identify those requiring more rigorous assessment. It also serves as an aid to structured problem solving and can help to focus attention on those factors in food production, processing, distribution and meal preparation that most affect food safety risk, and that may be the most appropriate targets for risk management strategies. D 2002 Elsevier Science B.V. All rights reserved.
Keywords: Food safety; Hazard analysis; Qualitative risk assessment; Relative risk; Spreadsheet

1. Introduction Formal risk assessment techniques have been developed and exploited in many areas of human enterprise and activity for decades (NRC, 1983, 1994, 1996;
Corresponding author. Tel.: +61-3-6226-1831; fax: +61-36226-2642. E-mail address: tom.ross@utas.edu.au (T. Ross).
*

Morgan, 1993; Bernstein, 1996). The application of risk assessment techniques to food safety issues is being strongly promoted by national and international organisations (CAST, 1994; Kindred, 1996; ILSI, 1996; WHO/FAO, 1999) and several authors have reviewed the application of risk assessment methods to food safety (Jaykus, 1996; Kindred, 1996; Lammerding, 1997; ICMSF, 1998; Voysey and Brown, 2000).

0168-1605/02/$ - see front matter D 2002 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 8 - 1 6 0 5 ( 0 2 ) 0 0 0 6 1 - 2

40

T. Ross, J. Sumner / International Journal of Food Microbiology 77 (2002) 3953

Risk assessment involves the identification of a hazard and the methodical description of a system, and its failures, which could give rise to that hazard, including all possible routes whereby that hazard can arise. This qualitative description can be made quantitative by expressing in mathematical terms, the system, and the relationships between those elements that contribute to the risk. Full quantitative assessment of the risk can be achieved if the distributions of values of the factors in the system that contribute to the risk are known. Approaches that use all of this information, the so-called stochastic, or probabilistic treatments, are the preferred option for risk assessment (Vose, 1996;Cassin et al., 1998a). Methods for microbial food safety risk assessment are being developed by various organisations (FAO, 1995; CAC, 1999; Kindred, 1996; ILSI, 1996; Buchanan, 1997; PCCRARM, 1997) and, since the mid-1990s, a number of microbiological risk assessments have been presented. These were summarised by Schlundt (2000). Others have since been released for public comment/peer review (WHO/FAO, 2000a,b,c, 2001; FDA/FSIS, 2001a,b). The effort expended to assess a specified risk should be commensurate with the magnitude of that risk and, in general, there is a large difference in effort and rigour between qualitative and quantitative risk assessment. The latter are typically expensive, labour intensive and technically demanding processes taking, in some cases, many person-years to complete (FSIS, 1998; FDA/FSIS, 2001a). Despite this, many food safety risk assessments have concluded that there are insufficient data to enable a reliable numerical estimate of risk within narrow confidence limits. Prescreening of the risk by simpler methods can aid decisions about the value of investing resources in fully quantitative risk assessments. Risk managers may have difficulty comparing risks from different sources for risk management prioritisation. The fundamental objective of risk assessment is to provide support for decisions, and there are a number of decision-support tools to assist in determining whether a pathogen is, or could be, an important hazard in a given food/food process combination. These include various semi-quantitative scoring systems, decision trees, etc. (see, e.g. Notermans and Mead, 1996; Todd and Harwig, 1996; ICMSF, 1996; Van Schothorst, 1997). van Gerwen et al. (2000) presented a step-wise approach and developed a computerised

expert system, named SIEFE, for quantitative microbiological risk assessment of food products and processes that begins to address this problem. Schemes to assist qualitative risk assessments have also been developed (Corlett and Pierson, 1992; Huss et al., 2000). While the approaches of Corlett and Pierson (1992) and Huss et al. (2000) are valuable in categorising risk and in directing broad mitigation strategies, neither provides good discrimination of risk (e.g. neither could be used to assess an as-yet undocumented risk), nor of the effect of contributions to risk of individual riskaffecting factors. Consequently, those schemes do not focus attention on steps where control could most effectively be applied. In this paper, we describe a simple and accessible food safety risk calculation tool intended as an aid to determining relative risks from different product/ pathogen/processing combinations, that addresses some of the shortcomings identified above.

2. Methods and materials 2.1. Development of the decision support tool The decision-support tool was developed to assist in translating an academic understanding of the risk assessment approach and philosophy into a useful tool for ranking the risk from different food/hazard combinations. In particular, the tool was intended to make the techniques of food safety risk assessment more accessible to non-expert users, both as a decision-aid and an educational tool. It was recognised that the tool would have to incorporate all factors that affect the risk from a hazard in a particular commodity including: (1) Severity of the hazard. (2) Likelihood of a disease causing dose of the hazard being present in a meal. (3) Probability of exposure to the hazard in a defined period of time. In turn, it was recognised that a number of factors affect each of the above. Disease severity is affected by: (a) the intrinsic features of the pathogen/toxin, and (b) the susceptibility of the consumer.

T. Ross, J. Sumner / International Journal of Food Microbiology 77 (2002) 3953

41

Exposure to the hazard will depend on how much is consumed per meal by the population of interest, how frequently they consume the food, and the size of the population exposed. Probability of exposure to an infectious dose will depend on: (a) serving size, (b) probability of contamination in the raw product, (c) initial level of contamination, (d) probability of contamination at subsequent stages in the farm-to-fork chain, and (e) changes in the level of the hazard during the journey from farm to fork, including, e.g. simple concentration and dilution, growth or inactivation of pathogens. The tool was developed to assist users to describe the product/pathogen/pathway of interest to them. In prototypes, the user was prompted to choose from a list of qualitative answers in response to each of a series of simple questions, so that risk could be estimated or compared without recourse to numerical data. After trials, it was realised that relying on a small and finite range of qualitative answers greatly limited the ability of the tool to discriminate levels of risk. Consequently, the capacity for users to provide numerical answers to some questions was included. 2.2. User interface The user interface represents a generic conceptual model of the factors that contribute to food safety risk. The model was developed in MicrosoftR Excel spreadsheet software, using standard mathematical and logical functions. The List Box macro tool, an inbuilt MicrosoftR Excel function available on the Forms toolbar, was used to automate the conversion from qualitative inputs to quantities for use in calculations. The list box tool allows users to select from a range of options by simply mouse-clicking on their choice. The software converts that selection into a numerical value. The user is required to answer 11 questions, related to the three main factors identified in Section

2.1. The underlying mathematical model equates each descriptor with a numerical value or weighting. The weightings currently used in the model are detailed in Table 1. Some of the weightings are arbitrarily defined, while others are based on known mathematical relationships, e.g. from days to weeks, or years. To assist users to make objective and reproducible responses, and to maintain transparency of the model, examples of the subjective descriptors are provided, or the weighting factors applied to the descriptors are shown in the user interface (see Fig. 1). Alternatively, where the options provided do not accurately reflect the situation being modelled, users can enter a numerical value that is appropriate. Different iterations of the spreadsheet model were tested by food safety managers. Through that process, ambiguities in the structure of the questions were revealed. Thus, the questions were modified to make their intent clearer. Questions 1 and 2 consider the susceptibility of the population of interest and the severity of the illness. The hazard severity is arbitrarily weighted by factors of 10 for increasing levels of severity. The weighting factors for susceptibility of various population subgroups include values for relative risk of infection/ intoxication for a variety of hazards. That weighting is loosely based on epidemiological data for relative susceptibility to listeriosis calculated by Jurado et al. (1993), Jones et al. (1994) and Nolla-Salas et al. (1993). Absolute risk is based on the population size, the proportion of the population consuming the food and how frequently people eat the food. This information is selected in Questions 3 5. The selection of a sub-population from the general population (Question 2) automatically reduces, by the proportions indicated in Table 1, the total population estimated to be exposed. Using Australian population age structure data (ABS, 2000) and 1998 data from the US Center for Disease Prevention and Control (cited in FDA/ FSIS, 2001a) for the proportion of listeriosis cases in defined age categories, we also estimated the relative susceptibility by age. The proportion of the population in these categories in Australia was estimated from ABS (2000), and also by Paoli (1999, pers. comm.) for North American populations. Both estimates were in the range of 15 20%, consis-

42

T. Ross, J. Sumner / International Journal of Food Microbiology 77 (2002) 3953

Table 1 Weighting values used in the current model (V.1) Comment 1. Hazard severity SEVERE hazardcauses death to most victims MODERATE hazardrequires medical intervention in most cases MILD hazardsometimes requires medical attention MINOR hazardpatient rarely seeks medical attention 2. How susceptible is the consumer? GENERALall members of the population SLIGHTe.g., infants, aged VERYe.g., old, very young, diabetes, alcoholic etc. EXTREMEe.g. AIDS, transplants recipients, cancer patients, etc. 1 0.1 0.01 0.001 arbitrary weighting factors

1 5 30 200

100% of population 20% of population 3% of population 0.1% of population arbitrary weightings, but based on relative susceptibility to listeriosis, population estimates based on Australian health statistics

3. Frequency of consumption daily weekly monthly a few times per year once every few years 4. Proportion of population consuming all (100%) most (75%) some (25%) very few (5%) 5. Size of population of interest 6. Proportion of product contaminated? Rare (1 in a 1000) Infrequent (1%) Sometimes (10%) Common (50%) All (100%) OTHER 7. Effect of process The process RELIABLY ELIMINATES hazards The process USUALLY (99% of cases) ELIMINATES hazards The process SLIGHTLY (50% of cases) REDUCES hazards The process has NO EFFECT on the hazards The process INCREASES (10 ) the hazards The process GREATLY INCREASES (1000 ) the hazards 8. Is there a potential for recontamination? NO YESminor (1% frequency) YESmajor (50% frequency) OTHER

365 52 12 3 0.3

simple algebra

1 0.75 0.25 0.05

arbitrary weighting factors

User selected or specified

0.001 0.01 0.1 0.5 1 user input

0.01% of samples 1% of samples 10% of samples 50% of samples all samples

0 0.01 0.5 1 10 1000

arbitrary weighting factors

0 0.01 0.50 user input

arbitrary weighting factors

T. Ross, J. Sumner / International Journal of Food Microbiology 77 (2002) 3953 Table 1 (continued ) Comment 9. How much increase from level at processing is required to reach an infectious or toxic dose for the average consumer? none 1 arbitrary weighting factors slight (10-fold increase) 0.1 moderate (100-fold increase) 0.01 significant (10,000-fold increase) 0.0001 OTHER user input 10. How effective is the post-processing control system? WELL CONTROLLEDsystems in place, audited, well-trained staff CONTROLLEDsystems in place, audited, well-trained staff NOT CONTROLLEDno systems in place, untrained staff GROSS ABUSE OCCURS NOT RELEVANTlevel of risk agent does not change 11. Effect of preparation for meal Meal preparation RELIABLY ELIMINATES hazards Meal preparation USUALLY ELIMINATES (99%) hazards Meal preparation SLIGHTLY REDUCES (50%) hazards Meal preparation has NO EFFECT on the hazards OTHER

43

1 3 10 1000 1

arbitrary weighting factors

0 0.01 0.50 1.00 user-input value

arbitrary weighting factors

tent with that of Lindqvist and Westoo (2000), Hitchins (1996) and Buchanan et al. (1997). The proportions in different susceptibility classes were used to modify the number of cases predicted, as described below. The frequency of contamination of food and the implications of subsequent processing and handling are considered in Questions 6 9 and Question 11. Some factors, such as processing or cooking, may completely eliminate the risk. The model includes, however, the possibility that re-contamination may occur subsequently, and re-introduce risk. Subsequent pathways of cross-contamination are not explicitly considered in the model. Neither the concentration of the hazard nor the size of the serving is considered explicitly in the model. Both factors are included indirectly in the response to Question 10, which requests an estimate of the increase required for the initial contamination level to reach ID50.1 In the calculation of relative risk,

for pathogens believed to have a high infectious doses, the distribution of pathogen loads in the food system has little effect (WHO/FAO, 2001). Rather, it is the total load of such pathogens in the food supply that determines the overall population health risk. The model multiplies the factors to calculate various measures of risk, described below. Some estimates consider only the probability of illness, while others also consider the severity to produce an estimate of the risk of the illness and the numbers of consumers affected. As a descriptor is selected or changed, the risk estimates are automatically recalculated. 2.3. Structure of the tool and mathematical bases Full details of the logic and equations leading to the risk estimates are detailed below. Four measures of risk are calculated. To simplify the description of the calculation of these values, it is useful to describe some intermediate calculations. These are the following. PDD: Probability of a disease-causing dose being present in a portion of the product of interest. This is

1 The dose expected to result in 50% of the exposed population becoming ill.

44

T. Ross, J. Sumner / International Journal of Food Microbiology 77 (2002) 3953

defined as whichever is the larger value of the product of the proportion of product contaminated value of Question 6 the effects of processing on the probability of contamination value of Question 7 the effect of post processing handling=storage value of Question 9 the increase in the initial level of the factor required to reach an infectious dose value of Question 10 the effect of preparation prior to eating value of Question 11 or the proportion of product re contaminated value of Question 8 the effect of post processing handling=storage value of Question 9 the increase in the initial level of the factor required to reach an infectious dose value of Question 10 the effect of preparation prior to eating value of Question 11 The probability of a portion of food being contaminated with a toxic dose cannot exceed 1. Accordingly, if the value of the above calculations exceeds 1, it is set equal to 1. Pexp: Probability of exposure to the product per person per day, given by: the frequency of consumption value of Question 3 proportion of the population that consumes the product value of Question 4 Exposure: Total number of portions of the product of interest eaten per day in the general population, given by: the frequency of consumption value of Question 3 proportion of the population that consumes the product value of Question 4 the total population considered value of Question 5

The first measure: Probability of illness per consumer per day is calculated as: PDD Pexp This value is not strictly a measure of risk, because it does not include the severity of the illness resulting from exposure to the hazard. The second measure Total predicted illnesses/ annum in population of interest does not differentiate severity either, but provides another measure that might be more readily understood. Total predicted illnesses/annum in population of interest is calculated as: 365 i:e: days per year Probability of illness per consumer per day as described above fraction of population considered in at risk class part of Question 2 the total population value of Question 10 The Comparative Risk in the population of interest is a measure of relative risk and is independent of the size of the population, but does consider the proportion of the population consuming. It is calculated as: Probability of illness per day per consumer of interest as described above Hazard severity Question 1 Proportion of population consuming Question 4 Proportion of total population in population of interest part of Question 2 and can be used to rank the relative risk of the pathogen/product/process combination and consumption patterns, independent of population size. When specific sub-populations are selected at Question 2, the estimate of the absolute number of cases among the total population is amended by the weighting factors shown in Table 1 for relative susceptibility to infection, and also the proportion of the total population in that sub-group. The model is constructed, and the weighting factors selected, so that the Comparative Risk can never exceed 1. A Comparative Risk of 1 represents the situation where every person in the population consumes the product of interest daily,

T. Ross, J. Sumner / International Journal of Food Microbiology 77 (2002) 3953

45

and that each portion of the product contains a lethal dose of the hazard. The Comparative Risk measure is cumbersome, and its numerical value is not readily understood as a measure of risk. Relatively small changes in one of the answers can lead to alarming changes in the predicted number of cases. Furthermore, the specification of the numerical value of risk is misleading, as it provides no information regarding the confidence one should place in that numerical estimate. To provide a more user-friendly and robust index of relative risk, the Risk Ranking measure was developed based on the Comparative Risk estimate. The Risk Ranking value is scaled logarithmically between 0 and 100, where 0 represents no risk, and 100 represents the opposite extreme where every member of the population eats a meal that contains a lethal dose of the hazard every day. To set the Risk Ranking scale, we chose a probability of mild food-borne illness of less than or equal to one case per 10 billion people (greater than current global population) per 100 years as a negligible risk. The Comparative Risk estimate that corresponds to this value is 2.75 10 17. We equated the Risk Ranking corresponding to this level to zero. Analogously, we set the upper limit of Risk Ranking at 100, corresponding to a Comparative Risk of 1. All the estimates generated by the model are based on the multiplication of factors, many set at factor of 10 differences. The end-points of the Risk Ranking scale lead to an increment of six Risk Ranking units, corresponding approximately to a factor of 10 difference in the absolute risk estimate. Thus, Risk Ranking is defined as: If Comparative Risk Q 2.75 10 17 then Risk Ranking = 0 or else Risk Ranking=(100/17.56) (17.56 + LOG10(Comparative Risk)). In the spreadsheet, the result is rounded to the nearest integer value, reflecting the level of discrimination we believe appropriate given the bases of the tool. 2.4. Evaluating the tool To relate the Risk Ranking scale to practical experience, we use predicted rates of food-borne illness

in Australia, estimated by ANZFA (1999), and the estimates of Mead et al. (1999) for food-borne illness in the USA, to generate Risk Ranking values. To evaluate the performance of the tool, several scenarios were modelled and compared to actual data or other risk assessments. Specifically, conditions leading to an outbreak of hepatitis A from consumption of oysters in Australia in 1997 were simulated using the tool, and compared to the epidemiological data reported by Conaty et al. (2000). Secondly, the data and assumptions of the quantitative risk assessment of Cassin et al. (1998b), for the risk of illness from enterohaemorrhagic E. coli in hamburgers in north American culture, were used to derive the answers to the questions of the risk assessment spreadsheet. The results of both assessments were compared.

3. Results The model interface is shown in Fig. 1. 3.1. Risk ranking ANZFA (1999) calculated the incidence of foodborne disease in Australia as f 4,000,000 cases per annum. The vast majority of these cases were considered to pass unreported. Thus, we set Hazard Severity (Question 1) to minor hazard. The ANZFA (1999) estimates are for the total population: we set Question 2 to general. We manipulated other inputs so that the Total Predicted Illnesses per annum in the Population of Interest equalled f 4,000,000. Australia was selected in Question 5. Under these, and all other conditions leading to a total predicted 4,000,000 minor food-borne illnesses among the Australian population of f 20 million, the Risk Ranking was 64. Mead et al. (1999) estimated that there were 76,000,000 cases of food-borne disease per year in the USA, of which 325,000 resulted in hospitalisation and 5000 caused deaths. Thus, we performed three assessments. In the first, the Hazard Severity (Question 1) was set to minor hazard and the other questions adjusted to yield an estimate of 76,000,000 illnesses. In the second assessment, Hazard Severity (Question 1) was set to moderate hazardrequiring medical intervention in most cases, and the other questions manip-

46 T. Ross, J. Sumner / International Journal of Food Microbiology 77 (2002) 3953 Fig. 1. User interface. The risk model user interface, using Australian populations as an example. Users mouse-click on their choice in each list box or provide numerical values as required. As choices are made and values entered, the risk estimates are automatically recalculated. The values shown are those used in Case Study 1.

T. Ross, J. Sumner / International Journal of Food Microbiology 77 (2002) 3953

47

ulated to yield an estimate of 325,000 illnesses. In the third, the Hazard Severity was set to severe hazardcauses death to most victims, and the other questions manipulated to yield an estimate of 5000 cases. In all cases General population was selected in Question 2, and other selected under Question 5, with the USA population estimated at 270,000,000. The Risk Ranking estimates for these three scenarios were 65, 63 and 58, respectively. 3.2. Case study 1: viruses in oysters An outbreak of hepatitis A involving consumption of oysters occurred in Australia in 1997. The outbreak is discussed in Conaty et al. (2000) from which it can be estimated that production from the affected area was approximately 280 bags of oysters per week. A bag of oysters contains approximately 200 serves of six oysters. The first positive samples were identified from oysters harvested on 24 December and the area was closed to further harvest on 14 February, indicating that contaminated oysters were harvested for up to 7 weeks. Thus, we estimate that the population was exposed to 390,000 servings of potentially contaminated oysters. If spread over the entire Australian population over an entire year, this would correspond to 0.02 serves per person per annum. To represent this level of exposure in the model we selected Once every few years at Question 3, and Very Few at Question 4. (Note that, even though the exposure occurred only over a 7-week period, we assume that it was spread over an entire year, and that even though the exposure was restricted to a geographic region, that all Australian consumers were potentially exposed, consistent with the above estimate of exposure level.) Conaty et al. (2000) report that of 63 samples of one dozen oysters, 6 tested positive for hepatitis A virus using a PCR method. From this, we assumed that 5% of servings of six oysters were HAV-positive (Question 5). The level of contamination was not reported. Where detection of enteric viruses in shellfish has occurred the levels of contamination are low, ranging from 0.3 to 200 plaque forming units (PFU) per 100 g of shellfish (Jaykus et al., 1994; Rose and Sobsey, 1993; CAST, 1994), a typical serving size. Rose and Sobsey (1993) presented a dose response model

relating the probability of infection with HAV to the amount ingested. It suggests that the ID50 for HAV is f 500 PFU. Assuming an exponential dose response relationship, 1 PFU would be expected to lead to infection in 1 in 500 consumers. Thus, it appears likely that not all contaminated serves would have a high probability of causing infection. To implement this relationship at Question 7, we selected OTHER and entered 65 (the ID50 divided by the geometric mean of the contamination per serving) at Question 10. The values used are summarised in Table 2. Australia-wide during the outbreak period (20 January to 4 April), there were 444 cases of hepatitis A associated with consumption of oysters, the vast majority of which were believed to be due to oysters from the contaminated area (Conaty et al., 2000). Under the assumptions given above, the number of cases predicted by the spreadsheet model was 225, and the Risk Ranking was 52. 3.3. Case study 2: comparison to risk estimate of Cassin et al. (1998b) Cassin et al. (1998b) developed a process risk model from which to determine the effectiveness of a range of strategies to reduce the risk and incidence
Table 2 Values used in the assessment of risk from viral contamination of oysters in Australia in an outbreak Risk criteria Dose and severity 1. Hazard severity 2. Susceptibility Probability of exposure 3. Frequency of consumption 4. Proportion consuming 5. Size of population Probability of infective dose 6. Probability of contamination 7. Effect of Process 8. Possibility of recontamination 9. Post-process control 10. Increase to infective dose 11. Effect of treatment before heating Input appropriate to outbreak Moderateoften requires medical attention Generalall population

once every few years very few (5%) Australia (19,500,000)

Other (5% of servings) Has no effect on the hazard None Not relevant other (65) Not effective in reducing hazard

48

T. Ross, J. Sumner / International Journal of Food Microbiology 77 (2002) 3953

of E. coli O157:H7 infections from hamburgers in the North American cuisine. It was difficult to make a direct comparison with the results of Cassin et al. (1998b). Many of the values that were required to be entered in the spreadsheet were not given explicitly by those authors, but were intermediate calculations in their model. However, using data and statistics for the general population presented in that report, we entered the values shown in Table 3 into the spreadsheet. We considered the total population, for whom infection with E. coli O157:H7 will generally cause mild disease (Questions 1 and 2). For more susceptible individuals, medical attention will be required. The population of the USA is approximately 270,000,000 (Question 5). Walls and Scott (1996) reported that on any day in the USA, 9% of the population consume a hamburger meal or, equivalently, that in any week 63% of the population eat a hamburger meal. We entered this as most people consume a hamburger weekly (Question 4). It was difficult to determine the predicted level of contamination during processing. We estimate a contamination rate on carcass meat of < 1%. Cassin et al. (1998b) discussed various factors that affect the contamination rate during the processing of meat, and concluded that overall, a reduction in the initial conTable 3 Values used in the assessment of risk from enterohaemorrhagic E. coli in hamburgers in north America Risk criteria Dose and severity 1. Hazard severity 2. Susceptibility Probability of exposure 3. Frequency of consumption 4. Proportion consuming 5. Size of population Probability of infective dose 6. Probability of contamination 7. Effect of process 8. Possibility of recontamination 9. Post-process control 10. Increase to infective dose 11. Effect of treatment before heating Input Moderateoften requires medical attention Generalall population

weekly most Other (270,000,000)

Other (1% of servings) Usually eliminates the hazard Other (2.9%) Controlled Other (1000) Other (0.05)

tamination of between 10 and 300 could be expected. We chose The process usually eliminates. . . at Question 7. However, in their calculations, Cassin et al. (1998b) predicted that 2.9% of the packages of retail ground beef of size 300 1000 g are contaminated. We implemented this directly at Question 8 which overrides the contamination changes predicted from the answers to Questions 6 and 7. The geometric mean of the contamination levels estimated by Cassin et al. (1998b) is f 20 CFU/ package. The average serving size is 83 g for adults. Thus, based on the average package size, average serving size and average contamination level, we estimate the average CFU/meal serving as 3 at retail. We have assumed that the USA has good temperature control and handling systems for raw meat and have selected Controlled for Question 9. By analogy with shigellosis, the ID50 for O157:H7 was estimated by Cassin et al. (1998b) as f 2000 CFU. Thus, we assumed that a f 1000-fold increase in dose would be required to cause infection in the average case. Cassin et al. (1998b) cited the results of MacIntosh et al. (1994) for hamburger cooking preference. Nineteen percent of consumers were reported to prefer rare or medium rare cooked meat products. We assume that all other degrees of cooking result in the elimination of the pathogen, and that of the remaining 20% preferring less thoroughly cooked meat, the cooking eliminates 75% of the pathogens present. We implement this as cooking eliminates 95% of the pathogens present in all hamburgers consumed (Question 11). The values used are summarised in Table 3. The model predicts a per-meal risk of 6.2 10 7, and predicts 45,800 cases per year in the USA. Cassin et al. (1998b) estimated the risk per meal to be 3.7 and 5.5 10 5 for children and adults, respectively, from their model. On the assumption that half of the 10,000 20,000 cases annually of E. coli O157:H7 illness in the USA are due to consumption of hamburgers, Cassin et al. (1998b) estimated the per-meal risk at between 5.7 10 7 and 1.2 10 6. The Risk Ranking estimate is 58.

4. Discussion The spreadsheet tool was originally developed as a means of quickly assessing the performance of various

T. Ross, J. Sumner / International Journal of Food Microbiology 77 (2002) 3953

49

conceptual models for food safety risk assessment. It was quickly realised that the spreadsheet itself was a valuable risk assessment and risk communication tool, and the conceptual model and the spreadsheet user interface then were developed in parallel. Refinement of the conceptual model was based largely on experimentation with the model. This involved trying to recreate scenarios for which data to describe the model inputs, and epidemiological data by which to evaluate the corresponding model outputs, were available. For example, early iterations of the model required a number of correction factors so that numerical predictions of cases of illness matched those reported. Other risk assessment models have used such factors to make the predicted number of cases better match the observed rates of illness (Farber et al., 1996; FDA/ FSIS, 2001a). Experience with the model enabled the refinement of the questions posed and data required so that the correction factors were eliminated from the model. Elimination of correction factors is important because a major tenet of the risk assessment approach is that all assessments should be transparent, i.e. the basis of all calculation should be made explicit (CAC, 1999). The spreadsheet interface has also been improved through feedback from a diverse range of users. We emphasise, however, that while the tool is presented as an example of how food safety risk assessments can be simplified and its benefits made more accessible to risk managers, the tool is not definitive. It can still be improved, and cannot be expected to be appropriate to all food safety risk assessment problems. We discuss some of the shortcomings and tangential benefits of the model below. 4.1. Evaluation of performance We compared the predictions of the model to independently obtained epidemiological data and estimates for food-borne illness in Australia and USA to calibrate the Risk Ranking value. The estimates obtained suggest that Risk Ranking in the range 60 65 describes the status quo for all microbial foodborne disease in Australia and USA. We consider those to be representative of many developed nations. This gives a reference point from which to evaluate Risk Ranking values for other product/hazard/pathway combinations. It should be noted that the Risk

Ranking is independent of population size, but reflects the relative risk to an individual within the selected population. Thus, the Risk Ranking can be used potentially to compare the risks across diverse foods, hazards and cultures. The USA data enabled the Risk Ranking to be estimated from different disease end-points (e.g. estimated total illness, estimated hospitalisations, estimated deaths) and revealed that the Risk Ranking value depended on the end-point chosen. Perhaps surprisingly, then, the Comparative Risk estimated from the USA fatality estimates was 10-fold lower than the Comparative Risk estimates based on total estimated cases or total cases requiring hospital treatment. In the conceptual model underpinning the tool, the weighting applied for disease severity (arbitrarily) assumed death to be 1000 times more serious than a mild case of illness not requiring medical attention. It is clearly difficult to deduce an objective, quantitative, measure to compare the severity of death to that of mild food-borne illness. The Risk Ranking values based on USA data but using different disease end-points suggest, however, that the weighting factors for illness severity used in the model are inappropriate (see discussion further on). The prediction of the model for a scenario based on a food-borne disease outbreak in Australia was 225 cases, within a factor of two of the observed number of cases ( f 440). The inputs to the model were as consistent with the events surrounding the outbreak as was possible given the available data. Where no data was available, reasonable assumptions or estimates based on analogous data or experience were used so that, in the scenario modelled, there was little opportunity to manipulate inputs to achieve the specific results. Similarly, using as inputs data taken from Cassin et al. (1998b) yielded results that were consistent with the results of that stochastic risk assessment. The permeal risk predicted by the spreadsheet model was 1 in 16 million, within the range predicted by Cassin et al. (1998b) of 1 in 17 million to 1 in 830,000 meals consumed. Clearly, these two examples do not prove that the model is reliable. In our experience, however, the spreadsheet model predictions are usually within an order of magnitude of independent estimates of the number of cases of food-borne illness for specific product/hazard/pathway combinations. This level of

50

T. Ross, J. Sumner / International Journal of Food Microbiology 77 (2002) 3953

accuracy is expected on the basis of the models reliance on multiplication of a series of weighting factors, many of which are in 10-fold increments. Further examples of the models performance and utility as a risk management aid are presented in Sumner and Ross (this issue), and provide perspective on the relation of risk ranking values with recognised hazards. 4.2. Limitations/weaknesses The creation of the model was largely a reactive process, i.e., during testing against available epidemiological data, when the model failed, the source of the failure was analysed and the model modified to overcome that deficiency. Despite the apparent utility of the model, we have not been able to systematically and objectively evaluate the models performance, because there are few detailed data sets describing exposure and food-borne disease incidence. There are other limitations and weaknesses. Some are general problems associated with risk assessment modelling, while others are specific to the tool presented here. Even though we have attempted to make the questions unambiguous, the intent of the question can still be misinterpreted. For example, Question 4 refers to the proportion of any population that consumes the product. It does not need to be adjusted by the user when a sub-population is selected in Question 2, because the spreadsheet model automatically modifies the size of the population exposed when a subpopulation is selected at Question 2. Similarly, the answer to Question 10 is intended to be based on the ID50 for a healthy member of the normal population, irrespective of whether a susceptible population is selected at Question 2. Again, as described in Methods and materials, the calculations in the spreadsheet make adjustment for the selection in Question 2. In modelling any complex and variable system, it is necessary to balance the need to make simplifying assumptions against the loss of detail that ensues. In general, the Australian and USA statistics infer a risk of mild food-borne illness of one case per person every 5 to 10 years, roughly equivalent to a risk of 1 in 5000 11,000 meals. While the risk of outbreaks is much less, food safety managers are often more interested in understanding the sets of specific circumstances that lead to these relatively rare events of

food-borne illness outbreaks. Using a small number of descriptors of those conditions hinders discrimination of small, but potentially critical differences, so that important information can be lost in the averaging process that results. Another problem associated with these low levels of discrimination is that many choices automatically lead to at least a factor of 10 difference in the estimated risk. The option within some questions for the user to enter a specific value other than those offered arose from the realisation that the model could not make accurate predictions, unless a wider range of values, or narrower intervals between levels, were available. Following from the above, it must be emphasised that some of the weighting factors employed in the model are arbitrarily derived. Other weights may be more appropriate. For example, the weighting of relative susceptibility to illness of consumers with known predisposing conditions (Question 2) is currently based on the relative risk of listeriosis. While those factors may be broadly appropriate to susceptibility to infections, they may be irrelevant to the risk of intoxications from microbial, or other toxins. Earlier, we referred to the weights applied to the disease severity descriptors. One way to make these weights more objective is to express the severity of diseases in terms of days of quality or life lost, a nonspecific approach to measuring the health burden of illness that is increasingly advocated in the domain of public health (HCP, 2000). One such measure is disability adjusted life years (DALY), which enables the integration of different disease end-points. Using this approach, the difference in weights given to life-threatening food-borne disease compared to mild gastrointestinal forms was suggested to be too small (calculations not shown). As discussed earlier, the Risk Ranking estimates based on different disease end-points for the USA data similarly raised the question whether the weights applied to disease severity were appropriate. Weight factors based on DALYs would also simplify the comparison of illness from diverse sources, e.g. the acute effects of food-borne infections compared to the chronic effects of intoxications from chemical residues, increasing the applicability and universality of the proposed model. The weights and values used in the spreadsheet for these, and other variables, can be easily changed as necessary or appropriate. Care

T. Ross, J. Sumner / International Journal of Food Microbiology 77 (2002) 3953

51

should be exercised, however, that such changes do not lead to unrealistic values in some of the intermediate calculations in the model. Stochastic approaches to risk modelling are preferred because risk involves the element of probability (Cassin et al., 1998a). A limitation of the tool is that while it provides an estimate of the most probable outcome, it does not provide information about the level of confidence we have in that estimate, or more importantly, the probable range of illnesses for different scenarios. A possible refinement of the model would be to allow users to enter a range of values, or distribution of values that would offer some of the benefits of stochastic modelling, but still in a relatively simple tool. 4.3. Peripheral benefits of the tool Apart from its use for ranking perceived risks, the spreadsheet tool helps to focus the attention of the users on the interplay of factors that contribute to food-borne disease. The model can be used easily to explore the effect of different risk-reduction strategies, or the extent of change required to bring about a desired reduction in risk. Users must remember, however, that some of the weighting factors are arbitrarily derived. Consequently, the predicted effect may not reflect reality but only the assumptions on which the model is based, and users should ensure that the model is appropriate to their risk assessment question. Whether the mathematical model underlying the tool is correct or not, we found the spreadsheet tool to be a powerful aid for teaching the principles of risk assessment. The model forces users to think about factors affecting food safety and can help train food safety managers to think in terms of risk, and the interaction among factors that contribute to risk, rather than in absolute terms such as zero tolerance of hazards. Using the model to recreate scenarios quickly reveals where data critical to estimating risk are lacking, and so can be used to prioritise research needs.

et al. (2000). The model is intended to be generic but robust, and to include all elements that affect food safety risks. We propose that the tool can be used by risk managers and others without extensive experience in risk modelling and as a simple and quick means to develop a first estimate of relative risk. It can also be used as a training and risk communication aid to help determine data needs. The tool is preliminary, and should be seen as a prototype, not a definitive model. The tool also requires that users understand the models limitations. Despite those limitations, the model includes all elements required to estimate the risk of illness from foods. It can be modified to suit the specific question of the risk assessor or risk manager, and we have indicated possible developments and refinements to improve the utility of the tool. Tools such as these can help managers to think about how risks arise and change and, thus, to help to decide where interventions might be applied with success. We consider the tool as a useful and convenient aid to help risk managers reach food safety decisions more objectively. The spreadsheet can be downloaded from: http://www.agsci.utas.edu.au/ downloads/ratool.zip. Acknowledgements The authors wish to acknowledge the helpful and constructive comments of Dr. D. Jordan, New South Wales Agriculture, Dr. D. Schaffner, Rutgers University; Dr. E. Todd of Michigan State University and Mr. A. Fazil of Health Canada that led to improvements in the model structure and interface. The spreadsheet tool had its inception in food safety risk assessments conducted for Australias Dairy Research and Development Corporation, SafeFood NSW and Seafood Services Australia. TR also thanks Dr. R. Chandler and Mr. C. Chan for the impetus and encouragement they provided to develop early prototypes of the tool. We are also indebted to Meat and Livestock Australia for ongoing support for microbial food safety research. References
ABS (Australian Bureau of Statistics), 2000. Download (14 July 2000) from: http://www.abs.gov.au/Ausstats/.

5. Conclusion The motivation for the development of the risk assessment spreadsheet was to facilitate risk management prioritisation. Its application, thus, is similar to the Level 1 risk assessment proposed by van Gerwen

52

T. Ross, J. Sumner / International Journal of Food Microbiology 77 (2002) 3953 the European Commission in the Area of Human and Environmental Health, 26 27 October. Published on the internet 20/12/ 2000. Download from http://europa.eu.int/comm/food/fs/sc/ssc/ out83_en.pdf. Hitchins, A.D., 1996. Assessment of alimentary exposure to Listeria monocytogenes. Int. J. Food Microbiol. 30, 71 85. Huss, H.H., Reilly, A., Ben Embarek, P.K., 2000. Prevention and control of hazards in seafoods. Food Control 11, 149 156. ICMSF (International Commission for the Microbiological Specifications for Foods), 1996. In: Roberts, T.A. (Ed.), Micro-organisms in Foods: 5. Microbiological Specifications of Food Pathogens. Blackie Academic and Professional, London, 508 pp. ICMSF (International Commission on Microbiological Specifications for Foods), 1998. Potential application of risk assessment techniques to microbiological issues related to international trade in food and food products. J. Food Prot. 61, 1075 1086. ILSI (International Life Science Institute), North AmericaRisk Science Institute Pathogen Risk Assessment Working Group, 1996. A conceptual framework to assess the risks of human disease following exposure to pathogens. Risk Anal. 16, 841 848. Jaykus, L.-A., Hemard, M., Sobsey, M.D., 1994. Human enteric viruses. In: Hackey, C.R., Pierson, M.D. (Eds.), Environmental Indicators and Shellfish Safety. Chapman & Hall, New York, pp. 92 153. Jaykus, L.-A., 1996. The application of quantitative risk assessment to microbial food safety risks. Crit. Rev. Microbiol. 22, 279 293. Jones, E.M., McCulloch, S.Y., Reevest, D.S., MacGowan, A.P., 1994. A 10 year survey of the epidemiology and clinical aspects of listeriosis in a provincial English city. J. Infect. 29, 91 103. Jurado, R.L., Farley, M.M., Pereira, E., Harvey, R.C., Schuchat, A., Wenger, J.D., Stephens, D.S., 1993. Increased risk of meningitis and bacteremia due to Listeria monocytogenes in patients with human immuno-deficiency virus infection. Clin. Infect. Dis. 17, 224 227. Kindred, T.P., 1996. Risk analysis and its application in FSIS. J. Food Prot., 24 30, Supplement. Lammerding, A.M., 1997. An overview of microbial food safety risk assessment. J. Food Prot. 60, 1420 1425. Lindqvist, R., Westoo, A., 2000. Quantitative risk assessment for Listeria monocytogenes in smoked or gravad salmon and rainbow trout in Sweden. Int. J. Food Microbiol. 58, 181 196. Macintosh, W.A., Christensen, L.B., Acuff, G.R., 1994. Perceptions of risks of eating undercooked meat and willingness to change cooking practices. Appetite 22, 83 96. Mead, P.S., Slutsker, L., Dietz, V., McCaig, L.F., Bresee, J.S., Shapiro, C., Griffin, P.M., Tauxe, R.V., 1999. Food-related illness and death in the United States. Emerging Infect. Dis. 5, 607 625. Morgan, M.G., 1993. Risk analysis and management. Sci. Am., 32 41 July. Nolla-Salas, J., Anto, J.M., Almela, M., Col, P., Gasser, I., Plasencia, A., the collaborative study group of listeriosis of Barcelona, 1993. Incidence of listeriosis in Barcelona, Spain, in 1990. Eur. J. Clin. Microbiol. Infect. Dis. 12, 157 161.

ANZFA, 1999. Food Safety Standards Costs and Benefits. AGPS, Canberra. Bernstein, P.L., 1996. Against the Gods: The Remarkable Story of Risk. Wiley, New York, 383 pp. Buchanan, R.L., 1997. National advisory committee on microbiological criteria for foods Principles of risk assessment for illnesses caused by foodborne biological agents. J. Food Prot. 60, 1417 1419. Buchanan, R.L., Damert, W.G., Whiting, R.C., van Schothorst, M., 1997. Use of epidemiologic and food survey data to estimate a purposefully conservative dose response relationship for Listeria monocytogenes and incidence of listeriosis. J. Food Prot. 60, 918 922. CAC (Codex Alimentarius Commission), 1999. Proposed Draft Principles and Guidelines for the Conduct of Microbiological Risk Assessment (At step 5 of the Procedure). ALINORM, 99/ 13, Appendix IV, pp. 58 64. Cassin, M.H., Paoli, G.M., Lammerding, A.M., 1998a. Simulation modeling for microbial risk assessment. J. Food Prot. 61, 1560 1566. Cassin, M.H., Lammerding, A.M., Todd, E.C.D., Ross, W., McColl, R.S., 1998b. Quantitative risk assessment for Escherichia coli O157:H7 in ground beef hamburgers. Int. J. Food Microbiol. 41, 21 44. CAST, 1994. Foodborne pathogens: risk and consequences. Council for Agricultural Science and Technology, USA. Task Force Report No. 122. Conaty, S., Bird, P., Bell, G., Kraa, E., Grohmann, G., McAnulty, J., 2000. Hepatitis A in New South Wales, Australia from consumption of oysters: the first reported outbreak. Epidemiol. Infect. 124, 121 130. Corlett, D.A., Pierson, M.D., 1992. Hazard analysis and assignment of risk categories. In: Pierson, M.D., Corlett, D.A. (Eds.), HACCP: Principles and Applications. Van Nostrand-Reinhold, New York, pp. 29 38. FAO (Food and Agriculture Organisation of the United Nations), 1995. Application of risk analysis to food standards issues. Report of the Joint FAO/WHO Expert Consultation. Geneva, 13 17 March. WHO, Geneva. Farber, J.M., Ross, W.H., Harwig, J., 1996. Health risk assessment of Listeria monocytogenes in Canada. Int. J. Food Microbiol. 30, 145 156. FDA/FSIS, 2001a. Draft assessment of the relative risk to public health from foodborne Listeria monocytogenes among selected categories of ready-to-eat foods. Download from: http://www. foodsafety.gov/~dms/lmrisk.html. FDA/FSIS, 2001b. Draft risk assessment on the public health impact of Vibrio parahaemolyticus in raw molluscan shellfish. Download (1 July 2001) from: http://www.foodsafety.gov/. FSIS, 1998. Risk assessment for Salmonella enteritidis in eggs. Food Safety Inspection Service. USA, WWW site: http://www. fsis.usda.gov/ophs/risk/contents.htm. HCP (European Commission, Health and Consumer Protection Directorate-General), 2000. First report on the harmonisation of risk assessment procedures. The Report of the Scientific Steering Committees Working Group on Harmonisation of Risk Assessment Procedures in the Scientific Committees Advising

T. Ross, J. Sumner / International Journal of Food Microbiology 77 (2002) 3953 Notermans, S., Mead, G.C., 1996. Incorporation of elements of quantitative risk analysis in the HACCP system. Int. J. Food Microbiol. 30, 157 173. NRC (National Research Council), 1983. Risk Assessment in the Federal Government: Managing the Process. National Academy Press, Washington, DC. NRC (National Research Council), 1994. Science and Judgement in Risk Assessment. National Academy Press, Washington, DC. NRC (National Research Council), 1996. Understanding Risk: Informing Decisions in a Democratic Society. National Academy Press, Washington, DC. PCCRARM (Presidential/Congressional Commission on Risk Assessment and Risk Management), 1997. Framework for Environmental Health Risk Management. The Presidential/Congressional Commission on Risk Assessment and Risk Management. 213 pp. Available online at http://www.riskworld.com. Rose, J.B., Sobsey, M.D., 1993. Quantitative risk assessment for viral contamination of shellfish and coastal waters. J. Food Prot. 56, 1043 1050. Schlundt, J., 2000. Comparison of microbiological risk assessment studies published. Int. J. Food Microbiol. 58, 197 202. Sumner, J.L., Ross, T., 2002. A semi-quantitative seafood safety risk assessment. Int. J. Food Microbiol., this issue. Todd, E.C.D., Harwig, J., 1996. Microbial risk analysis of food in Canada. J. Food Prot., 10 18, Supplement. van Gerwen, S.J.C., te Giffel, M.C., vant Reit, K., Beumer, R.R., Zwietering, M.H., 2000. Stepwise quantitative risk assessment as a tool for characterization of microbiological food safety. J. Appl. Microbiol. 88, 938 951. Van Schothorst, M., 1997. Practical approaches to risk assessment. J. Food Prot. 60, 1439 1443.

53

Vose, D., 1996. Quantitative risk analysis: a guide to Monte Carlo simulation modelling. John Wiley and Sons, N.Y. Voysey, P.A, Brown, M., 2000. Microbiological risk assessment: a new approach to food safety control. Int. J. Food Microbiol. 58, 173 180. Walls, I., Scott, V.N., 1996. Validation of predictive mathematical models describing the growth of Escherichia coli 0157:H7 in raw ground beef. J. Food Prot. 59, 1331 1335. WHO/FAO (World Health Organisation/Food and Drug Organisation), 1999. Risk assessment of microbiological hazards in foods. Report of a Joint FAO/WHO Expert Consultation. Geneva, Switzerland. 15 19 March, 1999. World Health Organisation, Geneva, 24 pp. WHO/FAO (World Health Organisation/Food and Drug Organisation), 2000a. Exposure assessment of Listeria monocytogenes in ready-to-eat foods. Download (28 October 2001) from: http:// www.who.int/fsf/mbriskassess/index.htm. WHO/FAO (World Health Organisation/Food and Drug Organisation), 2000b. Exposure assessment of Salmonella Enteritidis in eggs. Download (28 October 2001) from: http://www.who.int/ fsf/mbriskassess/index.htm. WHO/FAO (World Health Organisation/Food and Drug Organisation), 2000c. Exposure assessment of Salmonella spp. in broilers. Download (28 October 2001) from: http://www.who.int/fsf/ mbriskassess/index.htm. WHO/FAO (World Health Organisation/Food and Drug Organisation), 2001. Risk characterization of Salmonella spp. in eggs and broiler chickens and Listeria monocytogenes in ready-to-eat foods. Download (28 October 2001) from: http://www.who.int/ fsf/mbriskassess/index.htm.