Leisure Studies

ISSN: 0261-4367 (Print) 1466-4496 (Online) Journal homepage: http://www.tandfonline.com/loi/rlst20

An evaluation framework for the sustainable

operation of leisure farms

Sheng-Hshiung Tsaur, Chang-Hua Yen & Pi-Shen Ku

To cite this article: Sheng-Hshiung Tsaur, Chang-Hua Yen & Pi-Shen Ku (2017) An evaluation
framework for the sustainable operation of leisure farms, Leisure Studies, 36:6, 739-751, DOI:
10.1080/02614367.2016.1246589

To link to this article: https://doi.org/10.1080/02614367.2016.1246589

Published online: 19 Oct 2016.

Submit your article to this journal

Article views: 142

View Crossmark data

Citing articles: 1 View citing articles

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=rlst20
Leisure Studies, 2017
VOL. 36, NO. 6, 739–751
https://doi.org/10.1080/02614367.2016.1246589

An evaluation framework for the sustainable operation of leisure

farms
Sheng-Hshiung Tsaura, Chang-Hua Yenb and Pi-Shen Kua
a
Graduate Institute of Recreation, Tourism and Hospitality Management, National Chiayi University, Chiayi, Taiwan,
ROC; bDepartment of Leisure and Recreation Management, National Taichung University of Science and Technology,
Taichung, Taiwan, ROC

ABSTRACT ARTICLE HISTORY

The Taiwanese government’s promotion of domestic travel has generated Received 24 March 2016
high demand for leisure farming, resulting in the increasing establishment Accepted 26 September 2016
of leisure farms and an intensification of competition. A key concern for
KEYWORDS
farm operators is how to operate and develop farms according to the Leisure farm; sustainable
principles of sustainability. Through a literature review, in-depth interviews operation; sustainability;
and a modified Delphi technique, this study developed an evaluation Delphi technique; in-depth
framework for the sustainable operation of leisure farms. From the study interview; Taiwan
results, we derived 39 indicators encompassing the five dimensions, namely
social sustainability, economic sustainability, environmental sustainability,
resource sustainability and management sustainability. The implications of
these findings for managing leisure farms and future research directions
are discussed.

Introduction
In the contemporary travel market, farm tourism has received attention in developed and developing
countries (Capriello, Mason, Davis, & Crotts, 2013; Chen, Chang, & Cheng, 2010; Engeset & Heggem,
2015). Leisure farms have developed as a new form of leisure travel in Taiwan (Wu, 2015; Wu &
Cheng, 2004). Over the past 10 years, leisure farm operations have increased dramatically in number
throughout Taiwan, but most of them are small- and medium-sized businesses (Hsu, Hsieh, & Yuan,
2013; Huang, 2006). As farm managers transition from traditional agriculture to leisure farming,
they face many challenges. Sustainable development has become crucial for leisure farm managers.
The concept of sustainability was derived from environmentalism, becoming prominent in the
1970s. The World Commission on Environment and Development defined sustainable development
as ‘development that meets the needs of the present without compromising the ability of future
generations to meet their own needs’ (WCED, 1987, p. 43). In the past decade, the concept of sustainable
development has generally been applied in the area of business management. Corporate sustainability
refers to adopting business strategies and activities that meet the current needs of enterprises and
their stakeholders while protecting, sustaining and enhancing the human and natural resources that
will be needed in the future (Deloitte & Touche, 1992). Morrison (2003) indicated that corporate
sustainability involves integrating social, environmental and economic accountability for forging a new
framework in which leaders make decisions and assume related responsibilities. Therefore, corporate

CONTACT  Chang-Hua Yen  chyan@nutc.edu.tw

© 2016 Informa UK Limited, trading as Taylor & Francis Group
740    S.-H. Tsaur et al.

sustainability not only concerns corporate economic performance but also emphasises various levels
of sustainable operation.
Developing a leisure farm involves difficulties and demands. Farmers lack knowledge regarding the
marketing and management of leisure farms and have insufficient information on agriculture tourists
and customers (Chang, 2003). Furthermore, leisure farms must successfully perform tasks such as
developing managerial abilities, promoting appraisals, upgrading the technical abilities of human
resources, encouraging the development of creative value and establishing a marketing platform (Tseng,
Wu, Lee, & Liao, 2008). Although ample research has investigated the key elements of leisure farm
operations (Fotiadis, Vassiliadis, & Piper, 2014; Park, Doh, & Kim, 2014), few studies have focused
on frameworks for evaluating the sustainable operation of leisure farms.
Azapagic (2004) and Robèrt et al. (2002) have proposed systems approaches consistent with the
basic principles and requirements of sustainability in considering strategic sustainable development
planning at the organisational level. Labuschagne, Brent, and van Erck (2005b) argued that the cur-
rent indicator frameworks for measuring overall business sustainability do not effectively address all
aspects of sustainability at the operational level. Although sustainable development has been studied
in the context of various industries, no study has addressed the construct of sustainable operation in
the context of leisure farms. Therefore, the present study developed a framework for sustainable oper-
ation that can be empirically applied to leisure farms. The framework is based on extant approaches
and impact assessment guidelines, but specifically addresses the needs of leisure farmers. The results
can provide useful insights to assist tourism authorities in coordinating with stakeholders to achieve
sustainable development.

Literature review
Leisure farms
Farm and ranch vacations date to the late nineteenth century. Since the 1960s, a growing number of
people with increased discretionary time have sought to experience adventure; thus, the number of
farm vacations has increased markedly (Bryan, 1991). People seek authentic experiences that challenge
their minds, expand their horizons and educate them in an outdoor, nature-oriented setting (Holdnak
& Pennington-Gray, 2000). Sightseeing, exploring small towns, tasting local foods and learning about
the land are examples of frequently enjoyed activities. Leisure farms integrate agricultural production
with the functions of leisure, travel, recreation and accommodation (Chang, 2003). Leisure farming is
defined as a business that uses rural scenery, ecological environments and resources, in combination
with farming, forestry, fishing and rural culture, to enhance people’s experiences of farming and
rural lifestyles (Council of Agriculture, 2000). The various types of leisure farms include farm-based
accommodation, meals, activities, attractions, retail establishments, tourism centres, as well as
agricultural festivals and events (Holdnak & Pennington-Gray, 2000). Leisure farms in Taiwan are a
compound form of recreational destination in which various types of offerings are provided, such as
barbecues, camping, karaoke, storytelling, ceremonies, conferences, magic shows, orchards and zoo
areas (Tseng et al., 2008).
In the past 25 years, the growth rate of agriculture in Taiwan has been lower than that of other
industries such as engineering or services. This pattern was exacerbated in 2002 after Taiwan joined
the World Trade Organization and the policy of the five-day workweek was implemented. Increased
leisure time has enhanced the potential number of tourists. The Taiwanese government has actively
encouraged traditional farm owners to transform their farms into leisure farms. In 1989, the Council
of Agriculture of Taiwan selected 31 farms in rural areas to develop leisure farms. In 2007, there
were 139 leisure farms in Taiwan. The number of leisure farms has increased rapidly, reaching 359
by the end of 2014 and generating annual revenues of nearly NT$11 billion (Taiwan Leisure Farms
Development Association, 2015). The average number of employees was 16.41. Most (96.9%) of the
farms were owner-managed. In 2008, approximately 9.6 million tourists visited leisure farms. The
 Leisure Studies   741

Tourism Bureau of Taiwan estimated that the tourist population of leisure farms would exceed 20
million by 2015 (Taiwan Tourism Bureau, 2014). The primary reason for developing leisure farms in
Taiwan is to offset fluctuations in agricultural income (Chang, 2003; Liu & Wu, 2014); another reason
is the low value and income of agricultural production.
Compared with other industries, tourism is a more viable method of economic development
because it is economically feasible, relatively clean (i.e. it does not rely on manufacturing), enhances
the recreation opportunities of communities and can be undertaken by residents and farmers in rural
communities (Lewis, 1998). In the context of declining agricultural development, the optimal policy
for farmers is to transform their operations and develop leisure farms to generate revenue. Achieving
sustainable development in such a competitive environment has become a major concern of leisure
farm managers. Determining whether leisure farms have distinct resources and business models is
crucial for sustainable development. Leisure farm managers must consider these issues carefully.

Sustainable operation
In 1997, the United Nations Environment Programme and Coalition for Environmentally Responsible
Economies launched the Global Reporting Initiative (GRI), which uses a hierarchical corporate sus-
tainability framework in the following three focus areas: social indicators (e.g. labour practices and
decent work, human rights, social and product responsibility), economic indicators (e.g. economic
performance, market share and indirect economic impacts) and environmental indicators (e.g. factors
related to a company’s impact on society and natural systems such as the atmosphere, hydrosphere and
lithosphere). On the basis of the general indicators proposed by the GRI, Azapagic (2004) developed
a framework of sustainable development indicators for the mining and minerals industry, comprising
economic, environmental, social and integrated indicators. She argued that sector-specific indicators
must reflect industry characteristics.
Furthermore, Labuschagne et al. (2005b) proposed a framework for assessing the sustainability
performance and operational activities in the manufacturing sector, including social, economic and
environmental indicators. The criteria in the proposed framework are particularly applicable for assess-
ing projects undertaken in the process industry. Social sustainability involves internal and external
dimensions. The internal dimensions include employment stability, employment practices, health
and safety, and capacity development, whereas the external dimensions of social sustainability are
focused on the impact of company operations on the communities in which they operate. Economic
sustainability is aimed at evaluating the short- and long-term financial stability and survivability of
businesses. The criteria of environmental sustainability are for evaluating clean development mecha-
nisms and for assessing the overall effects of life cycle systems.
Roberts and Tribe (2008) proposed a set of sustainability indicators related to small tourism enter-
prises (STE) with fewer than 50 employees. The indicators cover the four dimensions of sustainability –
environmental, sociocultural, economic and management/institutional – which can be used in assessing
an STE’s progress towards its long-term sustainability goals. Mearns (2011) constructed an evaluation
framework using sustainable tourism indicators for measuring the sustainability of community-based
ecotourism ventures. He determined sustainability according to numerous social, economic and envi-
ronmental indicators. The evaluation framework proved valuable for measuring present sustainabil-
ity as well as recommending areas for improvement in sustainability performance. According to the
aforementioned studies, the sustainable operation of a company can be evaluated according to various
dimensions and indicators. Previous studies have shown that sustainable operation indicators for var-
ious industries encompass the three constructs of social, economic and environmental sustainability.

Sustainable operation dimensions of leisure farms

Leisure farms are farm-based recreational enterprises (Tseng et al., 2008). The ability of leisure farms’
resources to attract customers is an essential factor for ensuring sustainable operation. Although
742    S.-H. Tsaur et al.

research on leisure farm operation is scant, some general studies on rural tourism development have
been conducted (Hwang & Lee, 2015; Nair, Munikrishnan, Rajaratnam, & King, 2015; Zou, Huang, &
Ding, 2014). Similar to other economic development strategies, rural tourism requires several compo-
nents to be successful. Tourism development involves (a) attractions: natural and manmade features
both in and adjacent to a community; (b) promotion: marketing of a community and its tourism
attractions to potential tourists; (c) tourism infrastructure: access to facilities (roads, airports, trains
and buses), water and power services, parking, signsand recreation facilities; (d) services: lodging,
restaurants and various retail businesses catering to tourist needs; and (e) hospitality: how tourists
are treated by both community residents and employees in tourism businesses and attractions (Gunn,
1988; Komppula, 2014; Wilson, Fesenmaier, Fesenmaier, & Van Es, 2001).
Park et al. (2014) identified six essential areas for improving managerial performance in the rural
tourism sector: product and service development, business planning and evaluation, promotions,
human resource management, networking and cost reduction. Furthermore, many tourists seek
authentic experiences at leisure farms, such as leisure activities, learning about rural life and receiv-
ing outdoor education (Murphy, 2003). On the basis of a review of the literature, the present study
integrated the social, economic and environmental dimensions to develop an evaluation framework
for the sustainable operation of leisure farms. Additionally, tourism enterprises mainly offer tourism
resources and services; their sustainable operation and development depend on whether their tourism
resources attract customers and generate profits. Roberts and Tribe (2008) proposed that management
sustainability is a crucial sustainable tourism indicator for STEs. Therefore, resource sustainability and
management sustainability were incorporated into the framework. The relevant dimensions for the
sustainable operation of leisure farms are summarised in the following paragraphs.
Social sustainability is concerned with a company’s impact on the social systems in which it oper-
ates, as well as its relationship with stakeholders (Labuschagne et al., 2005b). Social sustainability can
be divided into internal corporate sustainability, which focuses on a company’s responsibility to its
employees, and external social sustainability, which focuses on residents and tourists in local commu-
nities. Social sustainability can be divided into employee, local community and tourist dimensions.
The employee dimension is focused on employment stability, employee training and welfare of lei-
sure farms (Azapagic, 2004; GRI, 2002; Labuschagne et al., 2005b). The local community dimension
represents the impact leisure farms have on the local community (Azapagic, 2004). Furthermore, Ko
(2005) indicated that tourist needs should be considered in assessing sustainability. Thus, in the present
study, tourist satisfaction and tourist health and safety were incorporated into the tourist dimension.
Economic sustainability refers to a company’s ability to generate profit for surviving and benefiting
economic systems at the local and national levels (Roberts & Tribe, 2008). This is focused on a com-
pany’s internal and economic performance with the aim of evaluating short- and long-term financial
stability as well as survivability (IChemE, 2002). Economic sustainability entails primarily evaluating
a company’s operational stability and its influence on internal and external stakeholders’ economic
dimensions, serving as a basis for comprehending the company’s operations and its sustainable devel-
opment. Studies have included social sustainability rather than economic sustainability in investigat-
ing the influence of leisure farms on local communities (Azapagic, 2004; Labuschagne et al., 2005b).
Thus, this study focused on the financial health, number of tourists and labour costs of leisure farms.
Environmental sustainability refers to the activities of businesses involved in the conservation and
preservation of natural and built environments in a manner that maintains their health and integrity
(Roberts & Tribe, 2008). Environmental sustainability dimension is concerned with an organisa-
tion’s impact on natural systems, including the atmosphere, lithosphere and hydrosphere (GRI, 2002;
Labuschagne, Brent, & Claasen, 2005a).
Resource sustainability is focused on the natural, human history and equipment resources of lei-
sure farms, which can be used to sustain long-term operation. In traditional strategic analyses, firm
resources are strengths that firms can use to conceive of and implement strategies (Porter, 1981).
When a firm’s resources and capabilities are valuable, rare and socially complex, they are likely to be
sources of sustained competitive advantage (Barney, 1995). Leisure farms utilise original productive
 Leisure Studies   743

activities, places and cultural resources related to agriculture and combine them with local landforms,
climate and natural hydrological resources to provide tourists with leisure experiences of agricultural
life (Chang, 2003). Therefore, it is crucial for farm managers to utilise leisure farm resources effectively
for generating revenue and gaining the advantage of sustainable operation.
Management sustainability refers to the ability of an enterprise to effectively maintain organisa-
tional functionality and performance, which includes its organisational structure and overall operating
procedures. Roberts and Tribe (2008) indicated that management and institutional sustainability
refer to capacity development, which involves research and development as well as human resource
development. However, human resource development entails employee training and development;
many previous studies have categorised these internal human resources in the social sustainability
dimension (e.g. Azapagic, 2004; Labuschagne et al., 2005b). Leisure farms are small enterprises that
generate profits through providing services. To maintain sustainable operation, leisure farms must
establish a suitable organisational structure and business operation protocol to ensure that they can
effectively maintain organisational performance. Therefore, this study focused on the organisational
structure, research and development, and production capacity of leisure farm operations to establish
indicators for management sustainability.

Methodology
The evaluation procedure employed in this study comprised the following steps. First, we collated a
series of indicators for sustainable operation from the literature on leisure farms. Second, we conducted
in-depth interviews with scholars and leisure farm managers to select the most appropriate indicators.
Finally, the modified Delphi technique was adopted to refine and identify the final indicators for the
sustainable operation of leisure farms.

Indicators for the sustainable operation of leisure farms

To develop an evaluation framework for the sustainable operation of leisure farms, we incorporated
the indicators and dimensions of sustainable development proposed by Azapagic (2004), Labuschagne
et al. (2005a, 2005b), and Roberts and Tribe (2008). We also applied the findings of previous studies
(Chang, 2003; Ko, 2005) on rural tourism development to develop a set of sustainable operation indica-
tors related to leisure farms. We obtained 33 items from the literature, encompassing the 5 dimensions
of sustainability, namely social sustainability, economic sustainability, environmental sustainability,
resource sustainability and management sustainability.

In-depth interviews
In-depth interviews are an open-ended discovery-oriented method for obtaining detailed information
about a topic (Easterby-Smith, Thorpe, & Lowe, 2002). In-depth interviews are a qualitative research
method considered to be the most appropriate for obtaining information about perceptions, attitudes
and beliefs (Finn, Elliot-White, & Walton, 2000). Furthermore, an individual in-depth interview allows
an interviewer to probe deeply into personal matters (DiCicco-Bloom & Crabtree, 2006). To under-
stand the sustainable operation of leisure farms, the following open-ended questions were asked in
the in-depth interviews conducted in the present study: (a) ‘What is the central issue for achieving the
sustainable operation of leisure farms?’; and (b) ‘To achieve sustainable operation and promote compet-
itiveness, what capacities do you think leisure farms must possess?’ After developing the questions, we
selected the most appropriate indicators through five interviews with three senior managers of leisure
farms and two scholars who had conducted studies or participated in projects related to leisure farms.
The dialogues in the in-depth interviews were tape-recorded and then transcribed. Incomprehensible
or repetitive indicators were eliminated, and new indicators were added accordingly. Table 1 lists the
48 indicators that were used for developing a Delphi questionnaire.
744    S.-H. Tsaur et al.

Table 1. Potential indicators for the sustainable operation of leisure farms.

Categories Items Relevant literature

Social sustainability The appropriate employment rate of full-time employees Azapagic (2004);
Lower labour turnover Labuschagne et al. (2005a,
Provided enough employee training and education 2005b); Roberts and Tribe
Support for employee career development (2008)
Employees’ job satisfaction
Provided enough employee welfare
Appropriate use of employee talent*
Legality of farm operations*
Compliance with public sector policies*
Provided employment opportunities*
Combined with local resources
Feedback to the community
Maintained tourist health and safety
Tourist satisfaction
Reasonable price*
Economic sustainability High operational revenue and net profit ratio Azapagic (2004);
High asset turnover Labuschagne et al. (2005b);
High average sales revenue per employee Roberts and Tribe (2008)
Increased number of tourists
The appropriate ratio of labour costs to total revenue
The link between foreign investment projects and the farm
Product and service prospect*
Environmental sustain- Maintained ecological environment* Azapagic (2004);
ability Maintained air quality Labuschagne et al. (2005a,
Water recycling and reuse 2005b); Roberts and Tribe
Environmental protection and usage of electrical energy (2008)
Recycled renewable resources
Resource sustainability Plant diversity Chang (2003); Ko (2005)
Animal diversity
Provided rich natural landscapes
Provided rich industry landscapes
Hosting special festival and activities*
Provided interpretation equipment*
Service quality of employees*
Provided appropriate recreational equipment
Provided appropriate accommodations
Provided appropriate dining equipment
Completed electronic development*
Management Sustaining traditional agricultural resources Labuschagne et al. (2005b);
sustainability Establish a stable organisational structure Roberts and Tribe (2008)
Clear positioning of the farm
Consistency between departmental and organisational goals
Level of interdepartmental coordination and cooperation
Research and development capability for product diversification
Control production capability*
Ability to build a farm brand*
Diversify marketing channels*
Establishing collaborative partnerships*
*Items added after the in-depth interviews.

Modified Delphi technique

The Delphi technique was adopted to refine and identify the final indicators for the sustainable operation
of leisure farms. The Delphi technique is a method for organising group communication without direct
discussion, with the aim of refining group opinions and reaching consensus (Tersine & Riggs, 1976).
The Delphi technique is used for systematically collecting and aggregating informed judgments from
groups of experts on specific questions or issues (Reid, 1988). The method maximises the benefits of
using an informed panel while minimising the disadvantages in collective decision-making (Jones &
Hunter, 2000). Furthermore, it enables access to larger groups, which meetings would typically inhibit.
When consensus is achieved, evidence of concurrent validity is attained because the participants have
 Leisure Studies   745

Table 2. Profiles of experts.

No. Gender Title Institute

1 Female Professor Department of geography, national Changhua university of education
2 Male Associate professor Department of travel management, Jinwen University of Science and Technology
3 Male Professor Department of marketing and tourism management, national chiayi university
4 Male Professor Department of agribusiness management, National pingtung university of science
and technology
5 Male Secretary general Taiwan leisure farms development association
6 Male Consultant Taiwan leisure farms development association
7 Male Technical specialist Council of agriculture, executive Yuan, R.O.C.
8 Male Farm manager Shin Kong Chao Feng ranch and resort
9 Male Farm manager tsou ma lai farm
10 Female Assistant vice president Shangrila leisure farming
11 Male Farm manager Flying cow ranch
12 Female Farm manager Sheng Yang leisure farm

identified and agreed on the components (Williams & Webb, 1994). Moreover, validity is achieved
because an expert panel from the real world provides confirmative judgments (Cross, 1999).
This study adopted a modified Delphi technique to collect data. Round 1 of the conventional Delphi
technique involves using open-ended questionnaires to gather the opinions of each participant; a
survey questionnaire is then devised accordingly. However, this step is time-consuming and difficult
to conduct. Hence, the modified Delphi technique proposed by Murry and Hammons (1995) was
adopted to bypass using an open-ended questionnaire. A literature review and in-depth interviews
with experts were conducted for developing a structured questionnaire to replace the typical Round
1 survey. Thus, a two-round modified Delphi survey technique was adopted.
Regarding the number of experts, Murry and Hammons (1995) suggested summarising the opin-
ions of 10–30 experts. Accordingly, 12 experts including scholars, representatives from a leisure farm
development association, government officials and leisure farm managers (the leisure farms were
owner-managed) from the industry were invited to participate in an expert survey (Table 2). The
first round was conducted in April 2014 and May 2014. The questionnaire packets were mailed to the
Delphi participants, containing surveys, one page of instructions and postage-paid return envelopes.
The experts were asked to express their perceptions of the degree of significance for each indicator
regarding the sustainable operation of leisure farms. All items were measured using a 5-point Likert
scale ranging from 1 (strongly disagree) to 5 (strongly agree). The same group of experts recruited for
Round 1 were recruited as the research participants for the Round 2 survey. Consensus was achieved
on completion of the second round at the end of July 2014. The final system of indicators totalled 39
items (Table 3). The proposed indicators are a comprehensive representation of the sustainable oper-
ation of leisure farms and were devised through a well-designed procedure.

Results
A descriptive analysis was performed to compare the mean, standard deviation (SD) and coefficient
of variation (CV) between the Round 1 and Round 2 questionnaire data. The constructs were ranked
according to their impact, from the strongest agreement to the strongest disagreement. The ranking
order was determined on the basis of each construct’s mean score. Additionally, the CV, which is the
SD expressed as a percentage of the mean of each indicator, was calculated for Rounds 1 and 2. A CV
of more than 50% indicates disagreement among panel members. Conversely, a reduction in SD and
CV over the two Delphi rounds indicates growing consensus among panel members (Green, Hunter,
& Moore, 1990). The sustainable operation constructs devised in this study were identified according
to the Round 2 survey results.
Table 3 lists the sustainable operation constructs, showing that social, economic, environmen-
tal, resource and management sustainability attained high mean scores (M ≥ 4.50). Environmental
746    S.-H. Tsaur et al.

Table 3. Statistical analysis of sustainability indicators.

Indicators Mean SD CV Rank

Social sustainability 4.50 0.52 11.6 5
Employees
The appropriate employment rate of full-time employees 4.30 0.48 11.2 3
Lower labour turnover 4.60 0.52 11.3 1
Provided enough employee training and education 4.40 0.52 11.8 2
Employees’ job satisfaction 4.40 0.70 15.9 2
Provided enough employee welfare 4.30 0.48 11.2 3
Local residents
Provided employment opportunities 4.30 0.48 11.2 2
Combined with local resources 4.40 0.50 11.3 1
Feedback to the community 4.40 0.52 11.8 1
Tourists
Maintained tourist health and safety 4.60 0.50 10.8 1
Tourist satisfaction 4.60 0.52 11.3 1
Economic sustainability 4.70 0.48 10.2 2
High operational revenue and net profit ratio 4.60 0.52 11.3 2
High asset turnover 4.50 0.53 11.8 3
High average sales revenue per employee 4.30 0.48 11.2 4
Increased number of tourists 4.70 0.48 10.2 1
The appropriate ratio of labour costs to total revenue 4.70 0.49 10.4 1
Environmental sustainability 4.80 0.42 8.8 1
Maintained ecological environment 4.50 0.53 11.8 2
Maintained air quality 4.50 0.53 11.8 2
Water recycling and reuse 4.20 0.79 18.9 3
Environmental protection and usage of electrical energy 4.60 0.50 10.8 1
Recycled renewable resources 4.60 0.52 11.3 1
Resource sustainability 4.60 0.51 11.1 3
Natural resources
Plant and animal diversity 4.20 0.79 18.8 2
Provided rich natural landscapes 4.50 0.53 11.8 1
Human history resources
Provided rich industry landscapes 4.80 0.42 8.8 1
Hosting special festival and activities 4.40 0.53 12.0 2
Equipment resources
Provided interpretation equipment 4.50 0.53 11.8 2
Provided appropriate recreational equipment 4.30 0.48 11.2 3
Provided appropriate accommodations 4.50 0.51 11.3 2
Provided appropriate dining equipment 4.70 0.48 10.2 1
Completed electronic development 4.50 0.50 11.1 2
Management sustainability 4.60 0.52 11.3 3
Sustaining traditional agricultural resources 4.80 0.42 8.8 1
Establish a stable organisational structure 4.20 0.79 18.8 6
Clear positioning of the farm 4.60 0.51 11.9 2
Consistency between departmental and organisational goals 4.30 0.52 12.1 5
Level of interdepartmental coordination and cooperation 4.40 0.52 11.8 4
Research and development capability for product diversification 4.30 0.48 11.2 5
Control production capability 4.40 0.52 11.8 4
Ability to build a farm brand 4.30 0.48 11.2 5
Diversify of marketing channels 4.50 0.53 11.8 3
Establish of collaborative partnership 4.60 0.52 11.3 2
Notes: Mean: 1 = strongly disagree to 5 = strongly agree; SD = Standard deviation; CV = Coefficient of variation [(SD ÷ M) × 100].

sustainability (M = 4.80) exhibited the most agreement in the dimension of sustainable operation. The
CVs were lower than 20% for these factors, indicating consensus among the panel members.
The results for the social sustainability indicators show that lower labour turnover, tourist health
and safety, and tourist satisfaction attained higher mean scores (M ≥ 4.6) compared with the other
indicators in the same construct. These three factors attained the most agreement among the sustain-
able operation indicators in the social sustainability dimension. These three indicators also reached
a fairly high degree of consensus on the basis of a smaller SD (SD ≤ 0.52) compared with the same
 Leisure Studies   747

indicators in the previous round as well as the other indicators in the same construct, attaining CVs
lower than 12%.
Regarding the economic sustainability indicators, the results indicate that high operational rev-
enues and net profit ratios, an increase in the number of tourists and an appropriate ratio of labour
costs to total revenues attained higher mean scores (M ≥ 4.6) compared with the other indicators in
the same construct. These three indicators also reached a higher degree of consensus on the basis of
small SD (SD ≤ 0.52) and lower CV (CV ≤ 11.3%) compared with the same indicators in the previous
round and the other indicators in the same dimension. These three factors attained the highest level
of agreement among the indicators in the economic sustainability dimension.
The results for the environmental sustainability indicators show that environmental protection,
electrical energy usage and using recycled renewable resources attained distinctively high mean scores
(M ≥ 4.6) in this construct. Their SDs were also smaller (SD < 0.53) than those in the previous round,
and the CVs were lower than 12%. Therefore, consensus among panel members was strong regarding
these two indicators. Moreover, these two factors attained the highest level of agreement among the
sustainable operation indicators in the environmental sustainability dimension.
Regarding the resource sustainability indicators, the results demonstrate that providing rich nat-
ural landscapes and providing appropriate dining facilities received higher mean scores (M ≥ 4.6)
compared with other indicators in the same dimension. These two indicators also reached a higher
degree of consensus, given their smaller SD (SD < 0.50) and lower CVs (CV ≤ 10.2%), compared
with the previous round and other indicators in the same construct. These two factors attained the
most agreement among the sustainable operation indicators in the resource sustainability dimension.
Finally, for the management sustainability indicators, the results showed that sustaining traditional
agricultural resources, clearly positioning the farm and establishing collaborative partnerships attained
higher mean scores (M ≥ 4.6) compared with the other indicators in the same construct. These three
indicators also reached a higher degree of consensus based on the smaller SD (SD ≤ 0.51) and lower
CV (CV ≤ 11.9%) compared with the same indicators in the previous round as well as the other indi-
cators in the same dimension. These three factors attained the highest level of agreement among the
indicators in the management sustainability dimension.

Conclusion and discussion

Through a literature review, in-depth interviews and modified Delphi technique, an evaluation frame-
work was developed comprising 5 dimensions and 39 indicators for the sustainable operation of
leisure farms. This study contributes significantly to research on evaluation frameworks in this field.
Particularly, in addition to the adoption of general business assessments of the three basic dimen-
sions of sustainable operation (i.e. social, economic and environmental indicators), the dimensions
of farm resource sustainability and management sustainability were integrated into the framework.
Furthermore, this evaluation instrument can be used by leisure farm managers to evaluate their sustain-
able operations. The results of the self-assessment can provide leisure farm managers with a reference
for improving their farm operations.
The results of this study show that environmental sustainability attained the highest level of agree-
ment among the four assessment dimensions. Roberts and Tribe (2008) stated that through protection
and preservation, environmental health and integrity can be preserved, contributing to the sustaina-
bility of STEs. Additionally, because of current international trends, environmental protection indus-
tries have received considerable attention from governments and the public. Moreover, owing to the
natural environment of leisure farms being the original selling point for tourism, the environmental
sustainability dimension has been particularly well received. The economic sustainability dimension
attained the second-highest level of approval of the dimensions. From the perspective of operational
management, generating profit enables leisure farms to survive and operate; hence, economic sus-
tainability is crucial.
748    S.-H. Tsaur et al.

Regarding the dimensions of social sustainability, lower labour turnover was the most approved
indicator in its construct. With lower turnover rates, leisure farm employees work for long periods
and thus become highly familiar with farm work. Accordingly, farm managers should consider organ-
ising training sessions to maintain or improve the quality of service, which would facilitate building
a positive reputation and benefit farm operations. In addition, the two indicators – tourist health and
safety and tourist satisfaction – attained high mean scores, showing that farms should implement cus-
tomer-oriented policies after becoming tourist destinations for recreational activities. Hence, tourist
health, safety and satisfaction are crucial factors that influence revisits and recommendations.
An increased number of tourists and an appropriate ratio of labour costs to total revenue are two
indicators in the economic sustainability dimension that attained a relatively high level of approval.
Sustainable development can be achieved when revenues exceed working capital. Hence, if the number
of tourists exhibits a growing trend, this implies an increase in revenues, enabling farmers to earn
more profit and contribute to the long-term operations of their farms. In addition, both tourism and
leisure industries are business units that depend highly on human delivery services; therefore, labour
costs are a key factor in their profit ratio. Hence, managing and controlling labour costs effectively is
crucial for lowering operating costs and improving profits.
Regarding the environmental sustainability dimension, environmental protection and usage of
electrical energy and recycled renewable resources attained the highest rating. When the number of
visitors increases, power consumption in the farm also increases, possibly causing electricity wast-
age. However, environmentally friendly energy infrastructure can effectively improve the efficiency
of energy use. Practical implementation not only directly reduces farm overheads but also creates
competitive advantages and a basis for market differentiation. Furthermore, during farm operations
and tourist visits, a large amount of waste might be generated. From the perspective of environmental
conservation, if recyclable waste is treated appropriately, then environmental pollution can be reduced
and recreation quality at the farm can also be enhanced.
In the resource sustainability dimension, providing rich natural landscapes was the indicator that
attained the highest level of approval. Thus, leisure farms can use existing agricultural landscapes
combined with natural farm landscapes to enhance visitor experiences. Therefore, rich natural land-
scapes, such as vast mountains and multilevel terrain landscapes, help attract visitors to leisure farms.
Providing rich industry landscapes attained the highest level of approval among the indicators in the
same dimension. An opportunity to experience the processes contributing to agricultural, forestry,
fishery and animal husbandry output, as well as operational activities and life and culture of farming,
are the main reasons tourists visit leisure farms; they also constitute the basic conditions for sustainable
operation. Providing farm-theme-related agricultural landscapes, such as those involving livestock
barns, grazing and farm produce production processes, can also increase the possibility of tourists
staying longer and spending more money. Thus, leisure farms should offer rich agricultural land-
scapes. Finally, providing appropriate dining facilities attained the highest level of approval among the
indicators in its dimension. When tourists participate in farm activities, the farm must meet tourist
needs for food and beverages; additionally, adequate dining facilities can also enhance the recreational
satisfaction of tourists. Thus, farmers should consider the number of tourists in establishing their
dining facilities, to ensure that the capacity of dining facilities is sufficient for catering to tourist needs.
Moreover, indicators such as providing interpretation equipment, providing appropriate accommo-
dation and complete electronic development also contribute to the sustainable development of leisure
farms, as demonstrated by their higher scores compared with the other indicators.
In the management sustainability dimension, sustaining traditional agricultural resources attained
the highest rating. Traditional agricultural resources are the cornerstone for developing farming opera-
tions. A traditional farm can be transformed into a leisure farm by combining a traditional agricultural
enterprise with the service functionality of tertiary industries, enabling the complete utilisation of
agricultural resources. The objective of the transformation is to ensure the sustainable use of agricul-
tural resources and to exploit the recreational functionality of the agricultural industry while consid-
ering the interests of the farmers. Thus, the sustained use of traditional agricultural resources is the
 Leisure Studies   749

cornerstone of the sustainable operation of leisure farms, and farms should endeavour to maintain
sustainable development and diversify their traditional agricultural resources. In addition, clearly
positioning the farm and establishing collaborative partnerships also attained high ratings. A clear
understanding of its market position enables an enterprise to define its market niche and differentiate its
services to derive competitive advantages. Thus, as a management strategy, leisure farms should clearly
define their market position to create a niche that will separate it from other leisure farms in order to
develop an effective marketing strategy. With rising consumer awareness, the marketing mix method
(i.e. the 4 P’s – product, price, place, promotion) can no longer satisfy the requirements of operational
management. A partnership strategy (i.e. the fifth ‘P’) is more practicable for the sustainable operation
of an enterprise, involving establishing close partnerships with distributing channel stakeholders.
According to the research results, we propose the following management practice implications.
First, leisure farm managers should organise employee education and training courses according to
the work content. In addition to adequately planned training courses for bridging skill gaps, internal
and external training courses offered by leisure farms can also assist in the career development of
employees. When such training courses benefit the career development of employees, employee turno-
ver is lowered and work satisfaction is enhanced. Second, leisure farms should establish clear financial
targets on the basis of the external environment and internal conditions through a self-assessment of
financial performance. For instance, during profit planning for farm operations, plans can be made
according to the operational revenues, expenditures, and profits of various farm divisions, including
food and beverages, accommodation and merchandise sales. Annual business targets can be planned
accordingly. Moreover, overly positive or conservative estimations that could lead to erroneous plan-
ning and underachievement of financial targets can be avoided, thereby contributing to the financial
sustainability of the farm. Finally, the performance of equipment resources should be strengthened.
To enhance farm resource sustainability, managers can use tourist demand as a planning premise to
determine the recreational demand of tourists visiting farms, and assess the adequacy of existing farm
facilities accordingly, thereby expanding recreational facilities where necessary to meet tourist demand
without affecting the ecology of the farms. Furthermore, according to the average number of daily
visitors from the previous year, the farms can estimate the growth in the number of visitors requiring
accommodation in the coming years to develop plans for expanding accommodation facilities.
Although this study focused on developing an evaluation framework for sustainable operation,
the 39 indicators can provide an instrument for evaluating the sustainable operation of leisure farms.
Leisure farm managers can use this instrument to determine whether their resources and manage-
ment style can support long-term farm operations. The framework provides a means for collecting
empirical evidence for measuring the sustainability of leisure farms, in which each indicator has its
own priority rank to indicate its relative importance among all indicators. Although the framework
proposes a generic approach for determining the sustainability of leisure farm operations, it can be
adapted to other contexts such as community-based ecotourism enterprises and other farm entre-
preneurial ventures. Thus, including site-specific indicators might be necessary to account for local
conditions and circumstances when reapplying the proposed evaluation framework in other settings.
A long-term monitoring system is also necessary to enable the early identification of how man-
agerial actions affect the sustainability of operations. The purpose of sustainability indicators is to
determine whether the operations of leisure farms have deviated from the principles of sustainability.
Although the proposed framework provides such information, the information it provides managers
might be inadequate. Therefore, future studies should consider adopting interview and observational
approaches to assist leisure farm managers in identifying the appropriate actions to ensure operational
sustainability. Furthermore, future applications of the evaluation framework must include indicators
of climate change for evaluating the associated preparedness of leisure farms.

Disclosure statement
No potential conflict of interest was reported by the authors.
750    S.-H. Tsaur et al.

Funding
This work was supported by the National Science Council, Taiwan [NSC 99-2622-H-415-001-CC3].

Notes on contributors
Sheng-Hshiung Tsaur is a professor in Graduate Institute of Recreation, Tourism and Hospitality Management, National
Chiayi University, Taiwan.
Chang-Hua Yen is a professor in Department of Leisure and Recreation Management, National Taichung University
of Science and Technology, Taiwan.
Pi-Shen Ku is a PhD Student in Graduate Institute of Recreation, Tourism and Hospitality Management, National
Chiayi University, Taiwan.

References
Azapagic, A. (2004). Developing a framework for sustainable development indicators for the mining and minerals
industry. Journal of Cleaner Production, 12, 639–662.
Barney, J. B. (1995). Looking inside for competitive advantage. The Academy of Management Executive, 9, 49–61.
Bryan, B. (1991). Ecotourism on family farms and ranches in the American West. In T. Whelan (Ed.), Nature tourism:
Managing for the environment (pp. 75–85). Washington, DC: Island Press.
Capriello, A., Mason, P. R., Davis, B., & Crotts, J. C. (2013). Farm tourism experiences in travel reviews: A cross-
comparison of three alternative methods for data analysis. Journal of Business Research, 66, 778–785.
Chang, T. C. (2003). Development of leisure farms in Taiwan, and perceptions of visitors thereto. Journal of Travel &
Tourism Marketing, 15, 19–40.
Chen, J. S., Chang, L. L., & Cheng, J. S. (2010). Exploring the market segments of farm tourism in Taiwan. Journal of
Hospitality Marketing & Management, 19, 309–325.
Council of Agriculture. (2000). Report of major policies in 2000. Taipei: Taiwan Council of Agriculture.
Cross, V. (1999). The same but different a Delphi study of clinicians’ and academics’ perceptions of physiotherapy
undergraduates. Physiotherapy, 85, 28–39.
Deloitte & Touche. (1992). Business strategy for sustainable development: Leadership and accountability for the 90s. New
York, NY: Deloitte & Touche.
DiCicco-Bloom, B., & Crabtree, B. F. (2006). The qualitative research interview. Medical Education, 40, 314–321.
Easterby-Smith, M., Thorpe, R., & Lowe, A. (2002). Management research methods. London: Sage Publications;
Examinership-Friel Stafford.
Engeset, A. B., & Heggem, R. (2015). Strategies in norwegian farm tourism: Product development, challenges, and
solutions. Scandinavian Journal of Hospitality and Tourism, 15, 122–137.
Finn, M., Elliot-White, M., & Walton, M. (2000). Tourism and leisure research methods: Data collection analysis and
interpretation. Harlow: Longman.
Fotiadis, A. K., Vassiliadis, C. A., & Piper, L. A. (2014). Measuring dimensions of business effectiveness in greek rural
tourism Areas. Journal of Hospitality Marketing & Management, 23, 21–48.
Green, H., Hunter, C., & Moore, B. (1990). Application of the Delphi technique in tourism. Annals of Tourism Research,
17, 270–279.
GRI. (2002). Sustainability reporting guidelines. Boston, MA: Global Reporting Initiative.
Gunn, C. A. (1988). Tourism planning (2nd ed.). New York, NY: Taylor and Francis.
Holdnak, A., & Pennington-Gray, L. (2000). Farm tourism: A case study in Florida’s orange groves. Parks and Recreation,
35, 146–156.
Hsu, S. M., Hsieh, P. H., & Yuan, S. T. (2013). Roles of ‘small- and medium-sized enterprises’ in service industry
innovation: A case study on leisure agriculture service in tourism regional innovation. The Service Industries Journal,
33, 1068–1088.
Huang, L. (2006). Rural tourism revitalization of the leisure farm industry by implementing an e-commerce strategy.
Journal of Vacation Marketing, 12, 232–245.
Hwang, J., & Lee, S. (2015). The effect of the rural tourism policy on non-farm income in South Korea. Tourism
Management, 46, 501–513.
IChemE. (2002). The sustainability metrics: Sustainable development progress metrics recommended for use in the process
industries. Warwickshire: Institution of Chemical Engineers.
Jones, J., & Hunter, D. (2000). Using the Delphi and nominal group technique in health services research. Quality
Management in Health care, 8, 40–51.
Ko, T. G. (2005). Development of a tourism sustainability assessment procedure: A conceptual approach. Tourism
Management, 26, 431–445.
 Leisure Studies   751

Komppula, R. (2014). The role of individual entrepreneurs in the development of competitiveness for a rural tourism
destination – A case study. Tourism Management, 40, 361–371.
Labuschagne, C., Brent, A. C., & Claasen, S. J. (2005a). Environmental and social impact considerations for sustainable
project life cycle management in the process industry. Corporate Social Responsibility and Environmental Management,
12, 38–54.
Labuschagne, C., Brent, A. C., & van Erck, R. P. G. (2005b). Assessing the sustainability performances of industries.
Journal of Cleaner Production, 13, 373–385.
Lewis, J. B. (1998). A rural tourism development model. Tourism Analysis, 2, 91–106.
Liu, M., & Wu, Y. L. (2014). Agriculture reform in Taiwan from a leisure farm visitors’ perspective. Journal of Food,
Agriculture & Environment, 12, 423–426.
Mearns, K. F. (2011). Using sustainable tourism indicators to measure the sustainability of a community-based ecotourism
venture: Malealea Lodge & Pony Trek Centre, Lesotho. Tourism Review International, 15, 135–147.
Morrison, J. L. (2003). Organizational change for corporate sustainability: A guide for leaders and change agents of the
future. Journal of Education for Business, 79, 124–126.
Murphy, A. E. (2003). Illustrating the utility of a modified gap analysis as a regional tourism planning tool: Case study
of potential Japanese and German travelers to the cowichan region. Journal of Travel Research, 41, 400–409.
Murry, J. W., Jr., & Hammons, J. O. (1995). Delphi: A versatile methodology for conducting qualitative research. Review
of Higher Education, 18, 423–436.
Nair, V., Munikrishnan, U. T., Rajaratnam, S. D., & King, N. (2015). Redefining rural tourism in Malaysia: A conceptual
perspective. Asia Pacific Journal of Tourism Research, 20, 314–337.
Park, D. B., Doh, K. R., & Kim, K. H. (2014). Successful managerial behaviour for farm-based tourism: A functional
approach. Tourism Management, 45, 201–210.
Porter, M. (1981). The contributions of industrial organization to strategic management. Academy of Management
Review, 6, 609–620.
Reid, N. (1988). The Delphi technique: Its contribution to the evaluation of professional practice. In R. D. Ellis (Ed.),
Professional competence and quality assurance in the caring profession, 230–262. London: Chapman and Hall.
Robèrt, K. H., Schmidt-Bleek, B., De Larderel, J. A., Basile, G., Jansen, J. L., Kuehr, R., … Wackernagel, M. (2002). Strategic
sustainable development – Selection, design and synergies of applied tools. Journal of Cleaner Production, 10, 197–214.
Roberts, S., & Tribe, J. (2008). Sustainability indicators for small tourism enterprises – An exploratory perspective.
Journal of Sustainable Tourism, 16, 575–594.
Taiwan Leisure Farms Development Association. (2015). Retrieved August 8, 2016, from http://www.taiwanfarm.org.tw/
Taiwan Tourism Bureau. (2014). Tourism statistics in Taiwan. Taipei: Tourism Bureau, Ministry of Transportation and
Communication.
Tersine, R. J., & Riggs, W. E. (1976). The delphi technique: A long-range planning tool. Business Horizons, 19, 51–56.
Tseng, M. L., Wu, W. W., Lee, Y. T., & Liao, C. H. (2008). An explorative model of customers’ service quality perceptions
for leisure farms in Taiwan. WASEA Transactions on Business and Economics, 12, 507–523.
WCED. (1987). Our common future. Oxford: Oxford University Press.
Williams, P. L., & Webb, C. (1994). The Delphi technique: A methodological discussion. Journal of Advanced Nursing,
19, 180–186.
Wilson, S., Fesenmaier, D. R., Fesenmaier, J., & Van Es, J. C. (2001). Factors for success in rural tourism development.
Journal of Travel Research, 40(2), 132–138.
Wu, C. W. (2015). Foreign tourists’ intentions in visiting leisure farms. Journal of Business Research, 68, 757–762.
Wu, T. C., & Cheng, J. (2004). Market segmentation and positioning analysis for leisure farm market in Northern Taiwan.
Journal of Tourism Studies, 10, 97–113.
Zou, T., Huang, S. S., & Ding, P. (2014). Toward a community-driven development model of rural tourism: The Chinese
experience. International Journal of Tourism Research, 16, 261–271.