Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Introduction
This bulletin reports on the fourth and final snapshot survey of the evaluation of the Local
Sustainability Fund (LSF). It asked all grant holders about the emergent impact and
outcomes of their projects, as well as some of the factors they thought could help to explain
these changes. It was sent to all 270 grant holders in July 2017 and we received 92
responses of which five were excluded due to being incomplete, giving 87 valid responses
(representing a response rate of approximately 33%). This is slightly lower than the previous
three surveys1. We would expect a decline in response rates throughout the evaluation
period and this response rate is nonetheless sufficient to draw conclusions, although our
ability to undertake analysis within sub-categories is limited due to the smaller number of
responses.
The survey contained nine questions, which were a mix of closed and open questions.
Because the number of respondents is less than 100, both the actual number of
respondents and the percentage figures will be quoted in this bulletin. All findings are self-
reported by grant holders and do not represent external verification of impact, outcomes
and changes.
We are very grateful to all of the grant holders who responded to this survey, and the
previous three surveys for providing invaluable data on their experiences throughout the
evaluation.
Changes in sustainability
Respondents were asked whether they thought they had seen any change in their
organisations strength, sustainability and resilience during the 12-month funded period.
Respondents were overwhelmingly positive, with 82 respondents stating that it had
improved (94%), of which 53 (61%) said it had improved greatly.
1
Snapshot survey 1 had a response rate of 67%, survey 2 of 49% and survey 3 of 41%.
LSF EVALUATION BULLETIN: SNAPSHOT SURVEY 4
Worsened a lot 1% 1
Respondents were also asked the extent to which they thought these changes could be
attributed to LSF. Again, respondents reported positively, with the vast majority (83
respondents, 96%) feeling that it had been partially or predominantly due to LSF.
Respondents were asked to choose from a list of six factors that could help to explain any
increase in their organisations sustainability they had seen; which were a mixture of factors
associated with how LSF functioned and how the grant holders organisation worked, as can
be seen in figure 1.
The most popular factors were the staff of the organisation (81 respondents felt that this
was important or very important; 93%), the advisor (69 respondents; 81%) and the board
of trustees (67 respondents; 78%). Only 32 respondents (37%) considered the business
advisor to have been important.
2
LSF EVALUATION BULLETIN: SNAPSHOT SURVEY 4
Figure 1: If you have seen an improvement in your organisation's sustainability, how important do
you think the following factors were in helping to explain this? (base 87)
Not important
Your business partner
Moderately/slightly important
Very important/important
Your volunteers (not including the
business partner or your Board)
Your advisor
Your staff
0 20 40 60 80 100
Respondents were also asked about the different elements of the LSF programme that had
helped their organisation improve its sustainability, choosing from five pre-determined
factors. In line with findings reported in the recent bulletin on outcome and learning, based
on the end-of-project reports, the most popular were freeing up staff time (76 respondents
or 88%) and bringing in external people to help and advise (67 respondents or 79%).
Figure 2: Which aspects of the way LSF worked have helped your organisation improve its
sustainability? (base 87)
Not at all 21
Sharing learning and insight with other grant holders 67
A little 13
A lot
9
Needing to report on progress and changes 65
26
7
Completing the ODT 48
45
1
Bringing in external people to help and advise 20
79
0 10 20 30 40 50 60 70 80 90 100
3
LSF EVALUATION BULLETIN: SNAPSHOT SURVEY 4
The high level of agreement with bringing in external people both backs up the earlier
finding of the perceived importance of having an advisor (see figure 1) but at the same time
contradicts the finding that the business partner was not valued so highly. This illustrates
that the advisor and business partner, while both a form of external input and support, have
often been seen and valued in quite different ways, something that will be examined in
more detail in the final evaluation report.
With the exception of a decline in volunteer numbers (72 respondents, or 84%, felt it had
not impacted at all)2, the majority of respondents tended to feel that these factors had
limited the impact of LSF to some degree (see graph 3), emphasising the importance of the
wider environment, something that will be examined in more detail in the final evaluation
report. The most notable was a rise in demand for your services, with 58 organisations
saying that it had impacted a lot or a little (67%).
Figure 3: Have any of the following wider factors limited the impact that LSF was able to have on
your organisations sustainability? (base 87)
Not at all 84
A little Decline in volunteer numbers 13
3
A lot
33
Rise in demand for your services 47
20
49
Decrease in income to your organisation 30
21
43
Staff changes within your organisation 36
21
0 20 40 60 80 100
2
This could also be because the respondents had not observed a decline in volunteer numbers during the
funded period. We will be examining the effect of changes in volunteer numbers in our forthcoming analysis of
the ODT scores for grant holders.
4
LSF EVALUATION BULLETIN: SNAPSHOT SURVEY 4
Outcomes
Respondents were asked about the extent which they agreed or disagreed with a series of
statements that described different changes and outcomes. These outcomes were based on
those identified within the Theory of Change which has been developed by NCVO and
Resources for Change as part of the evaluation of LSF, and the programmes original
objectives. For a more detailed exploration of outcomes and learning from projects, see the
evaluation bulletin examining the information provided in the grant holders end-of-project
forms.
Respondents reported very positively on all of the outcomes listed, with every one seeing
more than half of respondents saying they had observed these changes over the course of
the 12-month funded period. The most commonly cited, however, were thinking
differently (78 respondents; 91%), working more effectively (76; 89%), gathering better
evidence about our impact (70; 81%) and being better able to adapt to change (70; 80%).
Figure 4: Thinking about your position as an organisation now compared to before you started
your LSF project, to what extent do you agree or disagree with the following statements? (base 87)
0 10 20 30 40 50 60 70 80 90 100
In an open question, respondents were also asked whether they had experienced any
unexpected or unintended outcomes as a result of their LSF project, which they were able to
5
LSF EVALUATION BULLETIN: SNAPSHOT SURVEY 4
describe in their own words. Fifty-seven organisations provided a response to this question,
generally in limited detail, which can be summarised in three main areas:
Most commonly this included bigger changes to the governance of the organisation than the
respondent had originally thought, but also included deeper impacts around developing a
better understanding of services users, developing new partners and networks, or the staff
team becoming closer than expected:
We were able to develop a new strategic partnership with a new partner which was
also unplanned before the project started and this has helped us secure our future.
Didn't expect to lever in such a tranche of new funding, which has secured our
position for the next 3 years.
(b) Changes outside of the project (both internal and external to the organisation) had
unexpected impacts.
This included the unplanned and unexpected impact of staff restructures, staffing changes,
IT problems, events external to the organisation, and in one instance, the impact of an
unsuccessful merger, all of which created new challenges for the work of the organisation
and the project. In some instances, however, respondents noted that the receipt of LSF
funding had made them more able to cope with these challenges:
There were some things out of our control such as some key staff leaving, and a change
in staff with our key partner, both of which slowed the progress of our work, but the
impact would have been far worse had we not had the LSF funding.
(c) The involvement of the business partner had unexpected results and elements.
This included the business partner making less of an impact than the grant holder had
hoped for, cultural differences between the business partner and grant holder (including
differing timescales and working practices) which creating negative impacts, and the
business partner organisation collapsing in one instance:
We didn't gain from our Business Adviser's input in the way we had hoped. We
thought we would have a critical friend who could input from a business perspective
and support us with more efficient and effective ways of working.
6
LSF EVALUATION BULLETIN: SNAPSHOT SURVEY 4
differently, suggesting a high degree of satisfaction with the programme. Of those who did
make suggestions, they can be grouped in to three main areas:
(a) Changes to the timing of the programme and its funding (30 respondents).
Respondents most frequently stated that a longer timer period, typically between 18 and 36
months, would have allowed them to have a greater impact (23 organisations mentioned
this):
'Getting this project completed in a year was a real struggle. It takes much more time
for change to happen and for the organisation to understand the impact of the
changes we have made as a result of the funding.'
'A longer-term period would have been more sustainable and ensured a longer-term
impact.'
Timing issues were also mentioned around having had better and more timely delivery of
funding at the start of the project, as well as having had more time to find an advisor:
'There was a problem releasing the money at the beginning which created a lot of
uncertainty and delayed things at a crucial time for us.'
Connected at some level to timing, some respondents described how having options for
follow-on funding would have allowed them to do more. Funding for core costs, for capital
investment, or simply more money was also suggested:
'We would have been better able to focus on our internal systems and income
generation if we had had some core costs covered; which would have made for an
even more impactful project.
'We would really have benefited from being able to seek a significant proportion of
capital money to support some of the changes suggested by our advisors and advice
received prior to the start of the programme alongside the development money.'
(c) Changes to how the advisor and business partner worked (5 respondents).
Some respondents felt that having had less emphasis and focus on business partners and
consultants would have allowed them to have had a greater impact, while in one instance,
greater clarity about how much the business partner should have been paid was
recommended.
7
LSF EVALUATION BULLETIN: SNAPSHOT SURVEY 4
commonly they were mentioned. The responses reflect both perceived weaknesses of LSF
(the things that grant holders wanted to change) as well as strengths (the elements they
wished to retain and felt worked well). As will be seen there is some overlap with previous
comments on what The Fund could have done differently, although in this question
respondents were encouraged to think more broadly than LSF.
(a) Timing
This was the most common recommendation, with 30 respondents making some form of
recommendation around programme timing, most commonly in terms of having a longer
funding period than 12 months:
'Change projects take time to initiate and to filter through an organisation and day to
day constraints and commitments make it hard to focus on strategic tasks quickly.'
Similarly, some recommendations were made around allowing for project extensions, or
more specific elements such as ensuring that any advisors are involved from the start.
Fifteen organisations made recommendations around the role of external partners in similar
funding programmes. As has been seen in this bulletin and others, the role of business
partners and advisors in the LSF programme has been highly popular as well as more
controversial, with some respondents recommending that similar programmes should not
contain business advisors (as they had found them to be a more challenging and less
beneficial element of LSF) whilst others strongly recommended that it should be a part of
future programmes of this nature (seven respondents):
'We did get a business partner, but they really didn't add very much to the whole
process. So I would not make this mandatory if designing something in future.'
Other recommendations included making the process simpler or providing more support to
identify partners or throughout the relationship, as well as tailoring it to organisations:
'To think more carefully about the advisor and voluntary business partner roles and to
adapt these to different circumstances for each individual project.'
(d) Flexibility
Respondents recommended having greater flexibility, to allow for wider, external changes
that grant recipients may experience:
8
LSF EVALUATION BULLETIN: SNAPSHOT SURVEY 4
'Ensure there is enough flexibility built into the grant to allow for unexpected changes to
the organisation.'
(e) Clarity
These recommendations were focused on having greater clarity around timescales and
decision-making processes and milestones.
(f) Funding
Some recommendations were made around providing core costs or funding to cover the
salaries of staff (reflecting similar comments made in the previous section), as well as
retaining the ability to backfill staff roles, which was a popular element of LSF:
'Allow for funding directly for salaries so that senior staff can dedicate time to the
strategic development.'
Respondents also made recommendations around how a similar programme could function,
with comments including having less monitoring, using an improved internal assessment
tool, and encouraging applicants to be more realistic in their planning, in terms of the
ambitions of what their projects could achieve (the latter point is also reflected in the
bulletin examining end-of-project monitoring forms in which grant holders frequently felt
they had been overly ambitious in their applications).