Year Two Evaluation Report, executive summary
Research Scotland Methodology
We analysed quantitative data from SUP relating to delivery against HLF targets and spoke with SUP staff about development in year two. We created a survey to gather a wide range of information on outcomes, and also planned six case studies to gather more in depth information from participants.
The survey received a poor response and so the evidence around outcomes is not as strong as we had anticipated. We recommend running the survey again during year three, possibly offering an incentive to encourage participation.
We completed five case studies with new audiences or young people. We will work with traditional groups in the future, as they become more involved in SUP projects.
Delivery against targets
In year two there is strong evidence that the programme is on track with delivery against the agreed targets. It has already exceeded the five-year target for the number of people involved in projects or training sessions, and special skills workshops. Most other activity is on track and can reasonably be achieved in the remaining delivery period.
There have been challenges in relation to recording the number of people engaged that are new to heritage. There is conflicting data over the exact number of people engaged that are new to heritage and SUP’s figures estimate this to be 15%.
We recommend SUP management improves measurement against this target to increase accuracy of data and the addition of a routine questions on ethnicity, to understand the extent to which it is meeting the targets towards engaging new audiences.
There has been limited progress towards online registration, with only 288 registered users. Significant changes will be required to meet the five-year target of 4,000. We recommend SUP management considers their target for the number of registered SUP users, and either plans new activities to achieve this, or reviews the target.
There is some good evidence to demonstrate that SUP has delivered positive outcomes for participants. Qualitative and quantitative evidence suggests that participants enjoyed their activities and the programme helped to:
- improve their skills and capacity;
- improve their confidence; and
- improve their understanding and appreciation of the urban environment.
There is more limited evidence about empowerment of individuals, community level outcomes, and outcomes for urban heritage. We would expect outcomes such as this to become more apparent in the longer term, and potentially outside the delivery period of the programme.
The evidence we have gathered and reviewed highlights very positive messages about the programme and SUP staff. Case study participants said that they appreciated SUP staff for their skills and approachable manner, and also felt that they were flexible and adaptable to their needs.
Participants said that the practical activities were engaging, stimulating and useful. Where possible, SUP and project participants connected the project to other activities and frameworks. This worked well to link in with wider activities in the community.
The main challenges raised by people we spoke with were around the nature of the projects themselves, rather than SUP management or activities. These were mostly to do with defining project ideas, unrealistic expectations, or the continuity and capacity of the groups themselves. Generally, people felt that the team had supported them with these challenges.
Some participants said they would like more time and support (in particular, new audiences) from SUP staff, further support with contributing to the archives.
Quotes from participants
“I feel more motivated and knowledgeable about documenting our history and how to undertake the task with new ideas on how to present the information collected.” Participant from Clydesdale Rowing Club, Glasgow.
“I thought history was boring; but it isn’t now.” St John’s Primary School pupil, Rosyth.
“A brilliant, interesting day – introducing me to new avenues, resources and groups. A real eye opener!” Event participant, Aberdeen.