What has IcARUS taught us? Beginning the evaluation phase

Over the last three and a half years, our-multi-stakeholder consortium has been working tirelessly to co-produce, co-develop and co-deliver innovative prevention tools, in the pursuit of addressing some of the most challenging urban security concerns: safe public spaces, juvenile delinquency, radicalisation, and organised crime and trafficking. As we move toward the final stages of our four-year endeavour, we bring together all of the work of our partners in understanding the highlights and challenges of such a forward-thinking approach.

The University of York, alongside the Erasmus University Rotterdam are undertaking a comprehensive evaluation process across three key phases: training, delivery and outcomes. Taking indicators developed by Erasmus, we have begun an holistic analysis to understand how each tool has met the tailored objectives for each of the cities. We have observed training events across all cities, and obtained feedback from both trainers and trainees. We have implemented attitudinal / behavioural change surveys to understand the impact of tools on those in receipt of them, conducted observations of events and demonstrations to identify engagement levels, and administered focus groups with stakeholders and participants to assess satisfaction, understanding and sustainability, amongst a host of other measures.

Further, we are also assessing the extent to which each tool has met the objective principles of IcARUS. This involves examining how the tools have understood, incorporated and delivered in relation to communication, building trust, promoting gender-inclusivity, gaining political backing and creating tool sustainability. This not only enables us to speak to our overarching mission of the IcARUS project, to provide innovative, human-centred preventative solutions for urban security concerns, but to ensure that we can deliver lasting impact beyond the lifespan of the project.

Oversight and organisation of the evaluation work has been divided – York is responsible for Nice, Stuttgart and Riga, and Erasmus is responsible for Lisbon, Turin and Rotterdam. So too, our evaluation methods are varied and tailored to the particular tool and city. Nonetheless, our guiding principle is centred on providing space for the voices of those involved at the different stages of the tool to be heard. This is critical for several reasons. Firstly, hearing from different voices, at different stages of the process, ensures that our evaluation considers the multiplicity of experiences which have occurred throughout the tool’s lifespan. Secondly, it enables us to account for and reflect on the multi-layered challenges faced by different stakeholders involved in the process. Thirdly, it provides an evidence-base for identifying challenges and proposing solutions. Finally, it enables us to holistically reflect on the benefits and importance of multi-stakeholder solutions to urban security challenges. 

As we move forward, we will turn our attention to analysing the rich datasets obtained through the efforts not only on our part but critically also by our city partners and their stakeholders. Our results will be presented towards the end of the project, where we hope to share not only important insights into the lessons learnt during IcARUS, but also a series of evidence-based recommendations which will ensure each of our tools continue their important work beyond our project end date.