The Co-Evaluation Tool for citizen science can also be applied to Citizen Observatories. It aims at capturing how participation in a project has changed knowledge, practices and policy. This tool builds on the Place Standard Tool;
it focuses on ten indicators of the different types of impact that could result from a project.
Observatory.
You may also be interested in:
I want to measure impacts of the Citizen Observatory…
…via a suitable approach
CoP: The WeObserve Impact Community of Practice brings together practitioners of Citizen
Observatories and citizen science to share and learn di erent ways of capturing impacts, including via participatory evaluation.
This tool, which encompasses a method and canvas, is designed for community members and citizen science practitioners wanting to evaluate a project. It provides prompting questions for discussions among participants, allowing communities and project managers to consider these indicators in a methodical way. This tool can be used at the end of a project to assess the impacts and changes achieved. It can also be useful at the beginning of a project as a way to collect an understanding among stakeholders.
Example of completed Co-Evaluation Tool ©GROW Observatory
The Community Level Indicators (CLI) tool is designed for community members and citizen science practitioners wanting to start a new project and is designed to be used throughout a citizen science project. The CLI method involves asking community members to identify extra information that a community in a citizen science project can collect to complement and contextualise sensor data. For example, if a community is concerned about air pollution in their area, they might start a campaign to reduce the number of cars that drive on their street while using a sensor to monitor changes in air quality. The CLI, in this case, could be the measurement of car traffic on the street over the same period of time that the air quality is also measured and comparing the data to see if there is any correlation between the car traffic and the air quality. This toolkit helps participants to track the potential impact of their actions. The CLI tool helps participants to collaboratively choose what information will be collected, and how. This tool can also be used at the end of a data collection period, to see how actions have made a difference.
Participants of the Making Sense project in Barcelona complete the CLIs tool worksheet to decide
which type of indicators they would like to collect
©Making Sense
Insight Workshops
Insight workshops are a great way to reflect on the collaborative efforts of the participants and organisers, as well as engaging additional stakeholders who are not part of the core membership. The sessions can be formal or informal but should aim at iteratively presenting insights and results from the Observatory as they emerge rather than waiting for traditional publication, as well as conducting rolling evaluations on what went well and what could be improved.
There are many different ways of conducting an insight
workshop, but having a space where people feel comfortable A participant fills out a feedback questionnaire about the pilot during a final insight
Other helpful materials can be post-its for noting down pros and cons of the project, and sticker dots, which can be used for voting.
Asking participants to fill out feedback questionnaires during these
Impact stories
Example from the Making Sense project
sessions is also a good way to have them reflect on the activities of the project.
You can use impact stories to report on changes that result from your Citizen Observatory. An impact story is a narrative about your journey from when you started the Citizen Observatory. In your story, you will explain the issue you were trying to tackle and talk about moments or events when you were aware of a change that had occurred in relation to the problem you were working on. For example, you can use an impact story to report on a moment or event where you felt empowered in a discussion with authorities because you had gathered data and insights through your work as a citizen scientist.
The WeObserve Impact Community of Practice developed an approach that focuses on capturing and promoting policy impacts. The Impact Inquiry Instrument is available here.
For Making Sense, participatory evaluation was crucial to the development of the project and also the participants’ awareness of what they had learned about the environmental challenge. During a campaign in Barcelona, the participants got together for a workshop on CLI. They first agreed on a sensing strategy, which included where and when they would be collecting data on noise pollution using the Smart Citizen Kit.
They then used the CLI tool and worksheet to collectively decide on one or two other indicators that could be used for data annotation in combination with the sensor strategy. The workshop helped participants think about the problem differently. For example, some of the participants wanted to monitor the public presence in the area where the data on noise levels was being collected and then compare this information to the noise sensor data.
This allowed participants to overcome a culture of blame and to instead see the possibilities of collaborative decision-making.
In one case, a participant used her discussing the CLIs the participants were able to plan approaches that would build on the sensor’s datasets to reveal deeper insights into the issue of noise pollution.
The incremental and regular release of outputs can help stakeholders develop a common understanding of the issue and solutions to it. In the long run, doing this may help stakeholders reorient the Citizen Observatory to new and emerging questions. For public authorities, such openness also facilitates the transparency, accountability and continuity of public participation.