A data management plan will help you determine what type of data to collect, how to collect it, and what additional resources you will need when designing your protocol. With good data management, data tools, and quality control measures, data from citizen science projects can meet or even exceed the reliability of conventional science. You can gain quality data from volunteers by ensuring they are properly trained, the protocol is relatively easy to understand and conduct, and you can effectively store and manage data.
Identify quality control measures
Data quality is of high concern for all Forest Service monitoring and research projects. The Forest Service is a science-based organization and the agency’s credibility would be damaged if the data used to make land management decisions were found to be unreliable. Volunteers can be just as effective and accurate as professional scientists and technicians as long as your protocol is well defined and volunteers understand how to follow it. Volunteers are often more conscious while collecting data because they do not rely on their professional judgement. Below are suggestions for data quality control measures.
Assign simpler tasks
One way to control for data quality is to keep the volunteer’s tasks simple enough to limit opportunities for human error. Use terms and measurements that easily make sense to volunteers. For example, documenting the presence of one type of rare plant could be significantly easier than identifying up to 50 different mussel species. Either find a way for a professional to join volunteers for surveys, have a professional conduct some quality control on data, or avoid using non-professionals for complicated data collection that cannot be checked for accuracy.
Complete trial phase
You may also try a “trial phase” of your project to see if your protocol is easily understandable and collects useful information. You might find, for example, that your protocol does not have a realistic expectation of the volunteer's abilities, or that volunteers are observing lots of features you thought would be rare but is now cluttering your data. After the trial phase, you can modify the protocol as appropriate.
Test with professionals
If possible, you might consider selectively repeating your protocol with professional scientists/technicians to test for quality control.
Individual volunteers should be evaluated to make sure protocols are being followed so that data is reliable and useful. This usually happens by compiling notes from data and sharing the best examples. If there are a significant amount of mistakes it might be best to display examples anonymously and without embarrassing anyone. Some less common mistakes might be best discussed on an individual basis.
Design your protocol
How will you collect your data? First, decide whether your project’s data collection will work best in an online or field environment. Is this a project where participants can be trained and collect information virtually without needing in-person support, or is the project site-specific and formatted to have in-person training or data collection? Would this information be best collected on a paper form or in an app?
If you plan to collect data by having volunteers fill out paper forms, make sure volunteers know how and when to turn in data sheets. Consider entering data within 48 hours of collection rather than at the end of the season, and check if you have the staff time to upload the data to the relevant databases (e.g., Forest Service Natural Resource Manager databases and federal databases like the breeding bird survey). If that turns out to be too difficult, you might consider other options to enter data – for example, working with a partner like a state natural heritage program. Some projects require mail-in forms; consider how you will track and properly enter them.
Many projects can collect and store data with just volunteers and their smartphones or simple measurement tools that your unit likely already owns. Of course, relying solely on mobile devices can limit the ability to collect data in isolated areas and exclude people who do not use smartphones. Many apps currently allow users to collect data in areas that are out of service and then upload them when they reach service again – this should be researched ahead of time. Location accuracy of different devices and settings required to receive the proper metadata (e.g., must turn on location service) also need to be considered.
There are countless citizen science apps already out there, and you can even develop your own using ArcGIS Online. Listed below are some commonly used citizen science apps (we hope to expand this list soon).
- Tools from ESRI: The Forest Service has an institutional subscription for ESRI GIS mapping tools, some of which can be used to collect data for citizen science. Collector, Survey123, and GeoForm can be used to develop custom apps and forms for your project. ESRI has a page with citizen science resources, and the Forest Service Citizen Science and Crowdsourcing Community of Practice has held several webinars on ArcGIS Online (AGOL) tools.
- eBird App: Developed and managed by the Cornell Lab of Ornithology, eBird is an app and website for documenting bird abundance as well as the presence or absence of species. Birders simply enter when, where, and how they went birding, then fill out a checklist of all the birds they saw or heard during the outing. eBird provides various options for data gathering including point counts, transects, and area searches. Automated data quality filters developed by regional bird experts review all submissions before they are entered into the database, while local experts review unusual records that are flagged by the filters. eBird data are accessible to anyone via the eBird website and is integrated into the Avian Knowledge Network.
- iNaturalist App: iNaturalist is a smartphone app and website that can be used for recording observations of living things anywhere in the world. iNaturalist volunteers capture photos or sounds and upload them to the app or website. The observation can in turn be downloaded from the site as a KML or CSV and uploaded to ArcGIS. Users can identify their observation themselves, or they can simply give it a broad label like plant or mammal and iNaturalist’s community of experts will identify the picture. As of July 2017, iNaturalist automatically suggests an ID based on machine pattern recognition and what species are found in the area. Enter the geographic area where you are interested in gathering data, i.e. your Forest or Ranger District, and invite other users to join this project and upload their observations. A good example of the Forest Service using iNaturalist volunteers is the Kaibab National Forest Citizen Science Project. A downside to iNaturalist is that the forms and information collected cannot be customized.
Scheduling collection times
If possible, have double assignments on the same sites. This will help ensure full data sets in case of volunteers quitting or unexpected absences. Volunteers tend to feel camaraderie with those on the same site and also relief in case they cannot make the full commitment as intended. Some project leads have found that having 3-4 people per site at one time is the most sustainable depending on the amount of work needed.
If volunteers go out as a group with a Forest Service staff lead or partner lead, establish clear scheduling expectations and how they will be determined. Also, make it very clear how cancellations and changes will be communicated and when. For group monitoring dates, you may decide to use an online scheduling poll, like Doodle, to easily determine everyone’s availability and to pick the best date.
An end-of-season wrap up is recommended even though this may not allow enough time to have the current data compiled. It’s a good way to collect feedback, thank volunteers and discuss observations in person. To condense gatherings you may consider combing season wrap ups for multiple monitoring programs so everyone can meet each other and hear about other wildlife programs too.
Decide how you will analyze data
Who will analyze your data and how will they do it? This depends on your project design and goals. For example, youth centered projects often involve participants in analysis of data for educational benefits. In this case, consider the space, time, computers, and applications students might need to analyze data – a local school or college may be able to provide classrooms and computer labs. In other projects, resource specialists or researchers (who could come from either the Forest Service or other entities) undertake data analysis.
Ensure data preservation & availability
Though it depends on what kind of data you are collecting and its intended end use, it is important to ensure the data you have collected is easily available to Forest Service staff. Since there is often high turnover in Forest Service units, ensure the data will be accessible to staff who may join the unit in the future or if you leave the unit or the agency entirely.
Upload your data to the relevant FS enterprise database if possible. Evaluate ahead of time which data need to be restricted from public access (e.g., heritage or wildlife data) and confirm your data collection tool and/or method of sharing allows for privacy of sensitive information.
Plan to preserve your data for the long term, meeting agency data retention policies and practices as well as the standards of the National Archives and Records Administration.