What is the Aerial Firefighting Use and Effectiveness Study?
Aerial Firefighting Use and Effectiveness (AFUE) is a chartered study by the U.S. Forest Service Fire and Aviation Management Program. The study is an exploration of critical questions such as: where, when, why, how, and how well aerial firefighting resources are being used during wildland fire response. The desired outcome is to support training, mission selection and execution, and overall aerial fleet planning to enhance effectiveness and cost-effectiveness, potentially reducing aviation and fire suppression costs by answering a general, but complex question: "What are the best mixes of aircraft to do any fire suppression job?" AFUE products and outcomes are intended to assist and inform both aerial firefighting in particular and wildland fire management in general.
When did AFUE begin data collection?
The AFUE Study was implemented in mid-2012 to collect data and analyze the utilization and effectiveness of U.S. Forest Service and U.S. Forest Service contracted aerial assets that deliver wildland fire chemicals and water to support incident objectives. AFUE executed the first full fire season of aviation use and effectiveness data collection utilizing four full-time, dedicated ground crews across the lower 48 states from late May into October 2015. With the data collection aircraft reaching full operational capability in 2016, AFUE is targeting opportunities to collect data simultaneously from the air and ground on the same fire. Ground and aerial data collection with the four observation modules and dedicated aircraft is continuing throughout the nation during the 2017 fire season with plans to collect and analyze at least a five-year dataset.
What type of aviation data does the study collect?
AFUE collects aircraft drop location and information including the objectives and outcomes for each drop. Data also includes terrain, fuel type, fire behavior, weather conditions, and other factors that may influence drop effectiveness. The purpose and characteristics of the tactical missions that the drops support are documented so that mission effectiveness (and the contribution of individual drops to the mission) can be evaluated.
What is AFUE not doing?
AFUE is not a study of wildland fire chemicals, is not a program to evaluate individuals (pilots/incident commanders/air attacks), is not responsible for reporting chemical misapplications in avoidance or sensitive areas, and is not a comparison of federal vs. state resources.
What external reviews prompted the AFUE study?
The federal Fire Management Board, at the request of the federal Fire Executive Council, established and assigned the Wildland Fire Aviation Strategy Task Group to address the
the United States Government Accountability Office (GAO) Report 13-684. The resultant Interagency Plan for a Wildland Fire Aviation Strategy identified the National Wildfire Coordinating Group National Interagency Aviation Committee as the proper entity to accomplish many of the plan’s recommendations.
The AFUE study is supporting NIAC by conducting fire aviation data collection and analysis to meet the expectations of the GAO, including the development of the next generation Interagency Aviation Strategy for Wildland Fire. The collaboration between NIAC and AFUE addresses the following at the request of the Fire Management Board: 1) GAO Recommendation #1: Performance and Effectiveness of Firefighting Aircraft; and 2) Tasking Elements 1-4 of NWCG Tasking Memorandum 15-001.
How does the AFUE study collect data?
The AFUE study collects aircraft drop location and information including the objectives and outcomes for each drop along with terrain, slope, fuel type, fire behavior, weather conditions, and other factors that may influence drop effectiveness. The study employs four data collection modules consisting of three firefighters each, all of whom are operationally qualified and self-sufficient. The modules are made up of firefighters qualified at the single-resource level and above with numerous years of operational experience including interagency hotshots, smokejumpers, engine and helitack crewmembers and wildland fire use modules. The four data collection modules have duty stations in regions with high aviation usage, but are dispatched by an AFUE coordinator to all locations based on fire activity nationwide. Module crewmembers also staff a single aerial observation aircraft dedicated to AFUE.
AFUE ground and aerial observation modules work in partnership with incident commanders, local forests, dispatch centers and airtanker bases. The presence of AFUE resources is not intended to influence the decision process of incident management, nor individual outcomes of fires where data is collected. If incident support of the AFUE observation modules becomes critical for mitigating immediate hazards, crewmembers disengage the data collection mission and support the incident commander as requested.
A data collection aircraft became operational in August 2015. The data collection sensor carried by the aircraft is capable of identifying and tracking drop locations, fire location, and progression. The aircraft became unavailable in August 2016, but will be replaced to maintain the dedicated aerial data collection capability.
What is the AFUE hierarchical data structure and terminology?
The AFUE data collection hierarchical structure and terminology includes:
- Resource Actions (drops) – these are the smallest units of analysis, including individual aerial suppression drops.
- Tasks – linkages between multiple resource actions (drops) using the logic of purpose and proximity (both temporal and spatial) to focus efforts toward establishing operational and strategic objectives; both aerial and ground tasks are mapped.
- Campaigns – combinations of related operations aimed at achieving strategic and operational objectives within a given time (typically one shift or less) and location. Campaigns combine air and ground tasks into a set of independent or connected tasks for a measurable amount of time, such as during initial attack, or the period when an emerging fire transitions to extended attack or a division of a large fire. This is a unit of analysis where combinations of tactics are designed to execute strategies, and holds great potential to improve operational probabilities of success.
What is the methodology for the AFUE study?
The AFUE study employs a three-pronged approach to data collection. At the national and seasonal scale, existing databases containing aviation records are summarized to reveal overall use trends. At the incident level, four dedicated “non-standard” ground observation modules consisting of three specially trained, line-qualified firefighters document overall incident and campaign objectives. In addition, a management directed monitoring aircraft provides objective measurements of aerially applied chemical and water locations, fire characteristics including fuels, fire behavior and rates of spread at the time of application, and outcomes. This aircraft may or may not overlap with the ground observation modules. At the tactical level, ground observation modules document individual tactical objectives, supporting and/or primary resources, drop characteristics, fuel types, fire behavior, weather, timeframes and tactical outcomes.
Is AFUE data collected during an incident available to share with Incident Management Teams (IMTs) and local units?
AFUE modules frequently receive requests from IMTs or local units to share incident-specific information during and after data collection assignments. Some information can be made immediately available, such as general locations and times of aerial suppression activity. Some information may not be immediately available. In cases where the information requested includes data that requires quality-control review or aviation management approval for release, AFUE module crewmembers forward the request to AFUE leadership, who work directly with the requestor to document the request and ensure that the information released is accurate and managed appropriately after delivery.
AFUE ground modules had collected data on 76 fires in 2015 and about 80 fires in 2016. Over 13,000 individual resource action records (drops) and ground tasks are in the AFUE database. A majority of the fires in the 2015 dataset were in the initial attack phase when AFUE began collecting data on them. Ground modules and the data collection aircraft will continue to collect data in 2017 and for several more fire seasons. Preliminary analysis framework drafts have been provided to USFS Fire and Aviation Management leadership for feedback and review. The statistical significance of the dataset is growing quickly and the dataset will be augmented using available national data on wildfires and aircraft usage. Members are working with interagency partners to recruit and train additional data collectors using simplified data collection tools.
The GIS dataset being built by aerial and ground observations consists of features like spatial location and time, as well as additional information specific to each type of feature (or feature class). Data collectors use mobile devices (smartphones or tablets) with GPS capabilities to collect geospatial data in the field. Analysis includes spatial, temporal, probabilistic, data mining, visualization, and statistical modeling components.
What type of aerial firefighting aircraft will data be collected for in the AFUE study?
Single engine airtankers, large airtankers, very large airtankers, helicopters, and water scoopers are included in order to understand the relative contributions of all types of aerial assets that deliver wildland fire chemicals and water to support interagency incident objectives.
What type of personnel are involved with AFUE data collection?
The agency staff involved in AFUE data collection are dedicated full-time to the data collection mission and include four Module Supervisors, eight or more Crewmembers, one Data Manager, one Analyst/Economist, one Fire Technology Specialist, and one Project Leader/Operations Coordinator. All AFUE personnel who are dedicated to operational data collection are qualified at the single-resource level and above with numerous years of operational experience including interagency hotshots, smokejumpers, engine and helitack crewmembers and wildland fire use module members.
What are the primary responsibilities of the AFUE ground modules?
The AFUE ground module’s primary responsibilities include:
- Establish interagency contacts to introduce and explain the program and its purpose.
- Track fire danger indices and pre-position module as appropriate.
- Pre-position on Forests/Districts/Local Units or work on incidents with appropriate pre-approval.
- Conduct on-site data collection.
- Provide support and data to incidents when applicable or by special request.
- All AFUE ground module personnel are self-sufficient, fully funded, operationally experienced, and Red Carded.
What are the primary responsibilities of the AFUE aerial platform?
The AFUE aerial platform primary responsibilities include:
- Coordinate with AFUE Operations Coordinator for data collection priorities.
- Establish protocols with Forest or IMT.
- Coordinate with local GACC and dispatches.
- Communicate and coordinate with incident Air Attack as appropriate.
- Collect imagery showing fire behavior, drops and their footprints on the ground, and the fire interaction with drops with a geospatial data collection tool onboard the aircraft.
- Download data after returning to airport.
What kind of aircraft will collect the AFUE data?
AFUE is currently working with U.S. Forest Service Aviation Management to equip, staff and coordinate a new aircraft for the mission. AFUE aircraft are management directed resources and will not be ordered or assigned to incidents for suppression efforts. However, the aircraft and aircrew will coordinate with local dispatch centers and incident personnel before, during and after any data collection missions under the oversight of the AFUE Duty Officer. Positive communications and flight following will be maintained with the responsible dispatch center(s) for the duration of any mission utilizing existing flight following standards. The AFUE aerial platform will not impede or engage in tactical operations and will operate at 8,000 feet AGL or higher at all times.
How does the AFUE aircraft operate?
The mission of the AFUE aircraft is to capture firefighting aircraft drops using an on onboard sensor so that the drops’ effectiveness in meeting tactical and strategic objectives can be evaluated. The type of sensor being used allows the airplane to orbit over the fire at 10,000ft AGL, capturing individual drops at high resolution within a frame several miles wide. The AFUE Duty Officer and Operations Coordinator attempt to dispatch the aircraft to wildfires based on a combination of what aviation resources are sent to that wildfire and AFUE’s dynamic data collection priorities. A Fixed Wing Flight Manager is on board the AFUE aircraft during all missions. The AFUE aircraft does not execute the ATGS mission and is not requested nor dispatched through ROSS. Data collected by AFUE and from other sources will be utilized to inform decisions that determine the composition of the federal interagency aircraft fleet that supports the management of wildland fire.
What is the interaction between AFUE resources and the incident?
AFUE ground and aerial observation modules work in partnership with incident commanders, local forests, dispatch centers and airtanker bases as well as other cooperators and incident personnel. Their primary mission is data collection. They do rarely disengage from data collection and engage operationally, but only when asked by the Incident Commander. Relevant intelligence can be shared with incident resources, but the presence of AFUE resources is not intended to influence the decision process of incident management, nor the individual outcomes of fires where data is being collected.
How did the AFUE Study evolve and why?
Various external reviews of the Forest Service aviation program, and large airtankers in particular, have highlighted a lack of use and performance information to justify the size, scope and cost of both the existing and planned fleet. In response, the Forest Service formed the Aerial Firefighting Use and Effectiveness Steering Committee in 2012. The Committee initiated a pilot program to explore and develop data collection methods that could be used to address the findings of the external reviews, as well as additional questions that were specifically identified by Forest Service Fire and Aviation Management. The pilot program identified that an intensive, nationwide, multi-agency, multi-year field data collection program was required to enable a complete, unbiased analysis of aerial firefighting use and effectiveness in the context of all wildland firefighting missions, objectives, and environmental conditions.
Analyses of AFUE data are necessary to inform responses to a recent GAO report, and include plans to identify ways to improve the current fleet composition and management, providing a more robust foundation for future aviation strategic planning and operations. The GAO report was the latest review to highlight the need for empirical data on the effectiveness of aerial firefighting as a component of greater fire management strategies. Since the effectiveness of any tactical approach to wildland fire management is highly dependent on the environmental conditions at the time and place of tactical action, and the tactical action may or may not occur exactly as planned, simply measuring outcomes is insufficient for determining effectiveness under all circumstances.
What are the AFUE expectations and deliverables?
Expectations and deliverables of the AFUE study include:
- Develop, test, evaluate and implement processes, technologies, evaluation criteria, and performance measures which quantify the use and effectiveness of firefighting aircraft including single engine airtankers, large airtankers, very large airtankers, helicopters, and water scoopers.
- Provide program recommendations, advice and counsel to the Director Fire and Aviation (FAM) through the Assistant Director. This includes, but is not limited to: aircraft fleet; program standardization; operations; risk management and safety; best management practices; cost effectiveness; and the development, implementation, standardization and quality assurance of aerial firefighting aviation operations.
- Work with all applicable interagency partners, cooperators, vendors, manufacturers, and subject matter experts to ensure appropriate coordination, collaboration, and information sharing specific to AFUE’s mission.
Have there been any results developed from the USFS AFUE (Aerial Firefighting Use and Effectiveness) Study?
Although the AFUE study was initiated with voluntary data collection efforts in 2012, data collection from 2012-2014 was done mainly as a methods development process and is not sufficient to provide statistically defensible analysis and results supporting the objectives identified by senior USFS leadership or GAO. The 2015-2016 data includes detailed records on approximately 13,000 individual drops from 156 fires. Since this data includes fires from many jurisdictions, fuel, weather and terrain conditions, the process of statistically characterizing the sample in terms of the population it represents requires merging data from many different sources. This work is ongoing, even for the 2015 data, but study management expects the process to be much quicker for subsequent years.
AFUE’s progress to date has supported successful requests for closure of the first two GAO recommendations for both USDA and USDI. AFUE continues to work with USFS and BLM leadership on addressing the final open recommendation. Additionally, researchers associated with AFUE are seeking to publish an article in a peer-reviewed journal shortly that details the methodology being used for the study. For the long term, AFUE management has proposed releasing annual detailed suppression aircraft use summaries, followed by multi-seasonal outcomes, effectiveness and probability of success findings and then beginning work on real time decision support tools in 2018. Dedicated, intensive field data collection is expected to continue until at least 2019, while operational reporting and performance-tracking systems are developed, so that future aviation management decisions can be supported by improved information on the performance and effectiveness of the assets during operational use.
AFUE has always collected data on fires managed by any authority that has granted access to the dedicated data collection modules. This interagency scope is supported by an AFUE steering committee that is composed of representatives from USDI’s Office of Wildland Fire, Bureau of Land Management, and the National Association of State Foresters as well as the USFS. The steering committee helps ensure that there is true interagency participation in the study’s data collection efforts, as well as vetting of the planned analysis and reporting.
When will annual use summaries be released?
We plan to begin releasing annual use summaries for 2015 and 2016 during 2017. Additional use summaries will come out several months after each data collection season. The remainder of the products will be released after several more seasons, once our sample size and statistical confidence increases.