Understanding and modeling land cover and land use change is evolving into a foundational element of climate, environmental, and sustainability science. Land cover and land use data are core to applications such as carbon accounting, greenhouse gas emissions reporting, biomass and bioenergy assessments, hydrologic function assessments, fire and fuels planning and management, and forest and rangeland health assessments. Remote sensing-based monitoring efforts like the North American Forest Dynamics (NAFD) project, and the newly launched Landscape Change Monitoring System (LCMS), will provide land cover and land use change data on all U.S. lands for the longest possible historical period. Empirical models driving disturbance and causal maps rely on large quantities of high quality data. Many decisions need to be made about the size, shape, quantity, quality, and other details about the training plots themselves, i.e., the response design. Here, the authors explored best practices for collecting training data for these empirical models on 10 pilot scenes in the United States. Alternative designs were evaluated in terms of their costs and benefits for national mapping applications.