Hard times in economic data collection call for innovative data sourcing

A new article published by Faculty Associate Katharine Abraham in Review of Income and Wealth examines the challenges that economists have faced by relying on survey and administrative data for official economic statistics. She notes major barriers such as significant declines in survey response rates. For example, "The response rate for the [Current Population Survey], one of the U.S. statistical system’s most important household surveys, fell from an average of 92.4 percent in 2009 to an average of 82.7 percent in 2019, a dismaying decline for a survey that for decades had consistently maintained a response rate in excess of 90 percent."

Abraham goes on to discuss the potential utility of big data drawn from the private sector, including price and product data from retail websites, bank data, payroll data, mobile data, medical insurance claim data, and many other types of novel data historically excluded from official economic statistics. Not only would the inclusion of these data curtail common issues in survey methodology (e.g., respondent burden, nonresponsiveness, representativeness), but it may also allow agencies to produce more disaggregated statistics.

Some U.S. economic agencies have already found success in incorporating private sector data. However, Abraham cites technical issues and difficulties in reaching data-sharing agreements with private sector firms as new considerations for this transition. The economic shock caused by the COVID-19 pandemic required policymakers to answer questions that existing data had not been designed to answer, and the need for data was more urgent than previously anticipated. As the demand for disaggregated data grew, U.S. agencies and academic research teams moved toward collaborating with private companies to provide bank data that could inform policymakers about the economic impacts of the pandemic on the population.

The pandemic provided economists with a push toward innovation and collaboration between the public and private sectors. Abraham concludes her review by urging national statistics offices to prepare to use alternative data sources to address emerging questions and complement survey and administrative data reliant on respondents. 

Abraham, K. G. (2022). Big Data and Official Statistics. Review of Income and Wealth. https://doi.org/10.1111/roiw.12617