Communication of Uncertainty in Official Statistics
Explore the essential link between trust in Federal Statistics and data quality, emphasizing the significance of communication in ensuring credibility and transparency. Learn about evaluating uncertainty in data quality and the importance of building trust through communication and transparency.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Session A-4 Discussion Data Quality: Communication of Uncertainty in Official Statistics Jeffrey M. Gonzalez Economic Research Service 2021 Federal Committee on Statistical Methodology Research and Policy Conference Tuesday, November 2, 2021 Disclaimer: The findings and conclusions in this presentation are those of the authors and should not be construed to represent any official USDA or U.S. Government determination or policy.
Outline I. II. Discussion of presentations A. Evaluating Uncertainty in Multiple Dimensions of Data Quality (Eltinge) Background on themes of FCSM 1. Brief Summary 2. Relevance to themes 3. Additional considerations B. More Fully Capturing Uncertainty Associated with Official Estimates (Young) Tailored Transparency: Public Trust vs. Reproducibility (Miller) III. Concluding remarks C. 2
Trust as the Cornerstone of Federal Statistics and Evidence Building Federal data inform and improve our decisions, policies, and lives. Question: What does it take to build trust? 1. Communication 2. Helpfulness 3. Honesty 4. Transparency Question: Why do we need trust in official statistics? 1. Credibility for the role that the Federal Statistical System plays in data collection and outcomes research 2. Fosters a culture and society that values data and evidence-based policymaking 3
Communication of Uncertainty in Official Statistics Inherent connection between trust in Federal Statistics and data quality. Adapted from Groves et al 2009 Survey Methodology, 2nd Edition Citation: Federal Committee on Statistical Methodology. 2020. A Framework for Data Quality. 4
Evaluating Uncertainty in Multiple Dimensions of Data Quality (Eltinge) Brief summary 1. Provides a comprehensive introduction to the multiple dimensions of data quality, with distinctions between quantifiable and qualitative components 2. Illustrates concepts and poses a series of questions about quality components with an application to the integration of multiple data sources Relevance to themes 1. Framework and questions give practitioners a model, or process, to systematically consider potential threats to the utility, objectivity, and integrity all which impact trust 2. Openness about each data source, integration methods, and trade-offs being made Additional considerations 1. Extensions of backbone and bridge beyond supplemental surveys as bridges to crowd- sourced or other data sources compiled by a third party 2. All dimensions of data quality matter, but incorporating errors for all inputs and production steps may be unfeasible 5
More Fully Capturing Uncertainty Associated with Official Estimates (Young) Brief summary 1. Focus on the accuracy dimension of total survey error and quantifies errors due to sampling, nonresponse, undercoverage, misclassification, and calibration 2. Application to the 2017 Census of Agriculture to improve upon the 2012 methods by incorporating the impact of calibration and rounding in all measures of uncertainty Relevance to themes 1. Communication and transparency about standard error calculations 2. Inaccuracies can impact confidence in the data product and can erode users' trust Additional considerations 1. Models are imperfect characterizations and can be subject to some level of subjectivity, potential conflicts with objectivity as a domain of data quality 2. Accounting for and understanding the relative importance of other sources of uncertainty (e.g., linkage, editing, and imputation) is critical for reducing the overall error 6
Tailored Transparency: Public Trust vs. Reproducibility (Miller) Brief summary 1. Discusses issues related to disclosure of statistical and survey information from legal/regulatory and Federal Statistical System perspectives 2. Outlines a procedure for tailored disclosure that considers different purposes and audiences for the documentation Relevance to themes 1. Recognition that an aspect of effective communication is understanding the audience 2. Transparency relates to accessibility and utility Additional considerations 1. Focus on disclosure/dissemination of data products and statistical information to data consumers/users, applications of tailored transparency in survey recruitment 2. Transparency may be at odds with making undesirable features more salient 7
Concluding remarks 1. All data products have strengths and weaknesses across each dimension of data quality 2. Honest and transparent documentation enables appropriate use of the data products and reinforces credibility and utility, impacting trust 3. Implementation of best practices helps build and maintain trust and involves understanding the threats to each dimension and their trade-offs 4. Best practices and examples of applications of these concepts exist FCSM-20-04 A Framework for Data Quality (ed.gov) Data Quality and Data Comparisons in the Consumer Expenditure Surveys (bls.gov) Session F-2: Examination of the Data Quality Properties of USDA and Proprietary Databases (Milburn) 8
Thank you! Jeffrey M. Gonzalez Research Mathematical Statistician Economic Research Service Jeffrey.Gonzalez@usda.gov 9
