Well-Being Research Insights and Best Practices

Fantastic Well-Being Results and Where to Find them Caspar Kaiser and Anthony Lepinteur International Association for Research in Income and Wealth June 26, 2024 Discussed by Kelsey J. O Connor STATEC Research, Luxembourg IZA, GLO, UJ

New to subjective well-being research? How does one publish well (in top economics journals)? What are the best approaches? Which data to use? How should they be analyzed? Key variables and their parameterizations? Do the results make sense? Key findings (direction and magnitude) R-squareds What resource(s) should we use? 2

3

Contribution Well-Base: reproduction of the universe of the most recent evaluative SWB- regressions from papers published in the top-30 Econ journals Includes replication packages and data Closest alternative - World Database of Happiness (Veenhoven) Other databases of happiness coefficients, e.g., in Canada (Barrington-Leigh) and the UK (MacLennan 2021), have narrower scopes (focusing more on geography and quality) The World Database of Happiness reports tens of thousands of evaluative subjective well-being coefficients from more than 15,000 publications, using a standard format and terminology But they do not have access to the replication packages and data, and the authors analyze the papers and findings in novel ways 4

Uses of Well-Base 1. Best practices to publish SWB research in top journals 2. Collective insights 3. Advanced use: replication under different conditions 5

Creation of Well-Base

Procedure Google Scholar Search (January 2024): From: 2010 to 2023 Keyword: Life satisfaction , Cantril Ladder , Subjective Wellbeing , Subjective Well-Being Source: Top 30 Econ Journals (according to RePEc) Results: 345 published articles 7

Sample of studies 345 papers 246 did not analyze evaluative SWB data 29 transformed life satisfaction (e.g., part of index) 10 life satisfaction was not the dependent variable 3 analyses not at the individual level 10 studies data not available (e.g., administrative or proprietary) Leaving 47 replicated studies, with Over 1,000 regressions and 50,000 point estimates of which only about 9% were printed 8

Best Practices 9

Total number of articles per journal in Well-Base Total number of articles 8 8 7 6 6 3 2 2 2 2 1 47 Total number of w being articles Journal 1 2 3 4 5 6 7 8 9 10 Review of Economic Studies 11 Journal of Economic Perspective Total The Economic Journal Review of Economics and Statistics Journal of Development Economics American Economic Review Journal of Public Economics Journal of Labor Economics American Economic Journal: Applied Economics Journal of European Economic Association Quarterly Journal of Economics 6 5 1 3 5 1 0 1 1 2 0 25 An article is labelled as a well-being article when at least half of its regressions uses subjective well-being as dependent variable. 10

Descriptive Statistics of Well-Base at the regression level Panel A: At the regression level (N=1072) Dependent variable: Number of categories: 4-points scale 10-points scale 11-points scale Type of question: Life satisfaction Cantril ladder Happiness question Estimation sample: Average Age Share of women Number of observations (logged) Econometric model: Number of controls Number of controls (logged) Pooled OLS OLS with individual FE 2SLS Ordered logit/probit Mean 0.24 0.31 0.39 0.59 0.06 0.34 38.63 10.89 10.95 69.31 0.51 0.20 10.23 2.46 9.43 2.69 2.76 0.92 0.73 0.12 0.14 0.01 SD Min 0 0 0 0 0 0 Max 1 1 1 1 1 1 0 1 4.08 14.72 4.08 14.72 0 0 0 0 0 5.25 1 1 1 1 11

Common (missing) subjects Top 30 journals prefer micro studies Only 3 macro studies, which were dropped from Well-Base Age, gender, income, and unemployment are the most often tested relations Other subjects appear to be of less interest in these journals, e.g., the environment Children and elderly often missing 12

Collective Insights 13

Magnitudes: CDF of Well-Base beta coefficients ~ 75% of printed estimates have point estimates that range from 0-20% of a standard deviation 14

Summarizing findings Age, gender, income, and unemployment are most often tested relations Focusing on age, how should it be parameterized? 367 regressions use quadratic parameterizations, of which 74% are significant 275 use linear ones, of which < 50% are significant This is suggestive that the quadratic relation is superior Using the replication package and data, the authors can further interrogate the result The quadratic relation was often insignificant when including a collinear variable Omitting the collinear variable, ~93% of the quad. relations become significant The authors further plot the estimated ages of mid-life low (typically between 45-55) and shape curvature, and provide results for gender, income, and unemployment 15

Context-sensitivity of well-being results Estimation sample: Average age Share of women Number of observations (logged) Econometric model: No controls (in log) Individual FE Dependent variable: Number of categories on SWB scale Equality for Continent Coefficients p-values Mean of dependent variable SD of dependent variable Observations R2 -0.019*** (0.006) 0.004 (0.005) 0.018*** (0.002) -0.086*** (0.009) -0.000 (0.002) 0.662 0.111 0.044 130 0.861 Value of point estimate Income Unemployment (5) 0.012*** (0.003) Context matters (6) 0.011 (0.007) Based on regressions of point estimates on sample and regression characteristics 0.013 (0.009) Income matters more in samples with greater shares of older individuals or men 0.022* (0.012) 0.009 (0.006) Income point estimates increase with the number of controls, but decrease with fixed effects 0.202*** (0.034) -0.016*** (0.005) 0.000 -0.433 0.100 121 0.867 This sensitivity is very important for calculating marginal rates of substitution to infer monetary valuations of non- market goods 16

Challenges with summarizing results Not all estimates are created equal The UK database of coefficients (MacLennan) aims to include only causal estimates Should separate fixed effects from cross-sectional estimates, they are probably attenuated but more reliable and should be interpreted differently ~90% of the estimates are from non-printed control variables These point estimates are not intended for policy or behavioral insights Controls are more likely to suffer from endogeneity, collinearity, and mediation Standard practice to sequentially introduce more to less exogenous controls Collinear, recall the authors achieved expected age relations after excluding a collinear control Carefully choose controls to avoid mediators / bad controls 17

Age as a control or variable of interest Age serves as an example. There are two relations of interest that are estimated by varying the controls If age is a control (not the variable of interest), then it is usually one among many. At best, the resulting age relation can be interpreted as the pure-effect of aging Instead, if one wants to estimate the life-cycle pattern between age and life satisfaction, all age-varying controls should be excluded as bad controls And some results could be based on limited external validity In contrast to authors (cf. pg. 13), Blanchflower (e.g., 2021) argues that the type of SWB question and sample does not matter, and that the U-shape is indeed everywhere 18

Papers discussion The text should be rewritten to acknowledge throughout that their sample is not representative of well-being research The views represent what is accepted by top editors and referees not what well- being scholars think more broadly Very few well-being studies make it into the top 30 economics journals Geographic range is limited, but so is the subject matter Nearly half of the World Wellbeing Panel s Wellbeing Reading List is published outside the top 30 economics journals When summarizing results, drop non-representative samples, and maybe reduce the data to a broadly consistent population (e.g., age, gender) When expanding, could choose journals to address missings, e.g., China Economic Review (outside top 100, but still relevant?) 19

Advanced Use 20

Replication under different assumptions Thursday, August 29 Plenary Session 5-1: Measuring Well-being for Policy Reversing the Reversal? An Appraisal of the Robustness of Economic Research on Self-Reported Wellbeing Discussant: Richard Heys (Office for National Statistics, UK) Four more projects planned check with the authors 21

Thank You Kelsey J. O Connor kelsey.oconnor@statec.etat.lu National Institute of Statistics and Economic Studies of the Grand Duchy of Luxembourg L-1468 Luxembourg research@statec.etat.lu statec.lu 13, rue Erasme

Number of countries included in the Well-Base papers 23

Distribution of Adjusted R2 between cross-sectional and longitudinal data 24

Distribution of t-statistics printed estimates Mass after the 5% significance level 25

Distribution of the mid-life lows 26

Funnel plot Unemployment The coefficients for unemployment respectively use employed respondents as reference category Red line is the average effect size (-0.43). Grey lines are the 95% confidence intervals 27

Ongoing and planned research using Well-Base Project 2: Reversing the Reversal? Project 3 (with E. Oparina): about median regressions vs OLS Project 4 (With Benjamin et al.): about scale-use heterogeneity and CQ Project 5: how robust SWB results are when we exclude the top and bottom response categories? 28