Evaluating Unit Value Price Indices in a Dynamic Item Universe Discussion

Evaluating unit-value price indices in a dynamic item universe 16th Meeting of the Ottawa Group, 8-10 May 2019 Rio de Janeiro, Brazil Li-Chun Zhang Statistics Norway/University of Southampton/University of Oslo Ingvild Johansen Statistics Norway Ragnhild Nygaard Statistics Norway

Background Strategy: Increase the use of scanner data in Norwegian CPI/HICP Current official index methods do not fully make use of scanner data potential Until now mostly focus on stable long-lived items Possible to use scanner data of dynamic items in regular production? Requires new methods to be implemented Aims of Eurostat grant 2018-2020: 1. Generic calculation method to be applied across different commodity groups 2. Incorporate expenditure shares at most detailed level No international consensus on scanner data calculation method NSIs choose differently

TEF Total Effect Framework A systematic approach in order to empirically evaluate different choices Measure effects by defining a set of generic diagnostics Not just a question of formula - three necessary choices: 1) Homogenous products 2) Reference universe 3) Index base The choices dependent on the dynamism of item universe Dynamic items: 1) Replacement items 2) Regeneration items 3) Strongly seasonal items

Homogenous products (HPs) How to define the product? Defining the product at item code (GTIN) level may be too detailed Creation of HPs mostly motivated by replacement items For markets with high item churn the effects of missing replacements can be large and systematic Jackets, men Sweaters and blouses, women Item code MGK HP MGK Item code MGK HP MGK 110 110 100 100 90 90 80 80 70 70 60 60 In general, HP-based indices are not more volatile than the item-based, hence no strong indication of classification bias

Homogenous products (HPs) II How item codes are classified key choice in HP formation HP formation depends on available metadata often limited Approach implemented by Statistics Norway for price indices related to sport clothing and equipment Scanner data from one major sport clothing and equipment retailer HPs defined by brand blocking Store concept|Raincoat men|Helly Hansen (brand) In cases with too high heterogeneity only using metadata normal price as additional classification criteria The majority of the HPs entirely based on metadata Fixed HPs within the short-term link

Reference universe Multilateral index formula does not dictate the choice of reference universe Multilateral reference universe responsive to regeneration items Given bilateral fixed index base a new item/HP will not be captured until next time the base month is updated Diagnostic: Testing the sensitivity to the choice of reference universe Compare bilateral index to multilateral index (Fixed base monthly expanding window) with fixed item universe 0 to T Trousers, men Socks, men Item code MGK - multilateral Item code MGK multilateral Item code T rnqvist Item code T rnqvist Item code MGK - bilateral Item code MGK - bilateral 110 110 100 100 90 90 80 80 70 70

Reference universe II Diagnostic: Testing the sensitivity of different window lengths 1.09 1.16 1.08 1.14 1.07 1.12 Socks, men Trousers, kids 1.06 Trousers, men 1.1 Jackets, kids 1.05 Sweaters, men 1.08 Sweaters, kids Jackets, men 1.04 1.06 1.03 1.04 1.02 1 2 3 4 5 6 7 8 9 10111213 1 2 3 4 5 6 7 8 9 10111213 Sensitivity of the GEKS index varies across the commodity groups The GEKS increases with window length

Index base Given that HPs capture replacement items, the choice of index base is most affected by strongly seasonal and regeneration items Fixed base month index to capture strongly seasonal items will normally require imputation of both price and quantity Generally, moving base month t-1, cannot eliminate chain drift completely - May lead to heavy drift in chained bilateral indices, and may create bias in case of strongly seasonal items if first-appearance prices are not appropriately accounted for 12 month base - may be the best option for covering strongly seasonal items, but not necessarily for the inclusion of regeneration items

Index base II Diagnostic: Check the actual dynamics in given markets (expenditure shares) Jackets, men Fresh berries Matched HPs month-to-month Matched items month-to-month Matched HPs fixed base Matched items fixed base Matched HPs year-to-year Matched items year-to-year 1.0 1.0 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0 For commodity groups with seasonal patterns there can be large difference (bicycles, pork, fresh berries etc) Greater sensitivity of fixed base month than 12 month base No ideal choice of index base to all commodity groups

Splicing options Tested different splicing options; Movement splice - Rolling window of 13 months - Month-to-month movement spliced on to existing time series Fixed base moving window (FBMW) - December splice - Combine rolling window with December linking

Splicing options II Movement splice captures the long-term effects of strongly seasonal items Ex. strawberries only available in June and July More drift in movement splice compared to FBMW with December splice ? FBMW December splice does not capture the long-term effects of strongly seasonal items not present in base month Trade off between drift and capturing permanent effects of strongly seasonal items? 110 105 Socks, men Trousers, men 105 100 December splice - T December splice - T 100 95 95 90 90 85 Movement splice - T Movement splice - T 85 80

Splicing options III Marginal total effects on COICOP 01 Movement splice - T December splice - T Movement splice - C December splice -C Movement splice - T Movement splice - C 112 112 112 110 110 110 108 108 108 106 106 106 104 104 104 102 102 102 100 100 100 98 98 98 96 96 96 94 94 94 Movement splice seems to pull the price growth marginally up, while the effects of seasonal items (and regeneration items) pull in opposite direction Empirically difficult to conclude in general small differences between the options Mean splice - geometric mean of the indices by using every possible splicing alternative - suggested solution, but results in complex production routines

Index formula An additional separate choice - set of different index formulas tested and compared GEKS, GUV (Generalized Unit value) indices, official Jevons index (dynamic method) and direct bilateral superlative index All necessary choices matters, but the effects of using HPs seem to outweigh the other choices (more systematic) Bicycles Jackets, men HP GEKS HP T RNQVIST HP QU HP GEKS HP T RNQVIST HP QU HP MGK HP JEVONS HP MGK HP JEVONS 120.0 110.0 105.0 110.0 100.0 95.0 100.0 90.0 90.0 85.0 80.0 80.0 75.0 70.0 70.0 65.0 60.0 60.0

Conclusions Choices are many and answers are not obvious In favor of using HP in order to capture replacement items in dynamic universes Likely to implementing a multilateral price index formula in order to capture regenerations items in a timely manner A fixed length 13-months window seems to be a good choice Splicing a trade off? Own research as well as experiences and advises from NSIs and the statistical community very important for making final conclusion

Thank you!