Deciding Aggregated Outcomes in Wine Economics Conference
In the realm of wine economics, decision-making plays a crucial role. Explore how different voting methods impact outcomes, consensus rankings in wine studies, and the debate on using scores vs. rankings in assessments. Three distinct methods are compared to shed light on effective decision-making processes.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
How to Decide How to Decide The American Association of Wine Economists 12thAnnual Conference, 10-14 June 2018, Ithaca, New York Neal D. Hulkower, Ph.D. McMinnville, OR nhulkower@yahoo.com
Outline Different Outcomes Recent Wine-Related Studies Arrow s Impossibility Theorem Borda Count Cicchetti Example How to Decide How to Decide References 2
Different Outcomes Consider the profile (MA 111 Review for Exam 1 website): Number of votes 14 4 10 1 8 Ranking A B C D B D C A C B D A C D B A D C B A where A B means A is preferred to B Plurality voting (only first place candidate is counted) yields A C D B Pairwise or Condorcet method (ranking determined by adding the number of times each candidate bests the others in a pairwise vote) gives C B D A Borda Count (Borda, 1781) (3 points for top ranked, 2 points for second, 0 for 4th) returns B C D A How to decide how to decide what the aggregated outcome is? 3
Recent Wine-Related Studies Consensus ranking of vintages White Burgundy Vintage 1996 1995 2002 2005 1989 1985 1990 1986 1997 1999 1992 2004 1982 2000 2001 1998 1983 2003 1994 1988 1993 1991 1987 1984 Borda Rank 1 2 2 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Condorcet Rank 2 1 2 4 4 6 7 8 10 10 12 9 13 13 15 16 19 17 17 20 22 23 21 24 Borda Rank from Hulkower (2012b, Table 2), Condorcet Rank from Borges et al. (2012a, Table 10). See also Borges et al. (2012b) and Hulkower (2013a, 2014) for discussion. 4
Recent Wine-Related Studies Scores vs Rankings Cicchetti (2014) makes the case for using scores over rankings for determining the rater s ability to detect replicated samples in a tasting and inter-rater reliability Rankings do not inherently reflect quality assessments as scores do Scores are easily converted to rankings but not the other way around Hulkower (2013b) is concerned that using scores to determine rankings gives disproportionate influence to easy graders Selection of method depends on what question is being answered 5
Recent Wine-Related Studies Three Methods Compared Cao and Stokes (2017) compared original-score average, rank average, and Shapley ranking (Ginsburgh and Zang, 2012) in their ability to accurately determine the quality of a group of N wines Results mixed Shapley ranking allows judges to select any number of alternatives, 0 < m N for consideration For each judge, each selected alternative is assigned 1/m points; the sum of the points assigned by each judge determines the overall ranking If m < N, information is discarded Since judges can select different numbers of alternatives, their influence on the final outcome can be disproportionate 6
Recent Wine-Related Studies Ad hoc vs general properties Cicchetti (2014) and Cao and Stokes (2017) relied on specific examples rather than general properties to arrive at their conclusions Borges et al. (2012b) defended their use of Condorcet over Borda by comparing the two on the basis of a list of standard conditions for social choice Is there an objective way of selecting a method to aggregate voters ranked preferences that is context-free? 7
Arrows Impossibility Theorem (1 of 4) It is impossible to have a voting scheme that satisfies these five properties (Arrow, 1963): Complete transitivity outcomes: transitivity means if A B and B C then A C; complete means that each voter is rational, i.e., he/she will rank each pair to form a complete transitive ranking Unrestricted domain: all possible orderings are permitted Pareto condition: if all voters rank A B then the social outcome ranks A B Independence of Irrelevant Alternatives (IIA): if p1 and p2 are any two profiles for which each voter has the same relative ranking of some specified pair, then the societal ranking for this pair is the same for both profiles (Saari, 2008, p. 22) The social outcome is not determined by a single individual s ranking of preferences without regard to the rankings of others 8
Arrows Impossibility Theorem (2 of 4) In other words, the only voting method to satisfy the first four criteria is a dictator One consequence: for decades those needing to aggregate preferences chose a method based on ad hoc criteria Beware the Axiom, and shun The faithless Postulate. From Plane Geometry by Emma Rounds (http://www76.pair.com/keithlim/jabberwocky/parodies/plaingeometry.html) But Arrow s Theorem has a problem: IIA is inconsistent with the assumption of complete transitive outcomes; if a rule satisfies IIA, a nontransitive or irrational outcome may result (Saari, 2000a, Section 8) 9
Arrows Impossibility Theorem (3 of 4) Saari (2018) has proven that: A decision rule that provides rankings for each pair from a set of N 3 alternatives can be represented as a collection of separate and independent paired comparison rules if and only if the rule satisfies IIA So insistence on IIA limits the choice of decision rules to paired comparisons which by definition do not consider complete information The problem arises because IIA only looks at pairwise comparisons; thus when considering the ranking of A and B, it treats A B C D and A C D B the same, ignoring the number of alternatives that separate the two 10
Arrows Impossibility Theorem (4 of 4) Replacing IIA with the Intensity form of Independence of Irrelevant Alternatives (IIIA): society s relative ranking of any two alternatives is determined only by each voter s relative ranking of the pair and the intensity of that ranking [measured by the number of alternatives separating the two]. That is, for any pair of alternatives , if each voter s relative ranking and intensity ranking is the same for two profiles p1 and p2, then society s ranking of this pair is the same for both profiles (Saari, 2001, pp. 189-190) preserves the rational outcome and avoids Arrow s dictator 11
Borda Count (1 of 2) Saari (2000a, 2000b, 2001, 2008, 2018) proved that the Borda Count is the only decision rule that satisfies complete transitivity outcomes, unrestricted domain, the Pareto condition and IIIA Other unique properties of Borda [U]nique natural extension of a pairwise vote from N = 2 to N 3 alternatives The sum of an alternative s tallies over all pairs equals the Borda Count assignment for that alternative; i.e., the Borda Count is the aggregated form of pairwise voting (Saari, 2018) For example, if A B C, then in pairwise comparisons, A beats B, A beats C, B beats C and C beats neither. So A gets 2 points, B gets 1 point and C gets 0 points Always ranks Condorcet winner over Condorcet loser 12
Borda Count (2 of 2) Other unique properties of Borda (cont d) [A]ll possible inconsistencies in Borda rankings over subsets of candidates are strictly due to Condorcet terms (Saari, 2008, p.157) Condorcet n-tuples are rankings that end up in a complete tie, for example the Condorcet 4-tuple is A B C D, B C D A, C D A B, and D A B C The sums of the Borda Scores represent strength of preference [without distortion] among the alternatives in the societal outcome (Hulkower, 2013b) 13
Cicchetti Example (1 of 3) Taster Scores Three 90 92 96 Wine A B C One 60 62 66 Two 70 73 76 Four 94 96 98 Five 96 98 100 Cicchetti (2014, p. 8) Tasters Three, Four and Five like all three wines while Tasters One and Two are not impressed with any Consensus ranking based on scores is C B A but this does not reflect the disparity of opinion on the quality when one is interested in the perceived quality of a given wine - the usual desideratum then the only choice is to use wine rating scores. The application of ranking methods, by this reasoning, would seriously compromise the accuracy of results (Cicchetti, 2014, p. 8) 14
Cicchetti Example (2 of 3) Taster Scores Three 90 92 96 278 Wine A B C Totals One 60 62 66 188 Two 70 73 76 219 Four 94 96 98 288 Five 96 98 100 294 Proportion of total points (1267) assigned Relative weight 0.15 0.17 0.22 0.23 0.23 1.00 1.16 1.48 1.53 1.56 High scorers have a disproportional influence on the outcome Borda gives each taster an equal influence on the consensus ranking and can even allow dissenting opinions 15
Cicchetti Example (3 of 3) Taster Rankings Based on Scores One Two Three 4 4 3 3 2 2 1 1 Wine A B C None Four 3 2 1 4 Five 3 2 1 4 3 2 1 4 If the goal is simply to determine a consensus ranking of the wines and still accommodate dissent based on perceived quality, add None of these wines as an option and proceed with Borda with None as described in Hulkower and Neatrour (2018) Consensus ranking is C B None If Tasters One and Two rank the three wines tied and below None, the consensus ranking is the same A 16
How to Decide How to Decide (1 of 2) Know what question you are answering Objective Ranking of alternatives (comparing wines across a flight) Assessing judge s ability to identify replicates Analysis of judges performance Approach Have judges rank order wines directly or, if numerical ratings are used, normalize by converting them to rankings then aggregate with Borda (with None) Numerical ratings or rankings Numerical ratings for comparing individuals to each other or the societal outcome. Borda Scores of the societal outcome can also be used for the latter From Hulkower (2013b) 17
How to Decide How to Decide (2 of 2) Avoid ad hoc or example specific criteria Insist on a method that uses complete information contained in the profile, so avoid Plurality Pairwise method or Condorcet (any that satisfy IIA) Shapley ranking Select a method that has been rigorously proven to satisfy uniquely the fewest rational properties, uses complete information, avoids distortion, and is context-free In other words, use the Borda Count 18
References (1 of 3) Arrow, K. J. (1963). Social choice and individual values, 2nd edn. New Haven: Yale University Press. Borda, J. C. (1781). M moire sur les lections au scrutiny, Histoire de l Acad mie Royale des Sciences, Paris. In McLean, I., and Urken, A. B. (eds), Classics of Social Choice, 1995, 83-89. Ann Arbor: The University of Michigan Press. Borges, J., Real, A. C., Cabral, J. S. and Jones, G. V. (2012a). A New Method to Obtain a Consensus Ranking of a Region s Vintages Quality. Journal of Wine Economics, 7, 88 107. Borges, J., Real, A. C., Cabral, J. S., and Jones, G. V. (2012b). Condorcet versus Borda, a response to: Comment on A New Method to Obtain a Consensus Ranking of a Region s Vintages Quality. Journal of Wine Economics, 7, 245 248. Cao, J. and Stokes, L. (2017). Comparison of Different Ranking Methods in Wine Tasting. Journal of Wine Economics, 12, 203 210. Cicchetti, D. (2014). Blind Tasting of South African Wines: A Tale of Two Methodologies, AAWE Working Paper No. 164. Available at http://www.wine-economics.org/aawe- working-paper-no-164-economics/ Ginsburgh, V. and Zang, I. (2012). Shapley Ranking of Wines. Journal of Wine Economics, 7, 169-180. 19
References (2 of 3) Hulkower, N. D. (2012a). A Mathematician Meddles with Medals, AAWE Working Paper No. 97. Available at http://www.wine-economics.org/dt_catalog/working-paper-no-97/ Hulkower, N. D. (2012b). Comment on A New Method to Obtain a Consensus Ranking of a Region s Vintages Quality. Journal of Wine Economic,. 7, 241 244. Hulkower, N. D. (2013a). Clash of Sensibilities. Oregon Wine Press, Issue 327, June 2013, 23. Available at http://www.oregonwinepress.com/article?articleTitle=clash-of-sensibilities-- 1369939201--1525-- Hulkower, N. D. (2013b). Three Vignettes about Wine Tastings and Competitions. Presentation at The American Association of Wine Economists 7th Annual Conference, 26-29 June 2013, Stellenbosch, South Africa. Hulkower, N. D. (2014). Information Lost: The Unbearable Lightness of Vintage Charts. Presentation at The American Association of Wine Economists 8th Annual Conference, 22-25 June 2014, Walla Walla, Washington. Hulkower, N. D. and Neatrour, J. (2018). The Power of None. Submitted. 20
References (3 of 3) MA 111 Review for Exam 1 Website https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=3&ved=0ahUKEwiM u-Df_vjXAhVK- GMKHdDeA9cQFgg2MAI&url=http%3A%2F%2Fwww.ms.uky.edu%2F~mdailey%2Fma111 %2Freview%2FExam_1_Review_Solutions.pdf&usg=AOvVaw3pOteK90yIt5hbRaBV9RBI Accessed 7 December 2017 Saari, D. G. (2000a). Mathematical structures of voting paradoxes: I. Pairwise vote. Economic Theory, 15, 1-53. Saari, D. G. (2000b). Mathematical structures of voting paradoxes: II. Positional voting. Economic Theory, 15, 55-101. Saari, D. G. (2001). Decisions and Elections: Explaining the Unexplained. Cambridge, UK: Cambridge University Press. Saari, D. G. (2008). Disposing Dictators, Demystifying Voting Paradoxes: Social Choice Analysis. New York: Cambridge University Press. Saari, D. G. (2018). Arrow, and Unexpected Consequences of his Theorem. Public Choice. doi.org/10.1007/s11127-018-0531-7 21
