Research Article
A new rank correlation coefficient with application to the consensus ranking problem
Abstract
The consensus ranking problem has received much attention in the statistical literature. Given m rankings of n objects the objective is to determine a consensus ranking. The input rankings may contain ties, be incomplete, and may be weighted. Two solution concepts are discussed, the first maximizing the average weighted rank correlation of the solution ranking with the input rankings and the second minimizing the average weighted Kemeny–Snell distance. A new rank correlation coefficient called τx is presented which is shown to be the unique rank correlation coefficient which is equivalent to the Kemeny‐Snell distance metric. The new rank correlation coefficient is closely related to Kendall's tau but differs from it in the way ties are handled. It will be demonstrated that Kendall's τb is flawed as a measure of agreement between weak orderings and should no longer be used as a rank correlation coefficient. The use of τx in the consensus ranking problem provides a more mathematically tractable solution than the Kemeny–Snell distance metric because all the ranking information can be summarized in a single matrix. The methods described in this paper allow analysts to accommodate the fully general consensus ranking problem with weights, ties, and partial inputs. Copyright © 2002 John Wiley & Sons, Ltd.
Citing Literature
Number of times cited according to CrossRef: 71
- Mariangela Sciandra, Antonio D'Ambrosio, Antonella Plaia, Projection Clustering Unfolding: A New Algorithm for Clustering Individuals or Items in a Preference Matrix, Data Analysis and Applications 3, 10.1002/9781119721871, (215-230), (2020).
- Moniba Keymanesh, Saket Gurukar, Bethany Boettner, Christopher Browning, Catherine Calder, Srinivasan Parthasarathy, Twitter Watch: Leveraging Social Media to Monitor and Predict Collective-Efficacy of Neighborhoods, Complex Networks XI, 10.1007/978-3-030-40943-2_17, (197-211), (2020).
- Yeawon Yoo, Adolfo R. Escobedo, J. Kyle Skolfield, A New Correlation Coefficient for Comparing and Aggregating Non-strict and Incomplete Rankings, European Journal of Operational Research, 10.1016/j.ejor.2020.02.027, (2020).
- Pietro Amenta, Antonio Lucadamo, Gabriella Marcarelli, On the choice of weights for aggregating judgments in non-negotiable AHP group decision making, European Journal of Operational Research, 10.1016/j.ejor.2020.05.048, (2020).
- Manchun Fang, David Mason, undefined, 2020 9th International Conference on Industrial Technology and Management (ICITM), 10.1109/ICITM48982.2020.9080359, (25-31), (2020).
- Nana Liu, Zeshui Xu, Hangyao Wu, Peijia Ren, Fanlin Meng, undefined, 2020 International Joint Conference on Neural Networks (IJCNN), 10.1109/IJCNN48605.2020.9207510, (1-8), (2020).
- Raphaël Huser, Clément Dombry, Mathieu Ribatet, Marc G. Genton, Full likelihood inference for max‐stable data, Stat, 10.1002/sta4.218, 8, 1, (2019).
- Lin Su, Wenbin Lu, Rui Song, Modelling and estimation for optimal treatment decision with interference, Stat, 10.1002/sta4.219, 8, 1, (2019).
- Craig Wang, Reinhard Furrer, Efficient inference of generalized spatial fusion models with flexible specification, Stat, 10.1002/sta4.216, 8, 1, (2019).
- Amir Hossein Shabani, Hadi Jabbari, Vahid Fakoor, Weak convergence of TJW product‐limit estimator under association, Stat, 10.1002/sta4.217, 8, 1, (2019).
- Nger Ling Chong, Michael B.C. Khoo, Salah Haridy, Mohammad Shamsuzzaman, A multiattribute cumulative sum‐np chart, Stat, 10.1002/sta4.239, 8, 1, (2019).
- Lan Liu, Michael G. Hudgens, Bradley Saul, John D. Clemens, Mohammad Ali, Michael E. Emch, Doubly robust estimation in observational studies with partial interference, Stat, 10.1002/sta4.214, 8, 1, (2019).
- Antonella Plaia, Simona Buscemi, Mariangela Sciandra, A new position weight correlation coefficient for consensus ranking process without ties, Stat, 10.1002/sta4.236, 8, 1, (2019).
- Ivano Azzini, Giuseppe Munda, A New Approach for Identifying the Kemeny Median Ranking, European Journal of Operational Research, 10.1016/j.ejor.2019.08.033, (2019).
- Klara Pigmans, Virginia Dignum, Neelke Doorn, Group proximity and mutual understanding: measuring onsite impact of a citizens’ summit, Journal of Public Policy, 10.1017/S0143814X19000230, (1-23), (2019).
- Ilja Rausch, Yara Khaluf, Pieter Simoens, Scale-Free Features in Collective Robot Foraging, Applied Sciences, 10.3390/app9132667, 9, 13, (2667), (2019).
- Roberto Ghiselli Ricci, An axiomatic characterization of a class of rank mobility measures, Social Choice and Welfare, 10.1007/s00355-018-01171-5, (2019).
- Antonio D’Ambrosio, Carmela Iorio, Michele Staiano, Roberta Siciliano, Median constrained bucket order rank aggregation, Computational Statistics, 10.1007/s00180-018-0858-z, (2019).
- Boris Mirkin, Trevor I. Fenner, Distance and Consensus for Preference Relations Corresponding to Ordered Partitions, Journal of Classification, 10.1007/s00357-018-9290-x, (2019).
- Luca Frigau, Claudio Conversano, Francesco Mola, Consistent validation of gray-level thresholding image segmentation algorithms based on machine learning classifiers, Statistical Papers, 10.1007/s00362-019-01138-3, (2019).
- Mariangela Sciandra, Antonio D'Ambrosio, Antonella Plaia, Projection Clustering Unfolding: A New Algorithm for Clustering Individuals or Items In A Preference Matrix, SSRN Electronic Journal, 10.2139/ssrn.3209215, (2018).
- Prakash S Badal, Ashish Das, Efficient algorithms using subiterative convergence for Kemeny ranking problem, Computers & Operations Research, 10.1016/j.cor.2018.06.007, 98, (198-210), (2018).
- Juan A. Aledo, José A. Gámez, Alejandro Rosete, Approaching rank aggregation problems by using evolution strategies: The case of the optimal bucket order problem, European Journal of Operational Research, 10.1016/j.ejor.2018.04.031, 270, 3, (982-998), (2018).
- Robin Milosz, Sylvie Hamel, Exploring the Median of Permutations Problem, Journal of Discrete Algorithms, 10.1016/j.jda.2018.11.007, (2018).
- Dong Xu, Nazrul I. Shaikh, A Heuristic Approach for Ranking Items Based on Inputs from Multiple Experts, International Journal of Information Systems and Social Change, 10.4018/IJISSC.2018070101, 9, 3, (1-22), (2018).
- Andrej Bregar, Decision support on the basis of utility models with discordance-related preferential information: investigation of risk aversion properties, Journal of Decision Systems, 10.1080/12460125.2018.1468170, 27, sup1, (236-247), (2018).
- Abdullah Yasin Nur, Mehmet Engin Tozal, Identifying critical autonomous systems in the Internet, The Journal of Supercomputing, 10.1007/s11227-018-2336-3, 74, 10, (4965-4985), (2018).
- Madhuri S. Mulekar, C. Scott Brown, Distance and Similarity Measures, Encyclopedia of Social Network Analysis and Mining, 10.1007/978-1-4939-7131-2, (647-662), (2018).
- Thomas Rusch, Kurt Hornik, Patrick Mair, Assessing and Quantifying Clusteredness: The OPTICS Cordillera, Journal of Computational and Graphical Statistics, 10.1080/10618600.2017.1349664, 27, 1, (220-233), (2018).
- Yu-Shan Shih, Kuang-Hsun Liu, Regression trees for detecting preference patterns from rank data, Advances in Data Analysis and Classification, 10.1007/s11634-018-0332-3, (2018).
- Antonio D’Ambrosio, Willem J. Heiser, A distribution-free soft-clustering method for preference rankings, Behaviormetrika, 10.1007/s41237-018-0069-5, (2018).
- Antonella Plaia, Mariangela Sciandra, Weighted distance-based trees for ranking data, Advances in Data Analysis and Classification, 10.1007/s11634-017-0306-x, 13, 2, (427-444), (2017).
- Juan A. Aledo, José A. Gámez, Alejandro Rosete, Partial evaluation in Rank Aggregation Problems, Computers & Operations Research, 10.1016/j.cor.2016.09.013, 78, (299-304), (2017).
- Antonio D’Ambrosio, Giulio Mazzeo, Carmela Iorio, Roberta Siciliano, A differential evolution algorithm for finding the median ranking under the Kemeny axiomatic approach, Computers & Operations Research, 10.1016/j.cor.2017.01.017, 82, (126-138), (2017).
- Mehmet Engin Tozal, undefined, 2017 Annual IEEE International Systems Conference (SysCon), 10.1109/SYSCON.2017.7934814, (1-8), (2017).
- Madhuri S. Mulekar, C. Scott Brown, Distance and Similarity Measures, Encyclopedia of Social Network Analysis and Mining, 10.1007/978-1-4614-7163-9, (1-16), (2017).
- Roland Pongou, Bertrand Tchantcho, Narcisse Tedjeugang, Properties of ladder tournaments, European Journal of Operational Research, 10.1016/j.ejor.2017.03.082, 263, 1, (203-213), (2017).
- Andrea Bonanomi, Marta Nai Ruscone, Silvia Angela Osmetti, Defining subjects distance in hierarchical cluster analysis by copula approach, Quality & Quantity, 10.1007/s11135-016-0444-9, 51, 2, (859-872), (2016).
- Mariangela Sciandra, Antonella Plaia, Vincenza Capursi, Classification trees for multivariate ordinal response: an application to Student Evaluation Teaching, Quality & Quantity, 10.1007/s11135-016-0430-2, 51, 2, (641-655), (2016).
- Nourhène Alaya, Sadok Ben Yahia, Myriam Lamolle, Ranking with Ties of OWL Ontology Reasoners Based on Learned Performances, Knowledge Discovery, Knowledge Engineering and Knowledge Management, 10.1007/978-3-319-52758-1_14, (234-259), (2016).
- Antonio D’Ambrosio, Willem J. Heiser, A Recursive Partitioning Method for the Prediction of Preference Rankings Based Upon Kemeny Distances, Psychometrika, 10.1007/s11336-016-9505-1, 81, 3, (774-794), (2016).
- S. Amodio, A. D’Ambrosio, R. Siciliano, Accurate algorithms for identifying the median ranking when dealing with weak and partial rankings under the Kemeny axiomatic approach, European Journal of Operational Research, 10.1016/j.ejor.2015.08.048, 249, 2, (667-676), (2016).
- T. González-Arteaga, J.C.R. Alcantud, R. de Andrés Calle, A new consensus ranking approach for correlated ordinal information based on Mahalanobis distance, Information Sciences, 10.1016/j.ins.2016.08.071, 372, (546-564), (2016).
- Nourhene Alaya, Sadok Ben Yahia, Myriam Lamolle, undefined, 2016 IEEE 28th International Conference on Tools with Artificial Intelligence (ICTAI), 10.1109/ICTAI.2016.0165, (1076-1083), (2016).
- Teresa González-Arteaga, Rocío de Andrés Calle, Francisco Chiclana, A new measure of consensus with reciprocal preference relations: The correlation consensus degree, Knowledge-Based Systems, 10.1016/j.knosys.2016.06.002, 107, (104-116), (2016).
- Laura Vana, Ronald Hochreiter, Kurt Hornik, Computing a journal meta-ranking using paired comparisons and adaptive lasso estimators, Scientometrics, 10.1007/s11192-015-1772-6, 106, 1, (229-251), (2015).
- Valério Antonio Pamplona Salomon, Luís Alberto Duncan Rangel, Comparing Rankings from Using TODIM and a Fuzzy Expert System, Procedia Computer Science, 10.1016/j.procs.2015.07.019, 55, (126-138), (2015).
- Francisco Pedroche, Regino Criado, Esther García, Miguel Romance, Victoria E. Sánchez, Comparing series of rankings with ties by using complex networks: An analysis of the Spanish stock market (IBEX-35 index), Networks and Heterogeneous Media, 10.3934/nhm.2015.10.101, 10, 1, (101-125), (2015).
- Camilo Franco, Jens Leth Hougaard, Kurt Nielsen, Handling Risk Attitudes for Preference Learning and Intelligent Decision Support, Modeling Decisions for Artificial Intelligence, 10.1007/978-3-319-23240-9_7, (78-89), (2015).
- Baoli Wang, Jiye Liang, Yuhua Qian, Determining decision makers’ weights in group ranking: a granular computing method, International Journal of Machine Learning and Cybernetics, 10.1007/s13042-014-0278-5, 6, 3, (511-521), (2014).
- Roland Pongou, Bertrand Tchantcho, Narcisse Tedjeugang, Power theories for multi-choice organizations and political rules: Rank-order equivalence, Operations Research Perspectives, 10.1016/j.orp.2014.06.002, 1, 1, (42-49), (2014).
- Andrej Bregar, Towards a Framework for the Measurement and Reduction of User-Perceivable Complexity of Group Decision-Making Methods, International Journal of Decision Support System Technology, 10.4018/ijdsst.2014040102, 6, 2, (21-45), (2014).
- Madhuri S. Mulekar, C. Scott Brown, Distance and Similarity Measures, Encyclopedia of Social Network Analysis and Mining, 10.1007/978-1-4614-6170-8, (385-400), (2014).
- Willem J. Heiser, Antonio D’Ambrosio, Clustering and Prediction of Rankings Within a Kemeny Distance Framework, Algorithms from and for Nature and Life, 10.1007/978-3-319-00035-0_2, (19-31), (2013).
- Regino Criado, Esther García, Francisco Pedroche, Miguel Romance, A new method for comparing rankings through complex networks: Model and analysis of competitiveness of major European soccer leagues, Chaos: An Interdisciplinary Journal of Nonlinear Science, 10.1063/1.4826446, 23, 4, (043114), (2013).
- Yeong Kyu Kim, Eun-Sung Chung, Kil Seong Lee, Yeonjoo Kim, Group Decision Making Approach to Flood Vulnerability Assessment, Journal of Korea Water Resources Association, 10.3741/JKWRA.2013.46.2.99, 46, 2, (99-109), (2013).
- Josep Freixas, Dorota Marciniak, Montserrat Pons, On the ordinal equivalence of the Johnston, Banzhaf and Shapley power indices, European Journal of Operational Research, 10.1016/j.ejor.2011.07.028, 216, 2, (367-375), (2012).
- Mark Rempel, undefined, 2012 IEEE Symposium on Computational Intelligence for Security and Defence Applications, 10.1109/CISDA.2012.6291513, (1-8), (2012).
- RANDY MARTENS, MARK REMPEL, High‐Level Methodologies to Evaluate Naval Task Groups, Naval Engineers Journal, 10.1111/j.1559-3584.2010.00264.x, 123, 4, (67-80), (2012).
- Alexander Eisl, Manuel Lingo, Hermann Elendner, Re-Mapping Credit Ratings, SSRN Electronic Journal, 10.2139/ssrn.1836877, (2011).
- Jiří Mazurek, EVALUATION OF RANKING SIMILARITY IN ORDINAL RANKING PROBLEMS, Acta academica karviniensia, 10.25142/aak.2011.028, 11, 2, (119-128), (2011).
- Kurt Hornik, Rainer Jankowitsch, Manuel Lingo, Stefan Pichler, Gerhard Winkler, Determinants of heterogeneity in European credit ratings, Financial Markets and Portfolio Management, 10.1007/s11408-010-0134-x, 24, 3, (271-287), (2010).
- Yue-Ping Xu, Yeou-Koung Tung, Decision Rules for Water Resources Management under Uncertainty, Journal of Water Resources Planning and Management, 10.1061/(ASCE)0733-9496(2009)135:3(149), 135, 3, (149-159), (2009).
- Raymond P.L. Buse, Westley Weimer, undefined, 2009 IEEE 31st International Conference on Software Engineering, 10.1109/ICSE.2009.5070516, (144-154), (2009).
- Bohdan L. Kaluzny, R. H. A. David Shaw, Optimal aircraft load balancing, International Transactions in Operational Research, 10.1111/j.1475-3995.2009.00723.x, 16, 6, (767-787), (2009).
- Stefan A. Hajkowicz, Supporting multi-stakeholder environmental decisions, Journal of Environmental Management, 10.1016/j.jenvman.2007.03.020, 88, 4, (607-614), (2008).
- Stefan A. Hajkowicz, Sarah A. Wheeler, Evaluation of Dairy Effluent Management Options Using Multiple Criteria Analysis, Environmental Management, 10.1007/s00267-007-9060-8, 41, 4, (613-624), (2008).
- Kurt Hornik, Manuel Lingo, Rainer Jankowitsch, Stefan Pichler, Gerhard Winkler, Benchmarking Credit Rating Systems, SSRN Electronic Journal, 10.2139/ssrn.930376, (2007).
- Madjid Tavana, Frank LoPinto, James W. Smither, A Hybrid Distance-Based Ideal-Seeking Consensus Ranking Model, Journal of Applied Mathematics and Decision Sciences, 10.1155/2007/20489, 2007, (1-18), (2007).
- Wade D. Cook, Distance-based and ad hoc consensus models in ordinal preference ranking, European Journal of Operational Research, 10.1016/j.ejor.2005.03.048, 172, 2, (369-385), (2006).
- Kurt Hornik, Rainer Jankowitsch, Manuel Lingo, Stefan Pichler, Gerhard Winkler, Validation of Credit Rating Systems Using Multi-Rater Information, SSRN Electronic Journal, 10.2139/ssrn.871213, (2006).
