A simulation comparison of tournament designs for the World Men's Handball Championships

The study aims to compare different designs for the World Men's Handball Championships. This event, organised in every two years, has adopted four hybrid formats consisting of knockout and round-robin stages in recent decades, including a change of design between the two recent championships in 2017 and 2019. They are evaluated under two extremal seeding policies with respect to various outcome measures through Monte-Carlo simulations. We find that the ability to give the first four positions to the strongest teams, as well as the expected quality and outcome uncertainty of the final is not necessarily a monotonic function of the number of matches played: the most frugal format is the second best with respect to these outcome measures, making it a good compromise in an unavoidable trade-off. A possible error is identified in a particular design. The relative performance of the formats is independent of the seeding rules and the competitive balance of the teams. The recent reform is demonstrated to have increased the probability of winning for the top teams. Our results have useful implications for the organisers of hybrid tournaments.


List of Tables
The distribution of matches played in different tournament formats . . . . 11 3 Dependence of some tournament metrics on the number of iterations . . . 12 4 The probability that one of the best teams wins the tournament . . . . . 13 5 The probability difference that one of the best teams wins the tournament, compared to a round-robin tournament with 24 teams . . . . . . . . . . . . 14 6 The probability that at least one of the best teams plays in the final . . . 14 A.

Introduction
Finding the optimal design of sports tournaments is an important question of scientific research (Szymanski, 2003). Ignoring the assignment of referees (Alarcón et al., 2014;Atan and Hüseyinoǧlu, 2017) -who may be biased, for example, towards the home team (Garicano et al., 2005) -, organisers and sports governing bodies have supposedly no influence on match outcomes. However, they can certainly choose other characteristics of a tournament, including the format (Scarf et al., 2009;Scarf and Yusof, 2011;Guyon, 2018), the schedule of individual matches (Ribeiro, 2012;Atan and Hüseyinoǧlu, 2017;Durán et al., 2017), the seeding policy (Guyon, 2015;Laliena and López, 2018), or the progression rules (Csató, 2018a(Csató, ,b, 2019bDagaev and Sonin, 2018;Vong, 2017). The current paper considers tournament designs as probabilistic mechanisms that select high-quality alternatives (players or teams) in a noisy environment (Ryvkin, 2010). Operational Research (OR) can play a prominent role here by analysing the effects of different competition structures on particular aspects of the sporting event: given a particular metric as an objective, and respecting design constraints, it becomes possible to choose the most favourable version.
In sports involving pairwise matches, there are two fundamental tournament formats (Scarf et al., 2009). The first is the knockout tournament where matches are played in rounds such that the winners play against each other in the next round, while the losers are immediately eliminated from the tournament. The sole remaining player, the winner of the final gets the first prize. The second basic design is the round-robin tournament where every competitor plays every other such that they earn points based on their number of wins, draws, and losses. The winner is the team with the greatest point score. All other designs can be considered as variations, such as the double elimination (McGarry and Schutz, 1997;Stanton and Williams, 2013), the Swiss system (Appleton, 1995;Csató, 2013Csató, , 2017, and hybrids like the FIFA World Cup or the UEFA Champions League in association football. Tournament success measures can be defined in a relatively straightforward way. On the other hand, the identification of design constraints is usually more complicated because they are rarely communicated by the administrators. The only plausible assumption seems to be that a format used in the past for a given tournament remains feasible in the future. However, this consideration does not help much when the tournament receives a modification to its structure only in parallel with a change in the number of competitors. For example, FIFA World Cup was expanded to 24 teams in 1982, then to 32 in 1998, while the 2026 World Cup will have 48 finalist teams, but its format has remained the same for the same number of teams in these years. Similarly, the biannual European Men's and Women's Handball Championships started with 12 teams in 1994, and were expanded to 16 teams in 2002, but were organised according to the same structure for a given number of competitors. It means that suggesting a novel design has not much practical value unless it dominates the one applied in the real-world in (almost) every respect.
In contrast, some high profile events have received a regular modification to their structure. We will analyse here a probably unique example, the IHF World Men's Handball Championship. This event is held in every two years since 1993 and is one of the most important and prestigious championships for men's handball national teams along Olympic games and the EHF European Men's Handball Championship as handball is most popular in the countries of continental Europe, which have won all medals but one in the World Men's Championships. Attendance of the Championship in 2019, hosted by Denmark and Germany, was over 900,000, more than 9,000 per match.
The number of qualified teams has remained fixed at 24 since 1995, but the tournament format has changed several times over the last two decades. Between 1995 and 2001 (four events), there were group games in the preliminary round, followed by a knockout stage. This format was used again between 2013 and 2017 (three events). However, there were two subsequent group stages between 2003 and 2011 (five events), in three different variants, one of them returning in 2019. To conclude, there are four tournament structures implemented in recent years, including a change between the two recent tournaments. This indicates that the organisers experiment with finding the best design, which offers an extraordinary opportunity to compare them with the tools of OR.
It is clear that such complex designs, consisting of knock-out and round-robin stages, can be analysed only via Monte-Carlo simulations. Academic literature has made several attempts to address similar problems. Scarf et al. (2009) propose a number of tournament metrics and describe how they may be evaluated for a particular design. The authors use the UEFA Champions League to illustrate their methodology. Scarf and Yusof (2011) extend this investigation by considering the effect of the seeding policy on outcome uncertainty while taking competitive balance into account. Goossens et al. (2012) examine four league formats that have been considered by the Royal Belgian Football Association. Lasek and Gagolewski (2015) compare the recently introduced competition format for the top association football division in Poland to the standard double round-robin structure. Yusof et al. (2016) develop a system called 'E-compare of Soccer Tournament Structures' to assist decision makers in determining the fairest design for association football tournaments. Lasek and Gagolewski (2018) examine the efficacy of league formats in ranking football teams and find that the performance of formats consisting of round-robin stages mainly depends on the total number of matches played. Dagaev and Rudyak (2019) assess a recent reform of the seeding system in the UEFA Champions League. Csató (2019a) evaluates an alternative of the traditional multi-stage tournament design through the example of the EHF Champions League, the most prestigious men's handball club competition in Europe.
Most of these papers use specific models for simulating match results, however, we want to avoid the use of such sophisticated assumptions to compare the tournament formats for a number of reasons. First, we follow general works on the efficacy of sports tournaments (Appleton, 1995;McGarry and Schutz, 1997) or ranking methods (Mendonça and Raghavachari, 2000), which apply this choice. Second, at least according to our knowledge, there exists no particular prediction model fitted to handball results, contrary to the variety of methods making a good prediction on the outcome of a single match between two football teams (Maher, 1982;Dixon and Coles, 1997;Koning et al., 2003). The main difficulty is probably that handball is a fast, dynamic and high-scoring game, where professional teams now typically score between 20 and 35 goals each, therefore the technical analysis of a handball match poses a serious challenge (Bilge, 2012;Gruić et al., 2007). According to Dumangane et al. (2009), the dynamics of handball matches violate both independence and identical distribution, in some cases having a non-stationary behaviour. In addition, some tournament designs analysed here have been applied only once, so the lack of historical data prevents fitting a specific prediction model. Third,  prove that in round-robin tournaments among three or four symmetric contestants, there is a first-mover advantage driven by strategic effects arising from the subgame perfect equilibrium, while Krumer and Lechner (2017) give an empirical proof of this finding. Since all of our designs contain at least one group stage, even the schedule of the matches may influence the outcome of the tournament.
To summarise, the exact modelling of handball matches organised in such complicated hybrid designs seems to be beyond the current knowledge of the academic community. However, since our intention is only to compare the tournament formats, and not to estimate the chance of winning, a number of models within reason could be taken to determine the winners (Appleton, 1995). Nonetheless, this implies that all calculations are for comparative purposes only.
The main contribution and novelty of our research is the analysis of a particularbut by no means marginal -handball tournament by simulations, which has received several modifications to its format recently, indicating that the organisers are probably uncertain on its appropriate design. While the choice of tournament format is driven by a number of factors (Szymanski, 2003;Wright, 2014), we focus on its ability to give the first four positions to the strongest teams, and on the quality and competitive balance of the championship final. Consequently, in the following, a design will be called more efficacious if it performs better with respect to all these criteria -which can be conflicting, hence this definition of efficacy is not guaranteed to produce a strict ranking of all formats.
Note that our definition does not coincide with the standard meaning of efficacy, the ability of a tournament to produce accurate rankings with respect to teams' true abilities. However, in our view, the latter approach is reasonable only in round-robin tournaments (Lasek and Gagolewski, 2018), or if the number of competitors is small (McGarry and Schutz, 1997;Mendonça and Raghavachari, 2000). Now the ranking outside the top four is unreliable and almost irrelevant as all designs are centred around the semifinals, while the tournament final has a prominent role in creating media attention, so taking only the ranking ability into account is not enough if the key determinants of demand (Borland and MacDonald, 2003) are unfavourable for the most important match.
We have some surprising findings, for example, the most frugal design in the number of matches played is the second best with respect to efficacy, thus it seems to be a good compromise in the unavoidable trade-off. This is mainly caused by the smaller groups of four teams each instead of six in the first round-robin stage, a suggestion is worth further consideration. Our calculations also reveal that the recent format change of the World Men's Handball Championship has increased the probability of winning for the top teams.
In short, the results will have useful implications for hybrid tournaments that are applied in several sports such as basketball, handball, and volleyball, some of them are presented at the end of the paper.
The paper is structured as follows. Section 2 describes the tournament designs, the metrics used for the comparison of different formats, and the simulation experiment. The results and their sensitivity analysis are detailed in Section 3. Finally, Section 4 discusses our main findings and concludes.

Methodology
For the comparison of different tournament designs, it is necessary to use simulations as historical data are limited because some formats were applied only once.

Tournament designs
The IHF World Men's Handball Championships have been organised with 24 participating teams in four fundamentally different designs in recent decades. Our investigation is restricted to these tournament formats in order to avoid the question of whether the suggested design can be implemented in practice.
Each format contains one or two group stages. Groups are round-robin tournaments with all teams playing once against any other team in their group. In the case of two group stages, the results of the matches played in the preliminary round between teams of the same main round group are carried over to the main round (Csató, 2019b).
Organisers provide a strict final ranking at the end of the tournament, meaning that usually there are some placement matches played by the teams already eliminated. We focus on the first four places, our stylised model contains only a third-place game played between the two losers of the semifinals, similarly to the actual tournaments. Note that there were no playoffs for the 5-8th place in the 2013 and 2017 World Men's Handball Championships. Notes: Gr. = Number of groups in the preliminary and main round, respectively; Teams = Number of teams in each group of the preliminary and main round, respectively; Q = Number of teams qualified from each group of the preliminary and main round, respectively In the following, the designs that have been used recently in the World Men's Handball Championships are presented. Table 1

Two group stages with 6 and 4 teams per group ( 64)
This design, presented in Figure A.2, has been used in the 2003 World Men's Handball Championship, hosted by Portugal. It contains two group stages (see Figure A.2a). The preliminary round consists of four groups of six teams each such that the top four teams qualify for the main round. The main round consists of four groups of four teams each such that two teams in each main round group are from the same preliminary round group, the first and the third, or the second and the fourth. Therefore, all teams play two further matches in the main round. Only the group winners of main round groups qualify for the semifinals in the knockout stage (see Figure A.2b).

Two group stages with 6 and 6 teams per group ( 66)
This design, presented in Figure A.3, has been used first in the 2005 World Men's Handball Championship and has been applied in 2009, 2011, and 2019. It contains two group stages (see Figure A.3a). The preliminary round consists of four groups of six teams each such that the top three teams qualify for the main round. The main round consists of two groups of six teams, each created from two preliminary round groups. Therefore, all teams play three further matches in the main round. The top two teams of every main round group advance to the semifinals in the knockout stage (see Figure A.3b). ( 46) This design, presented in Figure A.4, has been used in the 2007 World Men's Handball Championship, hosted by Germany. It contains two group stages (see Figure A.4a). Teams are drawn into six groups of four teams each in the preliminary round such that the top two teams proceed to the main round. The main round consists of two groups, each created from three preliminary round groups. Therefore, all teams play four matches in the main round. Four teams of a main round group advance to the quarterfinals in the knockout stage (see Figure A.4b).

Round-robin ( )
While the 24 competitors have never played a round-robin tournament, we use this basic format as a reference.

Seeding policy
Seeding plays an important role in knockout tournaments (Hwang, 1982;Schwenk, 2000;Marchand, 2002;Groh et al., 2012;Karpov, 2016;Dagaev and Suzdaltsev, 2018;Karpov, 2018). It is not an issue in our case since the knockout stage of all formats is immediately determined by the previous group stage (see Figures A.1-A.4). On the other hand, all participants should be drawn into groups before the start of the tournament, and this policy may influence the outcome, too (Guyon, 2015;Dagaev and Rudyak, 2019;Guyon, 2018;Laliena and López, 2018). In the recent World Men's Handball Championships, the pots were determined on the basis of geography and other aspects such as qualification results. For example, in the 2009 tournament, Pot 1 contained the host (Croatia), the defending World Champions (Germany), the champions of Europe (Denmark), and the third-placed team of the recent European Championship (France), where Croatia and Germany were the second-and fourth-placed teams, respectively.
We consider two variants of each tournament design called seeded and unseeded. In the seeded version, the preliminary round groups are seeded such that in the case of groups ( = 6 for design 46 and = 4 otherwise), the strongest teams are placed in Pot 1, the next strongest teams in Pot 2, and so on. Unseeded version applies fully random seeding. In this case, some strong teams, allocated in a harsh group, may have more difficulty in qualifying than weaker teams allocated in an easier group, which is inefficient and can be regarded as unfair.
Naturally, there is no need to seed the teams in the reference format .

Tournament metrics
Following the literature (Horen and Riezman, 1985;Scarf et al., 2009;Dagaev and Rudyak, 2019), the following tournament success measures have been chosen: • the probability that one of the best teams wins the tournament; • the probability that at least one of the best teams plays in the final; • the average pre-tournament rank of the winner, the second-, the third-and the fourth-placed teams; • the expected quality of the final (the sum of the finalists' pre-tournament ranks); • the expected competitive balance of the final (the difference between the finalists' pre-tournament ranks).
We focus only on the first four places because there was a third place game in all World Men's Handball Championships since 1995, however, other placement matches were organised arbitrarily.

Simulation procedure
Given the design and a prediction model for match results, we are able to simulate a complete tournament repeatedly and obtain estimates of any metrics of interest.

Playing abilities
The probability with which a given team would beat another team is fixed a priori. We have chosen a generalised version of Jackson (1993)'s model for this purpose: where is the probability that team defeats team , , ≥ 0 are parameters and 1 ≤ , ≤ 24 is the identifier of the teams. The model was used by Jackson (1993) and Marchand (2002) with = 0. The role of this novel parameter is to lessen the sharp increase of winning probabilities for the strongest teams. The function of is similar to the original model, its smaller or larger values reflect situations where there is a smaller or larger dispersion in the teams' strengths, respectively.
Stationarity and independence of the probability that team beats team is assumed, it does not change throughout the tournament and is independent of the previous results. While in practice they are dynamic and changing probabilities are expected to alter the outcome of the tournament on a single occasion, it seems to be reasonable that stationary probabilities are good approximations of long-run averages (McGarry and Schutz, 1997).
Baseline results are obtained with = 4 and = 24 but a robustness check will be provided for both parameters. Figure 1 shows the probabilities of beating the opponents for certain teams as derived from formula (1). Our choice somewhat follows the idea behind Matrix I of McGarry and Schutz (1997): adjacent teams are closely matched (team − 1 defeats team with a probability of no more than 0.54), but the difference between a top team and an underdog is significant (the strongest team has greater than 90% chance to win against the last five teams).

Technical details
A handball game may be tied at the end of the regular playing time. If a winner has to be determined, namely, in the knockout stage of a tournament, it is followed by the first and (if it is necessary) the second overtime of 10 minutes, and the match is finally decided with penalty throws. It is a less frequent event than a draw in football, for example, in the 2017 World Men's Handball Championship, which was organised according to format , there were three draws from the 60 group matches, and one draw from the 16 matches of the knockout stage. Thus, following McGarry and Schutz (1997), draws are not allowed in the simulation. This is not to be confused with ties in the ranking of round-robin groups, resolved in our simulations with an equal-odds 'coin toss'.
Every simulation has been run one million times ( = 1,000,000) such that two matrices with match outcomes have been generated for each possible pair of opponents in every run because some teams may play two matches against each other (however, it is not possible before the semifinals). After that, these outcomes have been plugged into the competition formats analysed to study the outcome of the tournament: we have recorded the identifier of the first four teams and the teams which play the final in each run. Thus any differences in tournament metrics are solely caused by the designs.
The validity of the simulation procedure has been tested in several ways. First, a matrix representing equality among all teams ( = 0.5 for all combinations of and ) has led to, as expected, an outcome where all teams are placed first to fourth equally often. Second, simulations with a fully deterministic matrix ( = 1 if < ) have been analysed. It still shows the differences between our tournament designs. For example, in the seeded versions of 66 and 46, the four best teams are guaranteed to occupy the first four places in their natural order. However, in the seeded and 64, the two strongest teams can meet in the semifinals with a probability of 1/3. Regarding the unseeded variants, the worst team that may qualify for the semifinals is the sixth in 46, the seventh in , and the fourteenth in 64 and 66. Finally, some values have been changed in the fully deterministic matrix in order to see whether they function in an expected way.

The comparison of tournament designs
In the following, our findings on the four tournament designs that have been used in the recent World Men's Handball Championships, are reviewed.

Match distribution
By looking at the tournament formats, it can be realised that two teams may play at most two times against each other, and this number could be two only if one of these matches is a semifinal, the final or the third-place game.
Each design requires different number of matches: • A round-robin tournament with 24 teams contains 24 × 23/2 = 276 games.
• Format 46 contains 6×4×3/2 = 36 games in the preliminary round, 2×6×4/2 = 24 games in the main round, and 4 + 2 + 2 = 8 games in the knockout stage, that is, 68 in total. Besides the total number of matches, its distribution, presented in Figure 2, is also interesting. This reports the number of teams with a given number of matches, for example, under the design , eight teams play five matches. In 46, half of the teams play only three matches, however, the others play at least seven. For the three remaining designs, the minimum number of games to be played by a team is five. The maximum is nine in and 64, while ten in 66 and 46.

Number of independent runs
The proportion of finals played by the two highest pre-tournament ranked teams We have tested the simulations with the unseeded variant of tournament design for various number of independent runs (Figure 3). Since two success measures, the proportion of tournament wins for the highest ranked team, and the proportion of tournament finals between the two highest ranked teams are stable after one million (10 6 ) runs, we have decided to implement all of our following simulations with one million runs.
The first tournament metric to be analysed, the probability that one of the best teams wins the tournament is shown in Figure 4 for some tournament designs. As expected from the number of matches played in each design (Section 3.1), the round-robin is the format that maximises the probability of winning for the best teams. Furthermore, the four designs of the World Men's Handball Championships are almost indistinguishable, therefore it is worth calculating the difference between these formats compared to the reference , as presented in Figure 5. This reveals that design 66 is the best from the perspective of its ability to select the strongest teams as the winner, followed by 46, while and 64 perform similarly. Furthermore, seeding has not much effect, with the possible exception of format : while the seeded variants of and 64 are almost indistinguishable, becomes marginally better with random seeding. The same pattern is attested for our second tournament metric, the probability that at least one of the best teams plays in the final (Figure 6).
The remaining outcome characteristics for the nine tournament designs are summarised in Table 2. As it has already been mentioned in the Introduction, a format is said to be more efficacious if the average pre-tournament rank of the team finishing in the th place ( = 1, 2, 3, 4) is smaller, as well as the expected quality and the expected competitive balance of the final is lower (more favourable).
While the round-robin design shows the best performance in selecting the highest pre-tournament ranked teams as the winner, it requires a large number of matches, and only the other formats can be applied in practice. Among the seeded variants, 66 is the most efficacious, followed by 46, while the order of and 64 remains undecided, although the former has a marginal advantage. This order holds for all criteria of efficacy considered here, although they can be conflicting.
Eliminating the seeding procedure changes the metrics according to our expectations, for example, the unseeded 66 is approximately at the same level as the seeded 46, whereas the unseeded 46 is still more efficacious than the . An interesting observation -perhaps a kind of puzzle -is that the performance of the design , where the knockout stage plays the greatest role, is essentially not influenced by seeding.
The metrics of seeded 64 may refer to a flaw of this format because the average pre-tournament ranking of the bronze medallist is not substantially greater than the average ranking of the silver medallist. The unexpected phenomenon is perhaps caused by its strange knockout stage, where only the group winners of the main round compete.
Focusing on the averages of our success measures may mask some variance behind them. For instance, the same expected pre-tournament rank of the winner can be obtained if a format gives a higher probability for the top and the bottom teams, but harm the middle teams. Therefore, Figure A.5 plots the probability difference of reaching the first four places as a function of the pre-tournament rank, compared to competition design . For example, the probability of the strongest team winning the championship is more than 2.5% higher under format 66 than under format , a more than 10% increase. The graph reinforces that the difference between seeded and 64 is negligible, especially in the allocation of the first two places. On the other hand, designs 66 and 46 are preferred only by the four or five strongest teams. In addition, the lines do not converge to zero even for the weakest teams in the case of the fourth place (#4) because the presence of two well-constructed subsequent group stages is effective against the occasional It is reasonable to assume that if a given design is more efficacious than another in both its seeded and unseeded variants, then it remains more efficacious in real-life when the actual allocation of the teams into pots is somewhere between these two extreme cases.
Naturally, all results should be considered with respect to the number of games played. It is the smallest, 68 for the design 46, so its second-place according to efficacy has a favourable message for the organisers: there exists no clear trade-off between efficacy and the number of matches. This is in contrast to the intuition and the conclusions of many tournament design papers. For example, the performance of soccer league formats highly depends on the total number of matches played (Lasek and Gagolewski, 2018).
The underlying reason is probably that half of the participating teams play only three games in 46, which seems to be enough to determine the competitors with the greatest chance to win the tournament. In addition, this is the only design containing quarterfinals after two group stages. The remaining three formats are closer to each other from this point of view, all teams play at least five matches, and the total number of games is between 76 and 82. This fact also shows that 64 is a misaligned design because of the relatively high number of matches could not reduce outcome uncertainty.

Sensitivity analysis
Following Scarf et al. (2009), the robustness of the results is addressed by calculating our metrics for more and less competitive tournaments than the baseline version. It is achieved in two ways, by changing: the parameter in formula (1) from its original value of 4 to 3 (more competitive) and 5 (less competitive); and the parameter in formula (1) from its original value of 24 to 18 (less competitive) and 36 (more competitive). Figure A.6 reproduces Figure 5 for these cases. It can be seen that the ranking of the competition designs by their ability to select the best teams as the winner remains unchanged as 66 is better than 46, which outperforms and 64. The seeded and 64 formats are almost indistinguishable, but the former outperforms the latter from this point of view without seeding. The advantage of an ideal round-robin tournament becomes more significant if competitive balance is smaller, that is, the outcome of the matches is more difficult to forecast. Figure A.7 reinforces that seeding has not much influence on tournament outcomes, and, while the actual differences among the four designs are modest (at least compared to the round-robin format), they are robust with respect to the distribution of teams' strength.
Further tournament characteristics are summarised in Tables 3 and 4. Our conclusions do not change significantly, although the unseeded is clearly worse than its seeded variant if = 3 or = 36 when the teams' abilities are more similar. The final of the seeded is more exciting with higher quality and outcome uncertainty than the final of the seeded 64 in these cases, too. A possible flaw of design 64 under seeding (the average pre-tournament rank of the third-placed team is close to the average rank of the second-placed) can be observed as before, too.

Discussion
We have compared four tournament formats of recent World Men's Handball Championships. They have been evaluated by Monte-Carlo simulations under two seeding policies, namely, allocating teams perfectly into pots on the basis of their known strength, and a fully random draw of groups. Our main findings are as follows: • (applied from 1995 to 2001 and between 2013 and 2017, see Section 2.1.1): it is almost insensitive to the seeding rule. While this seems to be a somewhat surprising fact because the knockout phase plays the greatest role in this format, Marchand (2002) provides evidence that the outcome of the standard and random knockout tournaments may not vary as much as one might expect.
• 64 (applied in 2003, see Section 2.1.2): it turns out to be a questionable design because of its weak ability to select the best teams despite the relatively high number of matches, and the average pre-tournament rank of the third-placed team is not substantially higher than the average pre-tournament rank of the second-placed team in the seeded variant.  due to the highest number of matches played among the designs considered.
• 46 (applied in 2007, see Section 2.1.4): it means a good compromise between efficacy and compactness as the only format with better performance ( 66) requires a 20% increase in the number of matches.
Our analysis clearly shows that no single best tournament design exists. For example, format 46 allows only three matches for certain teams before they are eliminated, which may be regarded as the price for an appropriate selection of top teams. On the other hand, format is insensitive to the drawing of groups, therefore this competition structure minimises randomness in a sense by being independent of the seeding policy. In short, we can agree with Scarf et al. (2009) that one cannot come up with a unique definition of fairness that all would accept.
Nonetheless, the current paper has an important message for the governing bodies of major sports: the obvious conclusion from the intuition and the principle of statistics that a bigger sample lead to better estimates does not necessarily hold in the case of such complex hybrid tournament designs as the comparison of formats 64 and 46 reveals. Naturally, all results are based on a particular probabilistic model, which implies certain limitations. However, we have made great effort to minimise this sensitivity by studying a variety of robustness check, and it seems that a wide range of model assumptions are appropriate for comparative purposes (Appleton, 1995). These competition designs have been used in other team tournaments with 24 participants, too. The IHF World Women's Handball Championship is organised in every two years since 1993, and has 24 teams since 1997. Its format has followed the World Men's Handball Championship taken place in the same year, except for 2003 -when women handball teams competed under design 66, while men played in format 64, thus no women tournament was organised according to this dubious design -, and for 2011 -when women national teams competed under design , while men played in format 66. Similarly to the Men's Championship, the next Women's Championship to be held in 2019, hosted by Japan, will also use the format 66 instead of (IHF, 2018). In basketball, the 2006 and the 2010 FIBA World Championships as well as the 2014 FIBA Basketball World Cup (the tournament previously known as the FIBA World Championship), the EuroBasket 2015, and the EuroBasket 2017 applied the design . Format 66 was used in the 1986 FIBA World Championship, while the EuroBasket 2011 and the EuroBasket 2013 applied 66 with a slight modification that four teams advanced from each of the two main round groups to the quarterfinals (instead of only two to the semifinals). Finally, the 1978 and 1982 FIVB Volleyball Men's World Championships were organised in a structure similar to 46, but only the two top teams from the two main round groups qualified for the semifinals, while in handball, the second group stage was followed by the quarterfinals (see Figure A.4). In the view of our computations, perhaps it is not a coincidence that no further use of the strange design 64 has been found in practice.
Organisers of team championships are encouraged to consider our results when deciding on the design of future tournaments. For example, the recent change of the World Men's Handball Championship format (from to 66 between 2017 and 2019) has increased the probability of winning for the best teams as revealed by Figure A.5. The choice of tournament design is an especially important issue because it offers perhaps the only way to influence the expected value of certain success measures for sports administrators.
There is a great scope for future research. First, one can implement a more extensive sensitivity analysis. Second, as discussed in the Introduction, our simulation is not based on data from real tournaments since it is far from trivial to model handball matches. Third, other tournament designs or simple modifications of the formats analysed here (recall that a slightly modified variant of structure 46 was used in volleyball) can be investigated with the presented methodology. Finally, further properties of the competition formats are worth examining. For example, it is almost obvious to check that design satisfies strategy-proofness, while formats 64, 66, and 46 are incentive incompatible (Csató, 2019b