MEAN–VARIANCE VERSUS FULL-SCALE OPTIMIZATION: BROAD EVIDENCE FOR THE UK

Authors


Abstract

Portfolio choice by full-scale optimization applies the empirical return distribution to a parameterized utility function, and the maximum is found through numerical optimization. Using a portfolio choice setting of three UK equity indices we identify several utility functions featuring loss aversion and prospect theory, under which full-scale optimization is a substantially better approach than the mean–variance approach. As the equity indices have return distributions with small deviations from normality, the findings indicate much broader usefulness of full-scale optimization than has earlier been shown. The results hold in- and out-of-sample, and the performance improvements are given in terms of utility as well as certainty equivalents.

1 Introduction

It is well known that asset returns probability distributions in general feature both skewness and excess kurtosis. Investor preferences of these higher moments are attracting increasing attention by portfolio choice researchers and the financial industry, often arriving at more complex utility functions than traditionally assumed. In full-scale optimization1 empirical return distributions are used in their entirety, and the choice of utility function is completely flexible. In this paper we assess the performance of FSO using a wide range of utility function specifications applied to equity indices. We show that when utility functions include loss aversion or prospect theory, even assets with only small deviations from normality are better selected in an FSO framework than with the traditional mean–variance (MV) approach.

The strength of FSO is that no analytical solution to the portfolio choice problem is pursued. This allows the distributional properties of returns to be left non-parameterized and the utility function to be specified to reflect investor preferences uncompromised by mathematical convenience. The absence of simplifying assumptions, yielding theoretical appeal, comes at the cost of computational burden, however. As the optimization problem is not convex, a grid search or a global search algorithm has to be used to find the utility-maximizing portfolio.

Cremers et al. (2005) show in a hedge fund selection problem that the performance of FSO (in terms of utility) is substantially better than Markowitz's (1952, 1959) MV approach when investor preferences are modeled to include loss aversion or prospect theory (based on Kahnemann and Tversky, 1979). The results are confirmed in an out-of-sample application by Adler and Kritzman (2007). Several other portfolio choice papers resembling FSO but not using the term have appeared lately. Statistical properties of the estimator have been explored by Gourieroux and Monfort (2005). Scenario-based approaches (using hypothesized outcomes with probabilities attached instead of empirical return distributions) have been dealt with by Grinold (1999) and Sharpe (2007). Higher moments properties of utility-maximizing portfolios have been investigated by Maringer (2008), who also proposes heuristic optimization methods to deal with the computational burden of the optimization problem.

The idea of utility maximization as a methodology for portfolio optimization problems, based on the utility theory founded by Von Neumann and Morgenstern (1947), can be traced back at least to Tobin (1958), and also appears in several assessments of the MV approach (e.g. Levy and Markowitz, 1979; Markowitz, 1987). As the latter studies showed that the performance difference between MV portfolios and utility-maximizing portfolios (using power utility) was negligible, the less burdensome MV approach became the model of choice in the financial industry, and the benchmark in academia.2

The relevance of higher moments for investment decisions was pointed out by Levy (1969) and Samuelson (1970), and in realistic portfolio management situations, investors often express preferences that imply more complex utility functions than power or quadratic utility (Litterman, 2003, Ch. 2; Meucci, 2005, Ch. 5). It is when implementing such preferences (loss aversion and prospect theory) that the FSO has been proven superior to MV (Cremers et al., 2005; Adler and Kritzman, 2007). In these assessments, however, assets with extremely non-normal returns (hedge funds) have been used, and the results have only been shown to hold for a few examples of utility function specifications. Our assessment of FSO features a selection of equity indices with returns much closer to a normal distribution (in general still non-normal though) and a much broader spectrum of utility function specifications. Finding that FSO performance is substantially better than MV when complex investor preferences (in particular prospect theory) hold, our results indicate that the earlier studies are robust with respect to utility function specification, as well as to other asset classes (equity indices).

This paper is organized as follows. Section 2 discusses the portfolio choice problem and the characteristics of investors' preferences. Section 3 presents data, utility functions and methodology used in our assessment of FSO. The results are presented and analyzed in Section 4, and Section 5 concludes.

2 Portfolio Selection and Investor Preferences

2.1 The Portfolio Selection Problem

Single-period portfolio selection models can in general be described as utility maximization problems such as.

image(1)

Here, R is an (n × T) matrix containing expected returns of n admissible assets in T different scenarios. θ is a vector of length n containing the portfolio weights for each asset. Utility is a function U of expected portfolio return, θR, which θ is chosen to maximize, subject to the constraint matrix Ω. Typically, Ω includes a budget constraint such as θι = 1 (where ι is a vector of ones), but it may also include other constraints, e.g. a short selling constraint or a loss aversion constraint.

The portfolio return θR is a vector of length T. Its distribution will differ depending on the portfolio weights chosen in θ. The functional form of the utility function should mirror the investor's preferences to determine which expected portfolio distribution is preferred.

In the MV approach, the utility function is quadratic, which implies that the mean and variance of the expected returns for each asset in R are all that determine the utility. In FSO, R contains historical returns of each asset and the utility for each possible θ is evaluated at each point in time available in the sample. This means that all features of the empirical return distribution are considered, not only the first two moments.

Setting up the optimization problem in this way typically does not yield convexity, and analytical solutions cannot always be found. Instead search techniques must be used, such as a grid search or a stochastic search algorithm. The problem may be thought of as constructing an allocation matrix Θ containing each possible portfolio weight combination θ. The allocation matrix dimension is (n × m), where n is the number of assets considered and m is the number of possible portfolio allocation combinations, which is a function of n and a precision parameter p.3 Each column of Θ represents one allocation combination vector θ (n × 1). To find the optimal θ, the utility for each theta is evaluated for each asset returns vector Rt, which contains returns on each asset i (i = 1, 2, . . .  , n) at time t (t = 1, 2, . . .  , T). Each of the T Rt vectors will have the dimension (n × 1) and elements inline image, where Pi,t is the price of asset i at time t. The θ with the highest average utility over time will be the optimal allocation combination, θFSO. This is shown formally in equation (2):

image(2)

A key feature of FSO is that any utility function can be chosen, as the optimization is carried out numerically. This allows the optimization to consider more complex investor preferences than traditionally assumed.

2.2 Investor Preferences

Investor preferences implied by utility functions can be investigated by expanding the utility function in a Taylor series around the mean (µ1) and taking expectations on both sides, as shown in equation (3). This yields measures of the investors' preferences in terms of the distribution's moments. Let Uk denote the kth derivative of the utility function and µj the jth moment of the portfolio return; then, set up in the same fashion as in Scott and Horvath (1980) the expected utility takes the following form:4

image(3)

The expression shows that the expected utility equals the utility of the expected returns, plus the impact on utility of deviations from the expected return. The influence of each moment on expected utility is weighted by the corresponding order derivative of the utility function. Typically, U2(µ1) for variance is negative; U3(µ1) for skewness is positive; and U4(µ1) for kurtosis is negative.5

In the MV model the covariances of the assets play an important role. Implicitly, utility functions with higher moment preferences different from zero also take co-moments, such as co-skewness and co-kurtosis, into account.6

In principle there is no limit on the number of moments to consider, but higher moments than kurtosis (k > 4) have not been considered in the finance literature, and will not be discussed here. However, according to Scott and Horvath (1980), most investors have utility functions where moments of odd order (i.e. k = 1, 3, 5, . . .) have positive signs on the respective derivative and moments of even order have negative derivatives.

In this paper we consider four families of utility functions. Their general mathematical forms are presented in Table 1. Here, we define utility in terms of the portfolio return rather than over wealth. This approach may be interpreted as a normalization of initial wealth to one, i.e. W0 = 1. A motivation for defining utility directly in terms of returns can be found in Kahnemann and Tversky (1979). They argue that people focus more on the return on an investment than on the level of wealth. Figure 1 displays how the utility varies with returns for certain specifications of the four utility function families.

Table 1. 
Utility Function Equations
Utility functionUtility
  1. Notes: In all the utility functions, rp represents portfolio return. A represents the degree of (absolute) risk aversion in the exponential utility functions. In power utility functions, γ is the degree of (relative) risk aversion. The special case when γ = 1 is also called logarithmic utility. In the bilinear utility specification, x is the critical return level called the kink. P is the penalty level for returns lower than the kink. In the S-shaped utility functions, the critical return level (the inflection point) is indicated by z. A, B, γ1 and γ2 are parameters determining the curvature of the S-shaped utility.

Exponential−exp[−A(1 + rp)] 
Powerinline imagefor γ > 0
ln(1 + rp)for γ = 1
Bilinearln(1 + rp)for rp ≥ x
P(rp − x) + ln(1 + x)for rp < x, P > 0
S-shapedinline imagefor rp ≤ z
inline imagefor rp > z
Figure 1.

Utility Function Graphs
Notes: Panel (a) shows how exponential utility varies over returns when the risk aversion parameter is set to A = 6. In panel (b) the same relationship for power utility is shown, with risk aversion set to γ = 2. The bilinear utility function is shown in panel (c) and has parameters set to P = 5 and x = 0 per cent (kink). In panel (d) the S-shaped utility function depicted has parameters A = 1, B = 2, γ1 = 0.3, γ2 = 0.7 and z = 0 per cent (inflection point).

The parametric, closed-form utility functions that are most common in the finance literature are the families of exponential and power utility functions. The former is characterized by constant absolute risk aversion, and the latter by constant relative risk aversion, meaning that risk aversion varies with wealth level. In Fig. 1 examples of exponential and power utility function graphs are given in panels (a) and (b), respectively.

To consider the investor preferences of skewness and kurtosis in particular, two types of utility functions that have been suggested are the bilinear and the S-shaped utility function families. Both of these are characterized by a critical point of investment return, under which returns are given disproportionally bad utility. Graphical examples of bilinear and S-shaped utility functions are given in Fig. 1(c) and Fig. 1(d), respectively.

The bilinear functions capture a phenomenon that is central in investment management today: loss aversion. The objective of limiting losses is motivated by monetary as well as legal purposes. The issue is traditionally treated with value-at-risk models, and can also be incorporated in FSO theory through a constraint on the maximization problem (as shown by Gourieroux and Monfort, 2005). The bilinear utility functions have a kink at the critical point and are formed by straight lines of different slope on each side (i.e. the functions are linear splines), which obviously yields discontinuity even in the first derivatives. Consisting of straight lines, the bilinear function does not reflect risk aversion in the sense that marginal utility is not decreasing in returns (except for the jump at the kink). This type of function has previously been applied by Cremers et al. (2005) and Adler and Kritzman (2007). For an example of how power utility can be combined with bilinear utility to feature risk aversion, see Maringer (2008).

The S-shaped utility function is motivated by the fact that it has been shown in behavior studies that an investor prefers a certain gain to an uncertain gain with higher expected value, but he/she also prefers an uncertain loss to a certain loss with higher expected return (see Kahnemann and Tversky, 1979). The utility function features an inflection point where these certainty preferences change. The utility function implies high absolute values of marginal utility close to the inflection point, but low (absolute) marginal utility for higher (absolute) returns. The first derivatives are continuous, but second derivatives are not.

3 Empirical Application

We assess the FSO methodology by comparing its performance with portfolios produced by using the MV methodology. The portfolios are optimized in a setting of three equity indices as admissible assets. The utility outcome of FSO and MV optima are examined in- and out-of-sample. To get an economic interpretation of utility differences we also calculate certainty equivalents. The exercise is repeated for a wide range of utility function specifications.

3.1 FSO Specification

We use a grid search to find the FSO optimum. As the allocation matrix grows quickly when more assets are added or the allocation precision p is increased, we use a three-asset setting with p = 0.5 per cent, and we do not allow for short selling. This yields an allocation matrix of dimension (3 × 20301), which we evaluate over 96 monthly observations. This rather limited amount of assets and precision is due to the computational burden ofthe technique.7 We analyze the full grid of possible allocations—no search algorithm is applied.8

The choice of grid precision can be made on the basis of the trade-off between the marginal utility of increasing p and the additional computational cost of doing so. Setting the grid precision to 0.1 per cent instead of 0.5 per cent increases the grid size by a factor of almost 25 (from 20,301 to 501,501). For most portfolio choice problems, that increased computational cost cannot be motivated by the increased utility achieved.

An alternative to increasing the precision of the complete grid is to perform a second grid search around the optimum found in the first search. We performed a second-step grid search using p = 0.1 per cent around the p = 0.5 per cent optima, covering all possible allocations within the range of θi ± 1 per cent.9 The second grid dimension is in our case 67 ≤ m ≤ 331, depending on whether the first-stage optimum contains allocations close to the allowed limits 0 and 1 or not. We found that both the computational cost and the utility improvement of the second grid search are minute. None of the utility improvements exceeded 1 per cent, and only eight out of 132 utility functions had improvements exceeding 0.1 per cent. This utility improvement is what we sacrifice when choosing p = 0.5 per cent rather than p = 0.1 per cent, a convenience cost. As a grid search never can yield an exact solution, we also applied simulated annealing on the area around the optimum (for a description of this technique, see, for example, Goffe et al., 1994). Again, the convenience cost was very small. We hence concluded that p = 0.5 per cent was enough for this application.10 The computational cost of the second-stepgrid search, measured in computation time, was also small. Time needed for performing FSO on each utility function type used in this paper, for first- and second-step grid searches, respectively, is given Table 2.

Table 2. 
Computational Cost of FSO
Utility functionTime first stepTime second step
  1. Notes: Time is specified in seconds. Optimization was run over 96 time periods. The platform used was an Intel Core 2 2.16 GHz processor with 3 GB RAM. Software used was R v2.6.1, applying the function system.time.

Exponential1.250.00
Power1.470.01
Bilinear35.780.13
S-shaped44.750.14

The study is performed in a one-period setting—no rebalancing of the portfolio is considered.

3.2 Data

For the empirical application we use three indices that are published by the Financial Times, downloaded from Datastream (2007): FTSE100, FTSE250 and FTSE All-World Emerging Market Index (EMI).11 The FTSE100 includes the 100 largest firms on the London Stock Exchange and FTSE250 includes mid-sized firms, i.e. the 250 firms following the 100 largest. The EMI reflects the performance of mid- and large-sized stocks in emerging markets.12 All series are denoted in British pounds (£). We calculate return series for eight years of monthly observations (January 1999–December 2006), yielding 96 observations. As shown in Fig. 2, the data feature two expansionary periods and one downward trend.

Figure 2.

Development of the Three Indices over the Time Period Considered

The data properties are presented in Table 3. All three indices display positive means over the sample period. The least volatile choice of the three is the FTSE100, followed by FTSE250 and the FTSE EMI. All of them feature negative skewness, and excess kurtosis is observed for FTSE100 and FTSE250. It is shown with a Jarque–Bera test of normality (Jarque and Bera, 1980) that normality can be rejected for the UK indices, but not for the EMI.13

Table 3. 
Summary Statistics
 MeanVarianceSkewnessKurt.J–B stat.p
  1. Notes: The table shows the first four moments of the monthly data series. Kurt. is the estimator of Pearson's kurtosis using the function kurtosis in R (this is not excess kurtosis). J–B stat. is the JarqueBera test statistic and p is the probability that the series is following a normal distribution according to this test.

FTSE1000.00100.0015−0.933.9117.070.00
FTSE2500.00960.0025−0.834.3618.320.00
FTSE EMI0.01220.0043−0.242.910.930.63

In order to mitigate the probability of corner solutions, we scale all returns to conform to implied returns of an equally weighted portfolio.14 This does not change the shape of the probability distribution, and does not affect the comparison between FSO and MV.

3.3 Utility Functions

We apply our portfolio selection problem to exponential, power, bilinear and S-shaped utility functions. The same utility function types have been investigated before, but only a few cases of each type. We perform the exercise under several different utility function parameter values, chosen with the intention to cover all reasonable levels.

The bilinear and S-shaped utility functions are our main interest in this study. Using these, it has been shown in a hedge fund setting that FSO yields portfolio weights that differ substantially from those of MV optimization (Cremers et al., 2005; Adler and Kritzman, 2007). We seek to test whether this difference holds in a portfolio selection problem of equity indices. We also include the traditional investor preferences of exponential and power utility. Utility maximization using these functions has repeatedly been shown to differ only marginally from quadratic utility (Levy and Markowitz, 1979; Markowitz, 1987; Cremers et al., 2005), and we include them for the purpose of illustration. Gourieroux and Monfort (2005) apply the FSO model to the exponential and power utility functions, which allows them to derive the asymptotic properties of the FSO estimator. They establish in this context that the utility-maximizing estimator yields greater robustness than the MV counterpart, as no information in the return distribution is ignored.

The range of utility parameters tested is given in Table 4. For the exponential utility function, the only parameter to vary is the level of risk aversion (A), which we vary between 0.5 and 6. The γ parameter in the power utility function determines the level of risk aversion and how risk aversion decreases with wealth. As we let it vary between 1 and 5, we include the special case when the power utility function is logarithmic, which happens when γ is one. The higher γ and A are, the higher is the risk aversion. For the bilinear utility function, we vary the critical point (the kink, x, varied from −4 per cent to +0.5 per cent) under which returns are given a disproportionate bad utility. We also vary the magnitude, P, of this disproportion from 1 to 10. In the S-shaped utility function there are five parameters to vary. We test three levels for the inflection point, z: 0 per cent, −2.5 per cent and −5 per cent. The parameters γ1 and A respectively determine the shape and magnitude of the downside of the function, whereas γ2 and B determine the upside characteristics in the same way. The disproportion between gains and losses can be determined either by the γ parameters or by the A and B parameters, or both. We perform one set of tests where the γs vary (γ2 ≥ γ1) and the magnitude parameters are held constant and equal, and one set of tests where the γs are constant and equal, but where A and B varies (A ≥ B).

Table 4. 
Utility Function Parameters
Utility functionParameter values
Exponential0.5 ≤ A ≤ 6
Power1 ≤ γ ≤ 5
Bilinear−4% ≤ x ≤ 0.5%; 1 ≤ P ≤ 10
S-shaped−5% ≤ z ≤ 0%; 0.05 ≤ γ1 ≤ 0.5; 0.95 ≤ γ2 ≤ 0.5; A = 1.5; B = 1.5
−5% ≤ z ≤ 0%; γ1 = 0.5; γ2 = 0.5; 1.5 ≤ A ≤ 2.9, 1.5 ≤ B ≤ 0.1

3.4 Model Comparison

The methodology for comparing FSO with the MV approach is to a large extent inspired by that applied by Cremers et al. (2005) and Adler and Kritzman (2007), where the performance of the different approaches is measured in utility.

In order to compare the FSO and MV optima, the resulting total FSO portfolio return is calculated (inline image). The MV optimal portfolio, θMV, is then the variance-minimizing allocation that yields the same expected portfolio return, a point on the MV efficiency frontier.15 From each solution, a return distribution over time is calculated (θRt). By inserting each of these in the utility function applied for the FSO method, a measure of the MV approximation error, εMV, can be calculated as in equation (4):16

image(4)

For the bilinear and S-shaped utility functions, we also calculate success rates of the same type as in Cremers et al. (2005). These are the fraction of all points in time that yield portfolio returns superior to the investor's specified critical level (kink and inflection point, respectively).

Measuring the difference in terms of improved utility has the drawback that utility is hard to interpret in economic terms. Also, different utility functions yield different magnitudes of utility variation for a set of returns. An alternative measure, which is more straightforward to interpret, is certainty equivalents.17 The certainty equivalent of a specific risky investment in terms of return can be defined as the certain return that would render the investor the same level of utility as the uncertain return of the risky investment. In other words, the certainty equivalent, rCE, is the solution to the following equation:

image(5)

where E denotes expected value. Provided that the utility function is one-to-one the solution to this equation exists and is unique:

image(6)

For the utility functions presented above we are therefore able to compute the unique return rCE that satisfies equation (5) given the portfolio returns rp corresponding to the optimal allocation θ. Using the same approximation as in the FSO, the right-hand side of equation (5) can be evaluated. Expected utility at the optimal weights θFSO is denoted ū, and is calculated as in equation (7):

image(7)

where T is the number of time periods (which differs in- and out-of-sample). Now, equation (5) is equivalent to the expression.

image(8)

which is simple to solve for rCE. The solution for each utility function is presented in Table 5 (the rightmost column).

Table 5. 
Certainty Equivalent Equations
Utility functionCertainty equivalent
  1. Notes:aUnder bilinear utility when ū < u(x), no explicit solution exists. This case is handled by a standard search algorithm.

Exponential−exp[−A(1 + rp)]inline image
Powerinline imagefor γ > 1 ⇔ inline image
ln(1 + rp)for γ = 1 ⇔ rCE = exp(ū) − 1
Bilinearln(1 + rp)for rp ≥ x ⇔ rCE = exp(ū) − 1
P(rp − x) + ln(1 + x)for rp < x ⇔ a
S-shapedinline imagefor rp ≤ z ⇔ inline image
inline imagefor rp > z ⇔ inline image

When we are evaluating the results of the empirical application in the next section, we are mainly interested in the difference in certainty equivalents between the two portfolio selection techniques. Hence, we derive the following measure, showing the difference in rCE between FSO and MV:

image(9)

The certainty equivalent difference, ΔCE, is easy to interpret. It is the certain return on investment corresponding to the increase in utility (under the utility function in question).

3.5 Out-of-sample Testing

In order to further examine the robustness of the FSO methodology, we repeat the procedure described above in an out-of-sample setting. This is done using essentially the same methodology as in Adler and Kritzman (2007). For this purpose, the sample is split into two halves. The first half is used for estimating optimal portfolio allocations, and the performance of the optimal portfolio retrieved is measured on the second half.

We generate 10,000 samples of cross-sectional monthly returns by drawing from the second half of our sample (with replacement), which was not used for the portfolio optimization. This bootstrapping procedure allows us to study performance of the optimized portfolio out-of-sample.

The average utility difference between FSO and MV portfolios is calculated. The exercise is repeated using the second half of the sample for estimation and the first half for the diagnostics.

4 Results

The portfolio selection problem described was repeated 103 times using different utility function specifications. There were 12 tests with exponential utility, nine tests with power utility, 31 tests with bilinear utility and 51 tests with S-shaped utility. All results are presented in the Appendix. Below the test results are presented and interpreted for each utility function type separately. The section is concluded with some general observations.

4.1 Traditional Utility Functions

The tests performed with exponential utility and differing levels of risk aversion yielded, as expected, portfolios with high weights to risky assets when A was low and less so as the risk aversion parameter A was set higher (see Table A1). The portfolios identified were all very close to the MV frontier, resulting in extremely small utility improvements, if any, when using FSO instead of MV. As the FSO and MV solutions were close to identical, the differences in performance out-of-sample were close to non-existent.

Portfolio allocations based on power utility functions were selected with nine different levels of γ, implying different levels of relative risk aversion. As shown in Table A2, the power utility function yielded portfolios allocating the whole investment to the most risky asset (FTSE EMI) for γ ≤ 1.5, and then gradually leaned more towards the medium-risky asset (FTSE250) as γ grew. The deviation from the MV frontier was slightly bigger than in the exponential utility cases, but differences in utility outcome both in-sample and out-of-sample were still minute. In-sample utility improvement never exceeded 0.02 per cent, and out-of-sample differences displayed neither substantial magnitude, nor consistent directions on deviations from zero.

The improvement in terms of certainty equivalents is zero or close to zero under both exponential and power utility functions. Hence, an investor following these utility functions is not prepared to pay anything to move from MV to FSO. These results conform well to those of earlier assessments (Cremers et al., 2005; Adler and Kritzman, 2007), showing that portfolio allocations chosen by the utility-maximizing approaches constitute very small improvements relative to the MV approach, when the investor's utility is well described by the exponential or the power utility functions.

4.2 Bilinear Utility Functions

The tests on bilinear utility functions (i.e. linear splines) were performed with the kink (x) at different levels and various penalties (P) on sub-kink returns. Each kink value was tested for three different penalty levels. As was shown in Table 1, the disutility of sub-kink returns is amplified by the factor P. When P = 1, there is no kink, and the utility function features neither loss aversion nor risk aversion.

The results retrieved for bilinear utility functions are shown in Table A3. As expected, the risk level of the optimal portfolios retrieved with FSO decreased as the kink and penalty parameters increased. When the penalty parameter was set to 1 all wealth was allocated to the most risky asset (FTSE EMI), which is due to the lack of risk aversion in this case. For higher penalty levels, the portfolios were increasingly weighted towards the less risky assets (FTSE100 and FTSE250). At P = 10, no allocations were made to the most risky asset. This occurs when the incentive to avoid returns less than those associated with the kink dominates other investor incentives, such as maximizing returns or minimizing risk by diversification. Portfolio diversification was the highest at P = 5.

Whereas many of the portfolios optimized under bilinear utility were close to the MV frontier, there were cases where the utility improvement was substantial (up to 4 per cent in-sample and 13 per cent out-of-sample). Looking at certainty equivalents, there was a positive, fairly consistent, but small difference in favor of the FSO portfolios. On average, certainty equivalent improvement amounted to 0.02 per cent in-sample and 0.04 per cent out-of-sample (see bottom of Table A3). This means that usage of FSO rather than MV when investor preferences are correctly described with a bilinear utility function yields a utility improvement equivalent to that of a certain 0.02 per cent annual return. Of the 20,000 draws made for the out-of-sample test, 14 per cent yielded certainty equivalent improvements higher than 1 per cent, and 8 per cent yielded less than −1 per cent (in annual terms).

Looking at the distribution of returns out-of-sample, there were on average (of the 20,000 bootstrapped samples) no differences between FSO and MV in mean and variance. The solutions under bilinear utility, however, yielded higher skewness and higher kurtosis than the MV solutions did on average (as shown in Table A6). Bilinear utility does not punish kurtosis, as there is no decrease in marginal utility with returns (i.e. no risk aversion) except for the jump at the kink. Accordingly, it can be seen that the corresponding MV portfolios are more diversified than the FSO portfolios.

The success rates (ratio of returns exceeding the kink) were slightly higher on average using FSO to implement the bilinear preferences than for the MV portfolios, which can be related to the higher skewness resulting under bilinear utility (see bottom of Table A3). Differences appeared only in 10 cases, out of which two were in favor of MV. Success rates implemented by Cremers et al. (2005) yielded similarly small differences under the bilinear utility specification.

4.3 S-shaped Utility Functions

As discussed in Section 2.2, the S-shaped utility function features risk-loving behavior when returns are below a critical value z, and risk aversion when returns are above that value. We performed our tests with the inflection point set to zero, −2.5 per cent and −5 per cent, with varying settings of either the γ values (γ1 and γ2) or the magnitude parameters (A and B). These parameters regulate the curvature of the S shape. Exact specifications of parameters are given in the first five columns in Tables A4 and A5, followed by the results to the right in the tables.

The utility from the FSO approach under S-shaped utility was considerably better than the utility obtained with S-shaped preferences when evaluating allocations chosen via the MV approach. The average utility difference was 10 per cent in-sample and 15 per cent out-of-sample. Certainty equivalent improvements averaged 0.2 per cent both in-sample and out-of-sample (10 times higher than under bilinear utility). That is, using FSO causes an increase in utility corresponding to a certain annual return of 0.2 per cent on average—a substantial gain. This certainty equivalent was very consistent throughout the 20,000 out-of-sample draws. It exceeded 1 per cent in 35 per cent of the cases, whereas it was lower than −1 per cent in only 15 per cent of the tests.

Return distributions on average of the 20,000 bootstrapped samples have higher means and variances when optimizing with FSO than with MV (see Table A6). As for the bilinear utility case, skewness and kurtosis are also increased, the former with much higher magnitude than under the bilinear preferences (0.069). In other words, the portfolios based on S-shaped preferences have a higher risk level but a lower probability of very low returns on a single day. The success rates are also clearly in favor of the FSO approach in cases where S-shaped utility is believed to describe the investor's preferences well, which was also found by Cremers et al. (2005). In 60 per cent of the utility specifications the FSO success rate is higher than the MV counterpart—another reflection of the increased skewness of the FSO return distribution.

The variation of γ proportions causes only minor changes in the allocations. The γ primarily determines the bends of the S shape, and the influence on allocations is apparently marginal. The variation of A and B, on the other hand, is more influential on allocations. The higher the ratio A/B gets, the less risk is chosen for the portfolio. Our results also indicate that, as expected, higher inflection points correspond to higher loss aversion.

5 Conclusions

The empirical application of this study constitutes the widest FSO–MV comparison to date with respect to utility functions. The results extend earlier findings by Cremers et al. (2005) and Adler and Kritzman (2007), establishing the robustness of those studies' results. The FSO methodology is useful when investor utility function features a threshold, such as in the bilinear and the S-shaped utility functions (especially the latter). For traditional utility functions (exponential and power utility) there is no clear performance difference between MV and FSO.

The fact that these results appear in an application of equity indices increases the scope of the FSO applicability considerably. It has earlier only been shown that FSO is useful in allocation problems involving hedge funds, which have very non-normal return distributions. Using assets with return distributions closer to normality yields smaller gains in utility, but the improvements are still substantial.

FSO has great theoretical appeal in that it does not build on assumptions simplifying the world of the investor. Return distributions are used in their entirety and utility functions can be chosen with complete flexibility, without mathematical convenience considerations. Challenges remaining for the investment advisor include correctly specifying the investor's preferences in a utility function, and overcoming the computational burden of the technique. These issues should be subject to future research. Studies similar to the current one with other asset selection problems, such as selection between different asset classes or derivative selection, would also be useful.

Appendix

Full Results

Here the complete set of results from the empirical application for each utility function is presented. Each table is constructed in the following fashion: Utility function parameters; Optimal portfolio allocations under FSO; Optimal portfolio allocations under MV; Utility improvement; Certainty equivalent improvement. In the latter two categories, (IS) contains in-sample results and (OOS) out-of-sample results. Certainty equivalents improvements are given in annual terms. In Tables A3–A5, success rates columns are added, the first one containing FSO results and the second MV results (both in-sample for the full data set). Portfolio allocations notation are as follows: θ1 is allocation to FTSE100, θ2 is to FTSE250 and θ3 is to FTSE EMI. The given allocations are those estimated on the full data set. Allocations estimated on the first and second half of the data set respectively are available upon request. Table A6 gives portfolio return distributional properties averaged over the 20,000 bootstrapped samples, for FSO and MV allocations, respectively.

Table A1. 
Exponential Utility Results
AFSO approachMV approachεMVΔCE
θ1θ2θ3θ1θ2θ3ISOOSISOOS
0.50.0000.0001.0000.0000.0010.9990.0%0.0%0.1%0.0%
10.0000.0001.0000.0000.0010.9990.0%0.0%0.1%0.0%
1.50.0000.0001.0000.0000.0010.9990.0%0.0%0.1%0.0%
20.0000.2300.7700.0000.2300.7700.0%0.0%0.0%0.0%
2.50.0000.4000.6000.0000.4000.6000.0%0.0%0.0%0.0%
30.0000.5150.4850.0000.5150.4850.0%0.0%0.0%0.0%
3.50.0000.5950.4050.0000.5950.4050.0%0.0%0.0%0.0%
40.0000.6550.3450.0000.6550.3450.0%0.0%0.0%0.0%
4.50.0000.7050.2950.0000.7050.2950.0%0.0%0.0%0.0%
50.0000.7400.2600.0000.7400.2600.0%0.0%0.0%0.0%
5.50.0000.7700.2300.0000.7700.2300.0%0.0%0.0%0.0%
60.0000.7950.2050.0000.7950.2050.0%0.0%0.0%0.0%
Table A2. 
Power Utility Results
γFSO approachMV approachεMVΔCE
θ1θ2θ3θ1θ2θ3ISOOSISOOS
10.0000.0001.0000.0000.0010.9990.0%0.1%0.1%0.0%
1.50.0000.0001.0000.0000.0010.9990.0%−0.1%0.1%0.0%
20.0000.2200.7800.0000.2200.7800.0%−0.2%0.0%0.0%
2.50.0000.3950.6050.0000.3950.6050.0%0.1%0.0%0.0%
30.0000.5100.4900.0000.5100.4900.0%0.1%0.0%0.0%
3.50.0000.5900.4100.0000.5900.4100.0%0.0%0.0%0.0%
40.0000.6500.3500.0000.6500.3500.0%0.2%0.0%0.0%
4.50.0000.7000.3000.0000.7000.3000.0%0.0%0.0%0.0%
50.0000.7350.2650.0000.7350.2650.0%−0.1%0.0%0.0%
Table A3. 
Bilinear Utility Results
ParametersFSO approachMV approachεMVΔCESuccess rates
Pxθ1θ2θ3θ1θ2θ3ISOOSISOOSFSOMV
10.0000.0001.0000.0000.0010.9990.0%−0.1%0.1%0.0%
2.5−0.040.0000.7450.2550.0000.7450.2550.0%0.0%0.0%0.0%87.4%87.4%
2.5−0.0350.0000.7600.2400.0000.7600.2400.0%0.0%0.0%0.0%84.2%84.2%
2.5−0.030.0000.8600.1400.0000.8600.1400.0%−0.4%0.0%0.0%80.0%80.0%
2.5−0.0250.0000.9550.0450.0100.9120.0782.2%−2.5%0.0%0.0%76.8%75.8%
2.5−0.020.0001.0000.0000.0230.9020.0754.1%−0.2%0.1%0.0%75.8%74.7%
2.5−0.0150.0001.0000.0000.0230.9020.0753.1%−0.3%0.1%0.0%73.7%73.7%
2.5−0.010.0001.0000.0000.0230.9020.0752.5%10.2%0.1%0.4%69.5%69.5%
2.5−0.0050.0001.0000.0000.0230.9020.0751.7%5.7%0.1%0.2%67.4%67.4%
2.500.0001.0000.0000.0230.9020.0751.3%2.6%0.1%0.1%63.2%64.2%
2.50.0050.0001.0000.0000.0230.9020.0750.7%12.9%0.1%0.2%60.0%59.0%
5−0.040.0450.8050.1500.0230.9020.0751.5%0.1%0.1%−0.2%88.4%86.3%
5−0.0350.0300.8050.1650.0050.9160.0800.6%1.0%0.1%−0.1%84.2%83.2%
5−0.030.1600.7550.0850.1480.8070.0450.4%−1.5%0.0%−0.1%84.2%83.2%
5−0.0250.2250.7750.0000.2330.7420.0250.4%2.3%0.0%0.2%81.1%79.0%
5−0.020.3200.6800.0000.3210.6750.0040.1%7.2%0.0%0.2%76.8%76.8%
5−0.0150.4000.6000.0000.4000.6000.0000.0%4.4%0.0%0.1%73.7%73.7%
5−0.010.3350.6650.0000.3350.6640.0010.0%6.1%0.0%0.1%68.4%68.4%
5−0.0050.3050.6950.0000.3070.6860.0070.1%8.6%0.0%0.0%67.4%67.4%
500.2100.7900.0000.2190.7530.0280.3%8.5%0.0%0.1%65.3%66.3%
50.0050.1900.8100.0000.2000.7670.0330.6%12.9%0.0%0.1%59.0%57.9%
10−0.040.5950.4050.0000.5950.4050.0000.0%0.0%0.0%0.0%88.4%88.4%
10−0.0350.4500.5500.0000.4500.5500.0000.0%−0.1%0.0%0.0%85.3%85.3%
10−0.030.5200.4800.0000.5200.4800.0000.0%−0.1%0.0%0.0%83.2%83.2%
10−0.0250.5000.5000.0000.5000.5000.0000.0%−0.1%0.0%0.0%80.0%80.0%
10−0.020.5650.4350.0000.5650.4350.0000.0%5.1%0.0%0.1%77.9%77.9%
10−0.0150.5100.4900.0000.5100.4900.0000.0%1.1%0.0%0.0%72.6%72.6%
10−0.010.5100.4900.0000.5100.4900.0000.0%0.0%0.0%0.0%70.5%70.5%
10−0.0050.6000.4000.0000.6000.4000.0000.0%0.0%0.0%0.0%65.3%65.3%
1000.5900.4100.0000.5900.4100.0000.0%0.0%0.0%0.0%63.2%63.2%
100.0050.4850.5150.0000.4850.5150.0000.0%1.5%0.0%0.1%59.0%59.0%
Averages0.6%2.7%0.02%0.04%74.4%74.1%
Table A4. 
S-shaped Utility Results, First Half
ParametersFSO approachMV approachεMVΔCESuccess rates
γ1γ2ABzθ1θ2θ3θ1θ2θ3ISOOSISOOSFSOMV
0.050.951.51.500.0900.9050.0050.1050.8400.0565.3%0.2%0.1%0.0%66.3%64.2%
0.10.91.51.500.0900.9050.0050.1050.8400.0564.6%1.8%0.1%0.0%66.3%64.2%
0.150.851.51.500.0900.9100.0000.1070.8380.0554.0%1.8%0.1%0.0%66.3%64.2%
0.20.81.51.500.0900.9100.0000.1070.8380.0553.6%3.2%0.1%0.0%66.3%64.2%
0.250.751.51.500.0900.9100.0000.1070.8380.0553.4%5.4%0.1%0.0%66.3%64.2%
0.30.71.51.500.0900.9050.0050.1050.8400.0563.6%20.6%0.1%0.1%66.3%64.2%
0.350.651.51.500.0900.9050.0050.1050.8400.0564.8%11.2%0.1%0.1%66.3%64.2%
0.40.61.51.500.0000.7350.2650.0000.7350.2650.0%5.9%0.0%0.1%65.3%65.3%
0.450.551.51.500.0000.7350.2650.0000.7350.2650.0%0.0%0.0%0.0%65.3%65.3%
0.50.51.51.500.0000.7300.2700.0000.7300.2700.0%0.0%0.0%0.0%65.3%65.3%
0.50.51.71.300.0000.7350.2650.0000.7350.2650.0%0.5%0.0%0.1%65.3%65.3%
0.50.51.91.100.0900.9100.0000.1070.8380.05530.2%−3.4%0.1%0.1%66.3%64.2%
0.50.52.10.900.0900.9100.0000.1070.8380.0554.7%11.7%0.1%0.1%66.3%64.2%
0.50.52.30.700.0950.9050.0000.1110.8350.0542.2%12.2%0.1%0.0%66.3%64.2%
0.50.52.50.500.1300.8700.0000.1440.8100.0460.5%5.4%0.0%0.0%66.3%66.3%
0.50.52.70.300.2050.7950.0000.2140.7570.0300.3%0.7%0.0%0.0%66.3%66.3%
0.50.52.90.100.2050.7950.0000.2140.7570.0300.2%0.3%0.0%0.0%66.3%66.3%
0.050.951.51.5−0.0250.2950.7000.0050.2960.6940.0106.4%35.8%0.0%0.8%83.2%82.1%
0.10.91.51.5−0.0250.2950.7050.0000.2980.6930.0105.8%194.6%0.0%1.4%83.2%82.1%
0.150.851.51.5−0.0250.2950.7050.0000.2980.6930.0106.0%228.3%0.0%1.8%83.2%82.1%
0.20.81.51.5−0.0250.2950.7050.0000.2980.6930.01010.1%65.4%0.0%1.7%83.2%82.1%
0.250.751.51.5−0.0250.2950.7050.0000.2980.6930.01011.5%20.7%0.0%1.4%83.2%82.1%
0.30.71.51.5−0.0250.2950.7050.0000.2980.6930.0102.1%14.3%0.0%1.1%83.2%82.1%
0.350.651.51.5−0.0250.2950.7050.0000.2980.6930.0100.9%24.8%0.0%0.8%83.2%82.1%
0.40.61.51.5−0.0250.2950.7050.0000.2980.6930.0100.6%10.4%0.0%0.7%83.2%82.1%
0.450.551.51.5−0.0250.0250.9750.0000.0460.8850.0701.9%2.0%0.1%0.7%79.0%76.8%
Table A5. 
S-shaped Utility Results, Second Half
ParametersFSO approachMV approachεMVΔCESuccess rates
γ1γ2ABzθ1θ2θ3θ1θ2θ3ISOOSISOOSFSOMV
0.50.51.51.5−0.0250.0250.9750.0000.0460.8850.0701.4%1.1%0.1%0.6%79.0%76.8%
0.50.51.71.3−0.0250.2950.7050.0000.2980.6930.0100.5%1.5%0.0%0.6%83.2%82.1%
0.50.51.91.1−0.0250.2950.7050.0000.2980.6930.0100.6%3.3%0.0%0.6%83.2%82.1%
0.50.52.10.9−0.0250.2950.7050.0000.2980.6930.0100.7%10.2%0.0%0.6%83.2%82.1%
0.50.52.30.7−0.0250.2950.7050.0000.2980.6930.0101.2%21.8%0.0%0.7%83.2%82.1%
0.50.52.50.5−0.0250.2950.7050.0000.2980.6930.01010.8%6.9%0.0%0.7%83.2%82.1%
0.50.52.70.3−0.0250.3000.7000.0000.3030.6890.0080.9%18.6%0.0%0.7%83.2%82.1%
0.50.52.90.1−0.0250.3650.6350.0000.3650.6350.0000.0%20.8%0.0%0.0%83.2%83.2%
0.050.951.51.5−0.050.1150.5000.3850.0220.9020.07568.9%0.0%1.3%0.0%90.5%88.4%
0.10.91.51.5−0.050.1150.5000.3850.0220.9020.075270.5%−4.0%1.3%−0.2%90.5%88.4%
0.150.851.51.5−0.050.1150.5000.3850.0220.9020.07526.8%−1.3%1.3%−0.4%90.5%88.4%
0.20.81.51.5−0.050.1150.5000.3850.0220.9020.07510.0%−0.7%1.3%−0.3%90.5%88.4%
0.250.751.51.5−0.050.1350.5200.3450.0520.8800.0684.8%−0.4%1.0%−0.3%90.5%88.4%
0.30.71.51.5−0.050.1500.5350.3150.0750.8630.0632.4%−0.2%0.9%−0.1%90.5%88.4%
0.350.651.51.5−0.050.0000.8500.1500.0000.8500.1500.0%−8.0%0.0%−2.0%89.5%89.5%
0.40.61.51.5−0.050.0000.8350.1650.0000.8350.1650.0%−3.3%0.0%−1.1%89.5%89.5%
0.450.551.51.5−0.050.0000.8250.1750.0000.8250.1750.0%−2.1%0.0%−1.1%89.5%89.5%
0.50.51.51.5−0.050.0000.8150.1850.0000.8150.1850.0%−1.4%0.0%−0.9%89.5%89.5%
0.50.51.71.3−0.050.0000.8300.1700.0000.8300.1700.0%−0.9%0.0%−0.4%89.5%89.5%
0.50.51.91.1−0.050.0000.8400.1600.0000.8400.1600.0%−0.7%0.0%−0.2%89.5%89.5%
0.50.52.10.9−0.050.2150.5950.1900.1700.7900.0400.9%−0.1%0.3%−0.1%90.5%88.4%
0.50.52.30.7−0.050.2300.5950.1750.1880.7760.0362.0%2.6%0.2%−0.1%90.5%87.4%
0.50.52.50.5−0.050.8750.1250.0000.8750.1250.0000.0%3.1%0.0%0.1%90.5%90.5%
0.50.52.70.3−0.050.8800.1200.0000.8800.1200.0000.0%0.0%0.0%0.0%90.5%90.5%
0.50.52.90.1−0.050.8800.1200.0000.8800.1200.0000.0%0.0%0.0%0.0%90.5%90.5%
Averages10.1%14.5%0.2%0.2%79.6%78.5%
Table A6. 
Average Out-of-sample Differences in Distributional Properties between FSO and MV Solution under Bilinear and S-shaped Utility Functions
Utility functionMeanVarianceSkewnessKurtosis
Bilinear0.00000.00000.00290.0180
S-shaped0.00040.00010.06870.0150

Footnotes

  • 1

    Samuelson's article ‘When and Why Mean–Variance Analysis Generically Fails’ was never published, but the term FSO was picked up by Cremers et al. (2005).

  • 2

    In the MV model, risk is defined in terms of the second moment (variances and covariances) only. This makes the model simple to apply, but it is based on assumptions that are clearly unrealistic. For the MV solution to be optimal, either the return distribution must feature spherical symmetry, or investors must be indifferent to higher moments and equally averse to downside and upside risk (quadratic utility). Financial returns are rarely spherical (first pointed out by Mandelbrot, 1963), and the quadratic utility is known to be decreasing in wealth above a certain wealth level.

  • 3

    When using a grid search, p is chosen at a level suitable to the problem nature, yielding a finite dimension m of the allocation matrix. Search algorithms do not always specify p, making m infinitely large. Then a halting criterion is used to stop the search at suitable precision.

  • 4

    The term (rp − µ1)U′(µ1), where rp is the portfolio return, disappears when taking expectations, as E(θR − µ1) = 0.

  • 5

    As referred to above, the MV approach is based on assuming either quadratic utility or a normal return distribution. The quadratic utility function is expressed as E(U) = µ1 − λµ2. This implies U1(µ1) = 1, U2(µ1) < 0 (usually referred to as the risk aversion parameter, λ) and Uk(µ1) = 0 for all k > 2. If normally distributed returns are assumed, all odd moments (k = 3, 5, . . .) will be zero, and all even moments will be functions of the variance (see the Appendix to Chapter 1 in Cuthbertson and Nitzsche, 2004).

  • 6

    Co-skewness is a phenomenon extensively discussed by Harvey et al. (2003).

  • 7

    The number of possible solutions (m) is
    inline image
    where n is the number of assets and p the precision of the grid. This is the formula for combinations of discrete numbers with repetition, derived by Leonhard Euler (1707–83). See, for example, Epp (2003).

  • 8

    In Cremers et al. (2005) and Adler and Kritzman (2007), a search algorithm is applied to find the FSO optimum. Such algorithms are necessary when using larger numbers of assets. Cremers et al. consider 61 assets in their application, and use a precision of 0.1 per cent and do not allow for short selling. This implies m = 7.23 × 1098, which is the number of vectors to be evaluated over their 10 annual observations. They do not disclose their search algorithm. Cremers et al. (2003, 2005), Gourieroux and Monfort (2005) and Adler and Kritzman (2007) all argue that the computational burden of the FSO technique has become obsolete with the ample computational power on hand nowadays. To our knowledge, however, there is no study verifying this.

  • 9

    This is the range needed to cover all solutions not covered in the previous grid search, as if two assets change by 0.5 per cent in the same direction, the third has to change by 1 per cent. In general the range required for the second grid search is p(n − 1), where n is the number of assets in the problem and p is the precision of the first grid search.

  • 10

    In portfolio choice problems with large grids resulting from a larger number of assets, the two-stage technique presented here may be a viable option to decrease computational cost. Caution is needed, however, as non-convexity may lead to the optimum identified with an imprecise grid not being in the area of the global optimum.

  • 11

    The Datastream codes for the indices are FT100GR(PI), FT250GR(PI) and AWALEG£(PI).

  • 12

    For an exact definition, see http://www.ftse.com/Indices/FTSE_Emerging_Markets/Downloads/FTSE_Emerging_Market_Indices.pdf.

  • 13

    The Jarque–Bera test is appropriate for serially uncorrelated data (such as white-noise regression residuals) but inadequate for temporally dependent data such as certain financial returns. We do not pursue this further; the interested reader is referred to Bai and Ng (2005).

  • 14

    The difference between the average return of all three indices and the return corresponding to the variance of the equally weighted portfolio is added to each observation. In this case, the difference added amounts to 0.086 per cent.

  • 15

    We use the Markowitz MV model from 1952 as benchmark in this study. In this way, it can be established whether the FSO model is superior to that model. The rich supply of MV extensions, however, is yet to be compared with the FSO model.

  • 16

    The MV solution can be calculated with much higher detail than the FSO portfolio, which is limited to 0.5 per cent precision. Accordingly, a minor source of utility difference will be due to this limitation, which in the comparison is to the MV method's advantage.

  • 17

    Certainty equivalents have earlier been used for FSO–MV comparisons in a working paper by Cremers et al. (2003), which however was never published.

Ancillary