A Forecast Evaluation of the Riksbank's Policy‐Rate Projections

This paper evaluates the forecasting performance of the policy&#8208;rate path published by the Swedish central bank, the Riksbank. Using data from 2007 to 2019, I find that the Riksbank's forecast has been relatively inaccurate compared to a forecast inferred from market pricing. My analysis indicates that this result is primarily driven by events during the period 2010&#8211;2014. This coincides with a period during which the Riksbank arguably &#8220;leaned against the wind&#8221; and a potential link is discussed in the paper.

detrimental, that there is a risk that the path might be misinterpreted as a commitment and that it would be difficult for members of the board to agree on a policy rate path (see, e.g., Goodhart, 2009;Gosselin, Lotz, & Wyplosz, 2008;Mishkin 2004;Morris & Shin, 2002;Rudebusch & Williams, 2008). Proponents argue that publishing paths give the central bank better control over long-term interest rates, making it easier to achieve macroeconomic goals (see, e.g., Bernanke, 2013;Blinder, Ehrmann, Fratzscher, De Haan, & Jansen, 2008;Eusepi & Preston, 2010;Svensson, 2006;Woodford, 2005). Svensson (2002) further argues that publishing the path conveys important information to the market, particularly since optimal policy should target the future inflation rate. To publish a policy-rate path can also be in line with Faust and Leeper (2005), who conclude that unconditional forecasts provide more effective communication than conditional ones.
Sweden provides an interesting case in this debate, particularly for the period 2010-2014. During this period, there was disagreement on the board of directors, which in itself is an argument against publishing a policy rate path as it might reduce the credibility of the forecast. The disagreement was that one of the board members, Lars E. O. Svensson, claimed that the Riksbank was "leaning against the wind" (Financial Times, 2014; Svensson, 2014Svensson, , 2017. Leaning against the wind implies setting the policy rate above what would be considered appropriate based on inflation and output stability. The incentive for such a policy in Sweden at that time was to counter increasing household debt levels, mainly driven by rapidly rising housing prices. If the Riksbank was attempting to affect housing prices, the policy rate path might be a suitable instrument to use. According to proponents of publishing policy rate paths, it is particularly useful for long-term interest rates, which should be the most relevant rates for housing. How big impact leaning against the wind had on policy in Sweden is debated (Jansson, 2014). While it was not official policy, housing prices were used as an argument during board meetings and members on the board of directors have said that it did play a role (Goodfriend & King, 2016;Jansson, 2017).
In evaluating the Riksbank's policy rate path, this paper follows established literature. Typically, this literature either evaluates the accuracy of the central bank's forecasts (Beechey & Österholm, 2014a;Goodhart & Lim, 2011), investigate if the path has affected market forecasts (Ehrmann, Eijffinger, & Fratzscher, 2012;Kool & Thornton, 2012), or investigate the effect on the contemporaneous yield curve (Brubakk, ter Ellen, & Xu, 2017;Ferrero & Secchi, 2009). Though there might not be a consensus, the results tend to indicate that central bank forecasts are not very precise for longer horizons but that they are on par with other forecasts. However, Goodfriend and King (2016) note that the forecast accuracy of the Riksbank seems to fall short of the accuracy of a forecast implicit in market prices over the period [2010][2011][2012][2013][2014][2015]. A response to this criticism was published by Sveriges Riksbank (2017).
Although there is little formal analysis in Goodfriend and King (2016), their finding is interesting as it contradicts the previous result of Beechey and Österholm (2014a;BÖ henceforth), who find no significant differences in forecast accuracy between the Riksbank and the market. However, BÖ analyses the period 2007-2012. Unlike the study by Goodfriend and King (2016), this does not cover the period when the Riksbank leaning against the wind. Also, neither of the papers covers any extended period during which the Riksbank was faced with the zero lower bound. To formally address the accuracy of the paths published by the Riksbank's during these relevant periods, this paper extends the analysis of BÖ to 2019. Regarding forecast efficiency and unbiasedness, my findings are in line with previous results, as no studied forecast is efficient or unbiased. Regarding relative forecast accuracy, my findings are in line with what is argued in Goodfriend and King (2016). Over the full period, the Riksbank's path has been significantly less accurate than market expectations at the 2 year horizon. To further analyse this result, the analysis of BÖ is extended with regression analysis using a dummy variable for the period 2010-2014. Results from this analysis imply that it is forecast accuracy specifically during this period that explains the difference in the results of Goodfriend and King compared to BÖ. The approach using a dummy variable for the period 2010-2014 relates to the work of Natvik, Rime, and Syrstad (2019), who found that the Riksbank's leaning against the wind caused the policy rate path to carry less information to implicit market forecasts.
The main contribution of this paper is to formally address the forecast accuracy of the Riksbank's policy rate path over a sample which includes a hotly debated period. The results also have potential implications for the debate on the merits of leaning against the wind. This ongoing debate largely focuses on costs versus benefits of such policies (Adrian & Liang, 2016;Filardo & Rungcharoenkitkul, 2016;IMF, 2015;Juselius, Borio, Disyatat, & Drehmann, 2016;Svensson, 2014Svensson, , 2017Sveriges Riksbank, 2014). This paper adds to this debate by instead focusing on the effect on forecasting performance. One interpretation of the result of this paper is that if a central bank uses the policy rate path to lean against the wind, this might come at the cost of poor forecasting performance.
The rest of this paper is organized as follows: Section 2 presents the data and conducts the empirical analysis that evaluates the forecast performance of the Riksbank as well as a forecast implicit in market data. Section 3 discusses the results and Section 4 concludes this paper.

| DATA AND EMPIRICAL ANALYSIS
This paper is set up as a replication and extension of BÖ, whose data set I have appreciatively been allowed to use.
To make comparable analysis, data are appended with information that as closely as possible resembles the original data. Data covers forecasts from February 14, 2007 and outcomes through the first quarter 2019. Following BÖ, the analysis focuses on three different horizons: one quarter, 1 year, and 2 years. This gives 70, 65, and 59 observations on forecast errors at the one-quarter, 1-year, and 2-year horizon, respectively. The Riksbank publishes its forecast of the policy rate in conjunction with board meetings in February, April, July, September,   Figure A1 in the appendix presents the repo rate, Figure A2 the published paths, and Figures A3-A5 the forecast errors.

| Unbiasedness and efficiency
As a first step, I test if the forecasts have been unbiased and efficient. These two stages are standard in the evaluation literature (BÖ; Mankiw, Reis, & Wolfers, 2003;Mehra, 2002). A forecast was biased if the forecasts on average were below or above the realized policy rate. The test is performed by regressing forecast errors on a 2 The Riksbank estimates yield curves using the Nelson-Siegel-Svensson approach of Svensson (1995) rather than that of Nelson and Siegel (1987). Yield curves are estimated on government-issued securities. 3 The naïve forecast is the assumption that the policy rate at the date when the forecast is made will be the policy rate at the forecasted date.
constant. If the constant is significantly different from zero the forecasts have been biased. That is, I run the where ε t is the error term from the regression. The null hypothesis α = H : 0 0 is tested using a standard t test, where Newey-West standard errors are employed to address serial correlation in the residuals. As is evident from Table 1, both the Riksbank and the implicit market forecasts are biased at the 2-year horizon (based on the 1% significance level). The Riksbank forecast is also biased at the 1-year horizon (if a 10% significance level is employed). Given rational expectations and a standard quadratic loss function, we would expect the bias to be zero.
But perhaps this period was particularly difficult to forecast, as argued by Sveriges Riksbank (2017). BÖ suggests that the financial crisis and the euro debt crisis might cause a similar result in the shorter data set. Another possibility is that loss functions are not quadratic.
Regardless of the form of the loss function, rational expectations imply that all available information is used when forming expectations. One straightforward way to investigate if forecasts are efficient in this sense is to see if the forecast error is systematically related to information available at the time when the forecast was made. Here, this is tested by regressing the forecast error on the policy rate where r t is the policy rate that was known by the market and the Riksbank when the forecasts were made, | + e t h t is defined as above and ω t is the error term from the regression. The null hypothesis of no systematic relation δ = H : 0 0 is tested using a standard t test, where again Newey-West standard errors are used. The results are given in Table 1. The null hypothesis is rejected for both the Riksbank and the implicit market forecast at the 1-and 2- year horizons (based on the 1% significance level). For the Riksbank, the null hypothesis is rejected even at the onequarter horizon (if the 10% significance level is used). This implies that neither the market nor the Riksbank used
the available information efficiently. 4 These results are in line with BÖ who concluded that rational expectations might be too though a benchmark for forecaster who has to act in a world of incomplete information. 5

| Forecast accuracy comparison
Inspecting the root mean squared forecast errors (RMSEs) in Table 1, it appears as if the Riksbank outperforms both the implicit market forecast and the naïve forecast at the one-quarter horizon. As could be expected, the errors are small at the shortest horizon, 0.29 and 0.30 for the Riksbank and the market, respectively. At the 1-year horizon, the errors are considerably larger and it is instead the implicit market forecast that has the lowest RMSE of 1.07. For the 2-year horizon, both the naïve and the implicit market forecasts have lower RMSEs than the Riksbank.
To assess whether these differences are statistically significant, the modified Diebold-Mariano test of Harvey, Leybourne, and Newbold (1997) is used. The test is conducted under the assumption of quadratic loss functions and is based on the following regression: where η is the estimated difference between the squared forecast errors, υ t is the error term from the regression,  Table 2. As can be expected, the Riksbank forecast tends to perform well at the one-quarter horizon. It is important to keep in mind that, besides the fact that the Riksbank sets the repo rate, the Riksbank forms its forecast knowing the repo rate they will announce. Since the market forecast is from before the new repo rate was announced, this information was not available to the market. 6 According to Svensson (2009), the purpose of presenting a path for the repo rate is to convey the Riksbank's expectation at the longer horizons, typically 2 years. The primary focus is therefore on the longer horizon forecasts.
Interestingly, the implicit market forecast is significantly more accurate at the 2-year horizon (based on the 5% significance level). This result was not present in BÖ, suggesting that something has changed regarding the longest horizon. Figure 1 presents the difference in squared forecast errors over time. That the difference is larger for longer horizons could be because the Riksbank has more direct control over the short horizon, as it determines the policy rate that is in effect up until the next board meeting. Also, longer horizons are less determined by the repo rate today and more open to the board's judgment.
Given the shape of Figure 1 and also considering the proposed leaning against the wind period in the Riks- That δ is negative likely follows from that the forecast errors were negative most of the time (Figures A3-A5) and when the interest rate was negative, forecast errors were small. Also, errors were negatively large around the financial crisis when the interest rate was relatively high. 5 It would have been interesting to, as BÖ, append the analysis with data for the Norwegian case. Unfortunately, the Norwegian central bank has not been willing to provide the relevant market forecast estimates. 6 On a technical note, the very short horizon in a Nelson-Siegel estimation will necessarily follow from the repo rate at the time of the forecast. Considering that the flexibility of the Nelson-Siegel curve is limited, this will cause the market forecast to deviate from rational beliefs in the short end when the policy rate is changed.
where d lean is a dummy variable that takes on the value one for the period when the Riksbank leaned against the wind and zero otherwise. 8 The null hypothesis of no effect of the leaning against the wind period θ = H : 0 0 is tested using a standard t test (based on Newey-West standard errors). Furthermore, if this period does explain the lower accuracy of the Riksbank forecasts, we would expect no difference outside the period and the intercept would be zero. The null hypothesis η = H : 0 0 is also tested using a standard t test (based on Newey-West standard errors). As is evident from the results in Table 3, it is during the leaning against the wind period that the difference at the 2-year horizon forecast occurs. The differences in the squared errors are sizeable and the intercept for the 2-year horizon is not significant when the dummy is introduced. The straightforward interpretation of this is that it is the period February 2010 to July 2014 that is driving the results in Table 2.
Parameters from a regression on squared forecast errors can be hard to interpret. To get a better intuitive understanding of how the leaning against the wind period differs from the rest of the data, Table 4 presents the RMSEs for the whole sample, the leaning against the wind period, and the complement of the leaning against the wind period, respectively. For the 2-year horizon, the RMSEs of the Riksbank stand out as they are almost the same in all periods. For the other two forecasts, the leaning against the wind period clearly stands out, as the RMSEs are considerably smaller during this period.

| Effect of path announcements on market forecast accuracy
The focus of this paper is on the accuracy of the Riksbank's policy rate path. In the literature, this is often seen in conjunction with analysis of how the publication of the path affects the contemporaneous yield curve. Previous studies typically find that there is such an effect and for Sweden, this result has recently been established by T A B L E 3 Regression of difference in squared forecast errors between the Riksbank's path and other forecasters on an intercept and a dummy for the period February 2010-July 2014 Intercept Dummy Electronic copy available at: https://ssrn.com/abstract=3707486 Brubakk et al. (2017) as well as by Natvik et al. (2019). The latter paper also analyses the extent to which the announcements from the Riksbank improves the accuracy of implicit market forecasts. This is done by estimating the difference between the market's forecasting errors in a window around announcements. Interestingly, they find that accuracy improves less during the period 2010-2014. Their interpretation is that this reflects poor communication during a period when the Riksbank was leaning against the wind.
Utilizing the Nelson-Siegel estimates from the Riksbank I perform a similar analysis as in Natvik et al. (2019).
The absolute forecast error based on market data from 1 day before publication of the policy rate is tested using a standard t test (based on Newey-West standard errors). Results are found in Table 5 which also presents estimations without the dummy variable.
The results in Table 5 are in line with those in Natvik et al. (2019). Over most of the sample, the publication of the policy rate path improves the market's forecast accuracy μ ( < ) 0 . This result is statistically significant for all horizons (based on the 10% significance level). But during the period 2010-2014 there is essentially no effect on market accuracy on any horizon μ β ( + ≈ ) 0 .

| Sensitivity analyses
As discussed previously the start date of the leaning against the wind period is not well established. To analyze sensitivity to the start date regression (4) is estimated with start dates varied with 6 months and 1 year before and after February 2010. Hypothesis testing is performed as described before. Results show that the analysis is robust to this variation ( Table 6). The precise start date picked is therefore not crucial for the results. 10 Another concern could be that results associated with the less informative MPU might be driving the results.
To assess if this is the case regression (3) and regression (4) where φ captures how squared forecast errors based on the MPU deviate from others and all other parameters are as described in regressions (3) and (4), respectively. The additional null hypothesis of no effect of MPU θ = H : 0 0 is tested using a standard t test (based on Newey-West standard errors). As can be seen in Table 7 the dummy for the MPU is significantly different from zero in regression (6) for the 2 year horizon (using the 5% significance level).
However, the intercept in the results based on regression (6) is also significantly different from zero (based on the 9 If there is no estimate for the date, the closest preceding/following estimate is used. 10 In the result it is clear that the intercept increases when the start date is picked to be a later date. This is as expected if the initial choice of the period was correct. The intercept represents the mean outside the leaning period, which increases when more dates with high forecast errors are excluded from the leaning period. 5% significance level) which indicates that the MPU is not driving the results overall. This is confirmed by the results based on Equation (7), which suggests that it is the leaning against the wind period that drives the results also when the MPU is accounted for.

| DISCUSSION
Monetary policy tends to be described as forward-looking and concerned with the management of expectations.
Evaluations of it therefore often focus on how effective it has been at shifting forward-looking variables such as the yield curve or inflation expectations. But this paper studies forecast errors. Forecast errors are backward-looking T A B L E 5 Relative market forecast accuracy before and after path publication without and with a dummy for the period February 2010-July 2014 Intercept Dummy Note: "Intercept" gives η and "Dummy" gives θ in Equation (4). *, **, and *** indicate significance at the 10%, 5%, and 1% levels, respectively. Newey standard errors in parentheses. Regressions have 70, 65, and 59 observations at the one-quarter, 1-year, and 2-year horizon, respectively. and compare an old forecast with a realized outcome. One reason why this is fundamentally different is that the Riksbank has moderate control over the economy at longer horizons. Factors that determine whether the Riksbank manage expectations in the present are, therefore, not necessarily most central when studying forecast errors, especially for longer horizons. In particular, this holds for its credibility. That the then deputy governor Lars E.O.
Svensson disagreed with the board might, for example, have reduced the credibility of the published path and thereby reduced its impact on the yield curve. It is less obvious that this lower credibility affected the outcome in GDP growth, unemployment, inflation or consumption-and thereby interest rates-2 years later. Forecast errors are instead more dependent on the Riksbank's ability to forecast such variables and factors that might influence this ability.
Considering the whole period, the results of the above analysis suggest that the Riksbank's 2-year forecasts have been biased and inefficient, but so have the forecasts inferred from market prices. However, when comparing the two forecasts it becomes clear that the Riksbank has been significantly less accurate. This difference appears to be driven by something that occurs approximately around 2010-2014. Sveriges Riksbank (2017) has previously commented on the higher accuracy of market forecasts compared to the Riksbank. Regarding the bias of the forecasts, they note that the period was hard to forecast. All forecasters they look at overestimated future interest rates. 11 This result is partly attributed to the financial crisis and European debt crisis causing unexpected negative chocks to the repo rate. They also suggest that the long-run equilibrium interest rate had a downward trend, which was not detected during this period. These are both plausible explanations to why forecasts are in general biased.
To that discussion, this paper simply contributes by quantifying the biases and establishing that they are significant for the 1-and 2-year horizons.
Perhaps more interesting in the context of this paper, Sveriges Riksbank (2017) also discusses that the Riksbank's forecasts have been less accurate than the forecast inferred from market prices. The explanation they propose is essential that forecasts based on implicit forward rates under-estimate beliefs held by market participants. They quantify this by comparing the forecast based on forward rates to forecasts based on the Prospera surveys. The proposed mechanism is a negative term premium. If the premium is negative (or below a term premium adjustment made by the Riksbank) the inferred forecast would be lower than actual expectations.
Empirically there are two problems with this explanation. The first is the level. The average difference in forecast error for the period 2010-2014 is about one percentage point. Considering typical estimates of the term premium for that period their magnitude is not sufficient to fully explain such a large difference at the 2-year horizon (Alsterlind, 2017;De Rezende, 2017). Second, there is the evolution of the term premium compared to the Note: "Intercept" gives η, "Dummy MPU" gives φ and "Dummy 10-14" gives θ in Equations (6) and (7). *, **, *** indicate significance at the 10%, 5%, and 1% levels, respectively. Newey standard errors in parentheses. Regressions have 70, 65, and 59 observations at the one-quarter, 1-year, and 2-year horizon, respectively. 11 Though, they do not investigate if these biases are all significant. evolution of the difference in forecast errors. Over most of the period the term premium has been on a downward trend. 12 If the term premium caused the difference in forecast errors the latter should follow an upward trend. But we do not see such a trend in Figure 1. This is particularly clear for the period after 2014, where both the term premium and the difference in forecast errors are decreasing.
Regarding that, the market forecast differs from the Prospera survey it can as easily be the survey that diverges from the true beliefs as the inferred market forecast (Beechey & Österholm, 2014b). Dale, Orphanides, and Österholm (2011) argue that if the central bank publishes forecasts of poor quality this can decrease the quality of other forecasts. Traders had financial stakes in asset prices and larger incentives to scrutinize the Riksbank's forecast. One further point, due to Sveriges Riksbank (2017), is that the National Institute of Economic Research's forecast MSE was in line with the forecast inferred from market prices. This supports the notion that it was the Riksbank that deviated from the norm, not the market.
Since the implicit market forecast is taken from official communication from the Riksbank it is plausible that the Riksbank's forecast is conditional on the implicit market forecast. Since the implicit market forecast can be expected to incorporate the market sentiment regarding monetary policy this implies that the Riksbank also deviated from market expectations on monetary policy. Furthermore, during the period 2010-2014 the full path of the market's forecast was below the Riksbank's forecast for all but a few times. Since a higher policy rate tends to be contractionary a lower interest rate in the short run presumably implies counterfactually higher interest rates in the long run.
These factors indicate that the Riksbank strongly disagreed with market expectations for the future economy.
It should also be noted that even though neither the Riksbank, the market, nor the naïve forecasts were unbiased at longer horizons the Riksbank stands out regarding forecast accuracy. My results suggest that this is only true for a particular time period, 2010-2014. An explanation of the results should, therefore, be sought in something that occurred at that period and was specific for the Riksbank. The leaning against the wind policy meets these criteria. Considering that interaction between leaning against the wind and publishing a policy rate path is not part of the general discussion of either topic it might be relevant to say something about the potential mechanism.
Goodfriend and King (2016), as well as Svensson (2016), suggest that it was a combination of explicit choices by the board of directors as well as "optimistic" forecasts that caused the poor forecast accuracy. The motive for leaning against the wind was to increase long-term interest rates. The Riksbank, therefore, had the incentive to communicate its resolve to set the policy rate higher than what the market expected (than what was blowing in the wind).
One straightforward way to communicate this would be to publish a policy rate path that was above the expectations implicit in market prices. Actually this could apply even if there was no deliberate leaning against the wind, as long as the Riksbank was influenced by the incentive to lower housing prices. Second, regarding the "optimistic" forecast, Goodfriend and King (2016) and Svensson (2015Svensson ( , 2017 explicitly mention that unreasonable forecasts of foreign interest rates motivated a high repo rate. The question then remains why the staff of the Riksbank made unreasonable forecasts of foreign interest rates. Svensson (2015) associate this with the board's desire for a tight monetary policy, which the staff incorporated in the forecasts.
Obviously there could be other mechanisms at play. The analysis of this paper does not provide good insight into the motivation behind the results. Still the reasons for letting the path deviate from market expectations were discussed at the board meeting in September 2010 (Goodfriend & King, 2016). One way to interpret this discussion is that the board treated the policy rate as a policy instrument to manage expectations rather than simply as a forecast. Since there is not much written on how the central bank interprets the path it might be interesting to linger a bit on the potential implication of this paper on that interpretation. If the difference in the forecasts reflects an incentive for the Riksbank to increase long term interest rates to affect housing prizes my results indicate that this caused the Riksbank to publish relatively inaccurate forecasts. Regardless of whether leaning against the wind is an advisable policy (or even if it at all played a part in producing the results found in this paper) consistently being 12 See Figure 2 in Alsterlind (2017) and Figure 6 in De Rezende (2017).
inaccurate, in spite of data assembled, analyzed, and understood by the own organization, might be a problem for an honest and open institution.

| CONCLUSIONS
Extending the analysis of BÖ from 2012 to 2018 this paper rejects their conclusion that there are no significant differences between the Riksbank and market forecasts. At the 2-year horizon, the forecasts inferred from market prices are significantly more accurate. Results from regression analysis using a dummy variable for the period February 2010 and July 2014 suggest that events during approximately this period may have generated the difference, consistent with the view that the Riksbank's desire to lean against the wind during this period was associated with unrealistic forecasts. Though I do not investigate a precise mechanism I argue that the wish for high-interest rates to discourage household debt motivated the Riksbank to publish a repo rate path that was above what it would otherwise forecast. It is not clear if this caused market interest rates to rise. If it did the policy might be called successful in so far as that it had the intended effect on contemporaneous interest rates. However, a forecast predicts the future. If the Riksbank whished the future would unfold differently from what other forecasters predicted this was not enough to change what actually happened. When the 2 years were gone and past so were contemporaneous effects on expectations and all that was left was a relatively inaccurate forecast.

ACKNOWLEDGMENTS
I am grateful to Johan Lyhagen, Pär Österholm, two anonymous referees and the editor Giovanni Ferri for valuable comments and to Meredith Beechey Österholm, Pär Österholm and Sveriges Riksbank for sharing data. Data are available from the author upon request provided that permission to share the data is granted by Sveriges Riksbank.

DATA AVAILABILITY STATEMENT
Data are available from the author upon request provided that permission to share the data is granted by Sveriges Riksbank. Electronic copy available at: https://ssrn.com/abstract=3707486

Non-technical Summary
Since 2007 the Swedish central bank, the Riksbank, has published a path for the Swedish policy rate, presenting the Riksbanks view on how the policy rate will evolve over time. Using established methods this paper finds that this path has been a relatively poor forecast. During the period 20102-2014 the Riksbank was concerned with increasing household debt due to rising prices on housing. It is during this period that the path performs worse than comparable forecasts. Specifically, the path is too high. But if the Riksbank wanted to discourage household debt a high path is what it should aspire to, just as forward guidance only is effective if the path is lower than expected.
Unfortunately, further analysis suggests that the market forecast did not use the information in the path to improve their forecast during this period, unlike preceding periods. This is troublesome since it suggests that the market might lose faith in a central bank that for a longer period deviates from market expectations.
F I G U R E A 5 Forecast errors at the two-year horizon. Dates refer to when the forecast was made