A critical reflection on computing the sampling variance of the partial correlation coefficient

The partial correlation coefficient quantifies the relationship between two variables while taking into account the effect of one or multiple control variables. Researchers often want to synthesize partial correlation coefficients in a meta‐analysis since these can be readily computed based on the reported results of a linear regression analysis. The default inverse variance weights in standard meta‐analysis models require researchers to compute not only the partial correlation coefficients of each study but also its corresponding sampling variance. The existing literature is diffuse on how to estimate this sampling variance, because two estimators exist that are both widely used. We critically reflect on both estimators, study their statistical properties, and provide recommendations for applied researchers. We also compute the sampling variances of studies using both estimators in a meta‐analysis on the partial correlation between self‐confidence and sports performance.


Highlights
What is already known • The partial correlation coefficient quantifies the relationship between two variables while taking into account the effect of one or multiple control variables. • Meta-analyses on partial correlation coefficients are regularly conducted in different research fields.
What is new • The literature on estimators of the sampling variance of the partial correlation coefficient is diffuse. Two different estimators of the sampling variance are frequently used in practice. • Our critical reflection and assessment of the statistical properties of the estimators show that the estimator derived in Olkin and Siotani (1976) and Anderson (1984) is preferred for meta-analyzing partial correlation coefficients.
Potential impact for RSM readers outside the authors' field • The quality of meta-analyses on partial correlation coefficients will be improved if researchers are using the recommended estimator of the sampling variance of the partial correlation coefficient.

| INTRODUCTION
Meta-analysts often want to synthesize the results of a linear regression model where the effect of interest is the relation between an independent variable and a dependent variable when controlling for other variables in the model. Meta-analyses on these so-called partial effects are becoming more common 1 and are regularly conducted in, for example, psychology 2-6 and economics. [7][8][9][10][11] The partial correlation coefficient (PCC) can be used as effect size measure in these meta-analyses. The PCC quantifies the relationship between the independent and dependent variables where there is controlled for the effect of the other variables in both the independent and dependent variable. 12,13 Each study's PCC needs to be accompanied by its sampling variance, because effect sizes in a meta-analysis are generally weighted by the inverse of the sampling variance. The existing literature is diffuse on how to compute the sampling variance of the PCC. Two different estimators are available in the literature and both are used in practice. The goal of this paper is to compare both estimators and provide recommendations for researchers on how to compute the sampling variance of a PCC.

| STATISTICAL MODEL
Let N be the number of independent observations with Y i denoting the observed score on the dependent variable of participant i. We write the population linear regression model as where β 0 is the intercept, β 1 X i1 is the regression coefficient of the independent variable X 1 , β M X iM refers to the m ¼ 1,ÁÁÁ, M th regression coefficient of the independent variable X m , and ϵ i is the sampling error that is assumed to follow a normal distribution with mean zero and variance σ 2 .

| PARTIAL CORRELATION COEFFICIENT
The PCC between variables Y and X 1 controlled for variable X 2 can be estimated using 12-15 where the Pearson correlation coefficients between Y and X 1 , Y and X 2 , and X 1 and X 2 are denoted by r Y 1 , r Y 2 , and r 12 , respectively. Estimating Equation (1) can only be used when controlling for a single variable. An alternative estimating equation for the PCC between Y and X 1 that allows for controlling for one or more variables is [16][17][18] ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi b 2 where b 1 is the estimate of β 1 , df refers to the degrees of freedom that are equal to N À M À 1, and t 1 is the t-statistic of the test H 0 : β 1 ¼ 0. Estimating Equation (2) is especially useful in the context of meta-analysis, because Pearson correlation coefficients between all variables included in the model are usually not reported in the primary studies and it allows for more than one control variable. Fisher 19 noted that the probability density function (PDF) of the PCC is the same as of a Pearson correlation coefficient except for the degrees of freedom. Hence, the exact PDF of the PCC is (equation (25) of Hotelling 20 ) ð Þ 1 À ρr p À Á Àdf À0:5 2 F 1 0:5,0:5, df þ 1:5, where ρ is the PCC in the population, the degrees of freedom (i.e., df) are equal to N À M À 1, Γ is the gamma function, and 2 F 1 is the Gaussian hypergeometric function. The exact PDF if ρ ¼ 0 simplifies to a Student's tdistribution and is commonly used for testing H 0 : ρ ¼ 0, 21

| SAMPLING VARIANCE OF THE PARTIAL CORRELATION COEFFICIENT
We continue with describing two popular estimators for estimating the sampling variance of the PCC before we reflect on their statistical properties.

| Estimators
The first estimator of the sampling variance of the PCC that we describe has been derived in Olkin and Siotani 14 and Chapter 4 of Anderson. 22 Its estimating equation is Note that the numerator in estimating Equation (5) is equal to the numerator of the sampling variance of the Pearson correlation coefficient (e.g., see equation (11.35) in Borenstein and Hedges 23 ). Only the denominators differ, because the number of regression coefficients (i.e., M) are included in the degrees of freedom that are in the denominator of estimating Equation (5). This estimator is the large sample approximation of the variance of the PCC if r p is replaced by ρ in Equation (5).
The second estimator is reported on page 25 of the popular book on meta-analysis by Stanley and Doucouliagos. 17 Its estimating equation is The square root of this second estimator is actually equal to the standard error used in Equation (4) for computing the t-statistic. This implies that this estimator is derived conditional on ρ ¼ 0.
Preliminary observations when comparing both estimators are that the difference between the estimators s 2 1 and s 2 2 is in the numerators where the term 1À r 2 p is squared in s 2 1 but not in s 2 2 . This implies that both estimators yield the same estimates if r p ¼ 0. Furthermore, it also implies that s 2 1 < s 2 2 for r p ≠ 0, because the numerator of s 2 1 is always smaller than the numerator of s 2 2 .

| Comparing the estimators
We compared s 2 1 and s 2 2 with the variance obtained using the exact PDF in (3) for a PCC between two variables while controlling for one variable (i.e., M ¼ 2). That is, we computed s 2 1 , s 2 2 , and the variance using the exact PDF for 1000 equidistant values of ρ that range from À0.99 to 0.99. We assumed that the PCC is an unbiased estimate of ρ* and computed s 2 1 , s 2 2 by replacing r p in Equations (5) and (6) by ρ. The selected levels of the sample size were N ¼ 8; 16; 32; 64. The variance based on the exact PDF was obtained by computing the second moment of the PDF, The software R 27 (Version 4.2.2) was used for the analyses. The Gaussian hypergeometric function implemented in the R package "gsl" 28 (Version 2.1.7.1) was used for evaluating the exact PDF. R code of the analyses is available at https://osf.io/pqvyx. Figure 1 shows the variance of the PCC for the different estimators where each subfigure contains the results of a particular sample size N. The properties of the different estimators were most manifest for the top-left subfigure with N ¼ 8. The variances of the PCC using s 2 1 (green dashed line) and s 2 2 (orange dashed line) were equal to each other for ρ ¼ 0 but larger than the variance based on the exact PDF (black solid line). If ρ ≠ 0, s 2 2 approached the variance based on the exact distribution whereas s 2 1 was always larger than the variance based on the exact PDF and s 2 2 . These patterns are also apparent in the other subfigures of Figure 1 with larger N. The differences between both s 2 1 and s 2 2 and the variance based on the exact PDF decreased as a function of N where s 2 1 approached the variance based on the exact PDF more rapidly than s 2 2 .

| EXAMPLE
We computed s 2 1 and s 2 2 for the meta-analysis by Craft et al. 29 on the PCC between self-confidence and sports performance when controlling for cognitive and somatic anxiety. These data were obtained from table 16

| DISCUSSION
PCCs are frequently used as effect size measure in a meta-analysis. We have reflected on two estimators of  van AERT and GOOS the sampling variance of the PCC and examined their properties. This revealed that the estimator proposed in Stanley and Doucouliagos 17 was derived under the assumption of a zero PCC in the population. This estimator is especially biased when the PCC in the population is different from zero and the sample size is small. Hence, we recommend researchers to use the estimator that was derived in Olkin and Siotani 14 and Anderson, 22 because this estimator can be used for a zero and nonzero PCC in the population. Using a suboptimal estimator of the sampling variance biases the results of a meta-analysis, because the inverse of the variances are typically used as weights. Another reason why accurate estimation of the sampling variance is of importance is that the sampling variance or the square root of the sampling variance (i.e., standard error) are often used for assessing small-study effects. Small-study effects refer to the tendency of small studies to go along with larger effect sizes, and one of the possible causes of small-study effects is publication bias. 30,31 Commonly used methods to test and correct for small-study effects are Egger's regression test 30 and PET-PEESE 32 that require researchers to include the sampling variance or its square root as moderator in a meta-regression model.
We have focused on the PCC as effect size measure. An alternative option is to not meta-analyze PCCs directly, but first apply Fisher's z transformation and use the transformed PCCs as effect size measure in the meta-analysis. 18,22 This is analogous to how Pearson correlation coefficients are frequently meta-analyzed. A desirable property of this Fisher's z transformation is that the sampling distribution of a study's transformed effect size is approximately normally distributed. It is especially beneficial to apply the Fisher's z transformation for PCCs that are not close to zero, because the normality assumption is then more likely violated when metaanalyzing untransformed PCCs. Another desirable property of the Fisher's z transformation is that the sampling variance is independent of the PCC. This sampling variance of Fisher's transformed PCCs can be estimated with 1= NÀ 3 À M À 1 ð Þwhere M À 1 are the number of control variables. 19,22 To summarize, the frequently used estimating equation of the sampling variance of the PCC by Stanley and Doucouliagos 17 should only be used in a meta-analysis if the PCC in the population is zero. Hence, we recommend to abandon this estimator and use the estimator derived in Olkin and Siotani 14 and Anderson 22 instead.

ACKNOWLEDGMENTS
We would like to thank Prof. Dr. Tom Stanley for providing feedback on an earlier version of this paper.  (1)).