Misidentifying the Effects of Parental Incarceration? A Comment on

Throughout the last 40 years, the American imprisonment rate has increased from a fairly stable rate of about 100 prisoners per 100,000 members of the population in the early 1970s to about 500 per 100,000 now (Blumstein & Cohen, 1973; Glaze, 2011). Yet the American imprisonment rate is not only historically novel but also comparatively extreme, as it exceeds those in all other countries and dwarfs those in other developed democracies (Walmsley, 2008). This sea change in imprisonment may have implications for family life, especially among African American families residing in neighborhoods of concentrated disadvantage, for whom incarceration is now common (Sampson & Loeffler, 2010; Western & Wildeman, 2009). Fully 25% of African American children will ever have a parent imprisoned, and this is true of more than half of African American children whose fathers did not complete high school (Wildeman, 2009).

As a result of these increases in imprisonment, a large literature on the consequences of incarceration for family life has emerged, some of which has appeared in the Journal of Marriage and Family (Dyer, Pleck, & McBride, 2012; Green, Ensminger, Robertson, & Juon, 2006; Johnson & Easterling, 2012; Lopoo & Western, 2005; Poehlmann, 2005; Turney, Schnittker, & Wildeman, 2012). Although some research has considered the effects of having a male family member incarcerated on adult women's well-being (Wildeman, Schnittker, & Turney, 2012) or household finances (Geller, Garfinkel, & Western, 2011; Schwartz-Soicher, Geller, & Garfinkel, 2011), much of it considers how parental incarceration affects child well-being. In a recent Journal of Marriage and Family article, Johnson and Easterling (2012) reviewed research on the effects of parental incarceration on child well-being, focusing on the various conceptual frameworks linking parental incarceration and child well-being and the “important methodological and conceptual challenges related to selection bias” (p. 342) plaguing research in this area. After reviewing 10 relevant studies, the authors reached three conclusions. First, research on the effects of parental incarceration on child well-being is not particularly rigorous, as it has used few methods to diminish the often-substantial concerns about selection bias. Second, on the basis of the one study that found children with imprisoned fathers were no worse off than children of divorced parents (Moerk, 1973), the authors concluded there are good reasons to be skeptical that imprisonment has any effects above and beyond paternal separation more broadly (pp. 347–348). Finally, on the basis of rigorous studies that did not use children of divorced parents as the comparison, the authors concluded that, if there are effects of parental incarceration, it is unclear whether parental incarceration helps or harms child well-being, as the direction of average effects varies across studies (p. 348).

In this comment, we address the gaps in Johnson and Easterling's (2012) review of the empirical research—gaps we consider sufficiently large that we consider the review to be an inaccurate portrayal of the existing research on the consequences of parental incarceration for child well-being. We first briefly summarize their methodological suggestions for minimizing selection bias, their review of the literature, and their key conclusions. We then introduce 12 relevant studies that were excluded from their review and emphasize how including these studies not only changes the assessment of the methodological rigor of the literature in this area but also leads to substantively different conclusions. Because we are mostly in agreement with their discussion of conceptual frameworks, we do not address that portion of their article.

Understanding Unique Effects of Parental Incarceration on Children

Johnson and Easterling's (2012) argument can be summarized as follows:

Children whose parents are incarcerated often differ from children whose parents are not incarcerated on a number of dimensions other than parental incarceration status alone, making it difficult to infer whether the problems that have been observed among children whose parents are incarcerated are due to the parent's incarceration or other adversities in the child's ecology. (p. 344)

The authors are quite correct. Obstacles to causal inference in this area are steep, and overstating the case for causal effects of parental incarceration on children could lead to policy interventions that harm rather than help children of incarcerated parents (e.g., Giordano, 2010; Sampson, 2011).

So, what to do? Johnson and Easterling (2012, p. 344) noted that estimates garnered from an experimental design would be ideal, yet they acknowledged that experimentation in this area of inquiry is unethical (or at least difficult). Absent that, they suggested four methods: “controlling for factors that are associated with both parental incarceration and child outcomes; estimating pre- and postincarceration change models; using comparison groups that are matched on key selection variables; and exploiting natural experiments” (p. 344). We add one strategy that may be useful in estimating the link between parental incarceration and child well-being: placebo (or synthetic) regressions (Conley & Springer, 2001, p. 785). In contrast to the other strategies that look for null effects to show a spurious relationship, placebo regressions accomplish the same goal by showing effects where there cannot be or should not be effects. In this case, placebo regression models could demonstrate that parental incarceration has no effect on child well-being by showing that future parental incarceration predicts current child well-being.

After discussing these various strategies, Johnson and Easterling (2012) then reviewed the existing literature. We summarize these studies in Table 1. For each article included in their review, we document three things: (a) the general outcome considered; (b) whether the author(s) used one or more comparison groups (e.g., children who had a parent absent from the household because of divorce or death); and (c) whether the author(s) used covariate adjustment, propensity score matching (what Johnson and Easterling call matching), fixed effects (what Johnson and Easterling called pre- and postincarceration change models), or placebo regressions. We excluded natural experiments from the table because none of the studies reviewed used this design.

Table 1.  Studies Reviewed in Johnson and Easterling (2012)

Authors/Year	Outcomes	Comparisons		Methods Used
		One	More	Adjustment	Matching^a	Fixed Effects	Placebos
^aWe use the term matching to indicate propensity score matching, not solely research designs using matched pairs. ^bBecause the difference-in-difference design controls for unobserved time-invariant differences, we categorize it as using fixed effects.
Aaron & Dallaire (2010)	Delinquent behavior	Yes	No	Yes	No	No	No
Besemer et al. (2011)	Criminal justice contact	Yes	No	Yes	No	No	No
Cho (2009a)	Grade retention	Yes	No	Yes	Yes	Yes^b	No
Cho (2011)	High school dropout	No	Yes	Yes	No	No	No
Huebner & Gustafson (2007)	Criminal justice contact	Yes	No	Yes	No	No	No
Moerk (1973)	Psychological adjustment	Yes	No	No	No	No	No
Murray & Farrington (2005)	Delinquent behavior	No	Yes	Yes	No	No	No
Murray & Farrington (2008)	Internalizing problems	No	Yes	Yes	No	No	No
Murray et al. (2007)	Criminal justice contact	Yes	No	Yes	No	No	No
Wilbur et al. (2007)	Depressive symptoms	Yes	No	Yes	No	No	No

On the basis of these 10 studies, Johnson and Easterling (2012) reached three conclusions. First, few studies that consider the relationship between parental incarceration and child well-being are particularly rigorous, a statement strongly supported by Table 1. Indeed, few studies reviewed in Table 1 used more than one comparison group or the more rigorous methods. Only three of 10 used more than one comparison group, nine of 10 used covariate adjustment, one of 10 used matching, one of 10 used fixed effects (or other approaches that control for unobserved yet stable traits), and none used placebo regressions. Second, “the results of the one study that compared children who were separated from parents because of incarceration and children separated from parents because of divorce (Moerk, 1973) suggested few significant differences” (pp. 347–348). Thus, they saw little evidence that parental incarceration has effects above and beyond parent–child separation more broadly. Third,

The results of the more rigorous studies … are consistent in suggesting that parental incarceration may have unique effects on child well-being. They differ, however, in whether the results suggest that parental incarceration has negative or positive effects on child outcomes. (p. 348)

Put succinctly, if there are effects of parental incarceration on child well-being—and their second conclusion leaves that very much open for debate—it is unclear whether they are beneficial or harmful, because some of the most rigorous studies that did not use children of divorced parents as the comparison group suggest average negative effects on children, whereas others suggest average positive effects. Importantly, Johnson and Easterling do note that this final conclusion may be driven by the incarcerated parent's gender, which we will argue, in short order, is essential for understanding this literature.

Expanding the Studies Reviewed Leads to Divergent Conclusions

Each of these conclusions, in our opinion, results from the fact that Johnson and Easterling (2012) excluded a number of relevant studies in their review. As Table 2 shows, we discovered 12 studies the authors did not review. Some of these articles, to be fair, were likely not published when Johnson and Easterling sent in the final version of their article. Yet regardless of whether these more recent articles are included, there are substantial differences between the conclusions reached on the basis of Table 1 as opposed to both Tables 1 and 2. It also worth noting, before going into greater detail, that these conclusions differ from those of other reviews (Murray, Farrington, & Sekol, 2012; Murray, Farrington, Sekol, & Olsen, 2009; Wakefield & Uggen, 2010, pp. 397–398; Wildeman & Western, 2010, pp. 167–169), many of which, we would argue, were quite sensitive to concerns about selection bias, even if it was not the sole focus of those reviews.

Table 2.  Additional Relevant Studies Not Reviewed in Johnson and Easterling (2012)

Authors/Year	Outcomes	Comparisons		Methods Used
		One	More	Adjustment	Matching^a	Fixed Effects	Placebos
^aWe use the term matching to indicate propensity score matching, not solely research designs using matched pairs. ^bThese studies went into print fewer than 6 months before the publication of Johnson and Easterling's (2012) review was published (and hence may have been unavailable to them), but we include them because we do not want to exclude relevant recent research. ^cAlthough mentioned in their review, this article was cited as showing “that parental incarceration may have some benefits for children” (Johnson & Easterling, 2012, p. 353), which is an inaccurate description of the average effects shown elsewhere in the article, so it is reviewed here.
Cho (2009b)	Reading and math test scores	Yes	No	Yes	Yes	Yes	No
Craigie (2011)	Behavioral problems	Yes	No	Yes	No	No	No
Foster & Hagan (2007)	Social exclusion	Yes	No	Yes	No	No	No
Foster & Hagan (2009)	Educational attainment	Yes	No	Yes	Yes	No	No
Geller et al. (2012) ^b	Behavioral problems	No	Yes	Yes	Yes	Yes	Yes
Johnson (2009)	Behavioral problems	Yes	No	Yes	No	No	No
Murray, Loeber, & Pardini  (2012)^b	Delinquent behavior	Yes	No	Yes	Yes	Yes	No
Roettger & Swisher (2011) ^b	Delinquency and arrest	Yes	No	Yes	No	No	No
Roettger et al. (2011)	Drug use	Yes	No	Yes	No	No	No
van de Rakt et al. (2012) ^b	Criminal justice contact	Yes	No	Yes	No	No	No
Wakefield & Wildeman (2011)	Behavioral problems	Yes	No	Yes	Yes	Yes	No
Wildeman (2010) ^c	Physical aggression	Yes	No	Yes	Yes	Yes	Yes

On the most basic level, the methods used in the additional 12 studies are significantly more rigorous than those in the studies Johnson and Easterling (2012) reviewed, which leads to different conclusions about the rigor of this subfield. Only one of 12 studies used more than one comparison group, but all 12 adjusted for covariates, six of 12 utilized propensity score matching, five of 12 used fixed effects, and two used placebo regressions. Thus, Johnson and Easterling identified one of seven studies that used propensity score matching, one of six that used fixed effects, and zero of two that used placebo regressions. Using these methods does not allow for an estimation of causal effects, of course, but it does suggest that, had Johnson and Easterling identified these 12 studies, their conclusion about the rigor of this literature would have been quite different.

Johnson and Easterling's (2012) second conclusion, that there is little evidence parental incarceration is more harmful to children than other forms of separation such as divorce (or paternal absence more broadly), is trickier. We do not agree with this conclusion, and the studies reviewed in Tables 1 and 2 provide little evidence to support it. It appears that Johnson and Easterling (pp. 347–348) came to this conclusion on the basis of one study (Moerk, 1973) that matched 24 children of imprisoned fathers with children of fathers absent because of divorce. If Johnson and Easterling meant to focus solely on divorce, we concede that evidence on this one reference group is limited, as studies in this area consider paternal absence more broadly (Geller, Cooper, Garfinkel, Schwartz-Soicher, & Mincy, 2012; Murray & Farrington, 2005).

Yet because most ever-imprisoned men are not married to their children's mothers, the question of whether paternal incarceration matters more for children than paternal absence may be the more interesting one. In response to this question, we do not think the literature is as unclear as Johnson and Easterling (2012) suggested, because the most rigorous research using contemporary (Geller et al., 2012) and less recent but exceptional (Murray & Farrington, 2005) data comes to consistent conclusions. On the basis of data from the Fragile Families and Child Wellbeing Study, Geller and colleagues (2012, p. 66) showed that paternal incarceration is associated with significantly greater increases in aggressive behaviors than is paternal absence among 5-year-old children. Data from the Cambridge Study in Delinquent Development show that paternal incarceration is associated with significantly greater risks of antisocial personality disorder and self-reported delinquency at age 18 than is paternal absence for any reason other than hospitalization or death (Murray & Farrington, 2005, p. 1275). Both studies also show a number of null findings. Of the five measures Geller and colleagues (2012, p. 66) considered, for only one measure was the effect of paternal incarceration significantly greater than the effect of paternal absence across all specifications—and even it is only marginally significant in fixed-effects models (p < .10). Of the 11 outcomes considered by Murray and Farrington (2005, p. 1275), parental incarceration was significantly more detrimental than paternal absence for only two outcomes, although statistical insignificance in this study should not be oversold in light of the small number of observations. Thus, although there is no definitive evidence the parental incarceration–child well-being association is stronger than the parental absence–child well-being association, reviewing these studies in more detail would have led to more confidence that paternal incarceration may have negative consequences for child well-being above and beyond paternal absence, at least for some vital outcomes.

The final essential conclusion reached by Johnson and Easterling (2012) concerns the most rigorous studies reviewed that did not use children of divorced parents as the reference cell. Thus, these insights should be considered in light of their preexisting conclusion that parental incarceration may have no effects beyond paternal absence. Put concisely, they concluded that there are effects of parental incarceration on children in these studies, but those effects are, on average, sometimes positive and sometimes negative. As with their first two conclusions, we are convinced that a more complete review of the literature would have led to a different conclusion.

On the basis of a review of the works listed in Tables 1 and 2, it is clear that the direction of the effects of maternal incarceration are indeed inconsistent. As Cho's (2009a, 2009b, 2011) exceptionally rigorous work using administrative data from Cook County makes transparent, it is often unclear whether maternal incarceration, on average, improves or diminishes child well-being. Given that incarcerated mothers are a more select group than incarcerated fathers (as indicated by the fact that far more fathers than mothers will go to prison during their children's formative years; Wildeman, 2009), it is perhaps not surprising that, depending on the outcome considered, the average effects of maternal imprisonment on child well-being could be positive, negative, or null.

Though it may be unclear whether maternal incarceration has any effect on children (and, if it does, its direction), the same can hardly be said for paternal incarceration. Indeed, in each of the 11 studies reviewed in Table 2 that considered paternal incarceration, the relationship between paternal incarceration and child well-being was either null or negative. This suggests the debate has not to do with the direction of effects, as Johnson and Easterling (2012, pp. 347–348) concluded, but with whether the average negative effects are causal or products of social selection forces (and hence null). To acknowledge a nuance in the literature that merits a moment's attention given how much it has been considered (Geller et al., 2012; Murray, Farrington, & Sekol, 2012; Murray, Loeber, & Pardini, 2012; Wildeman, 2010), evidence using the most rigorous research designs suggests that paternal incarceration's effects on children's behavioral and mental health problems may be largest for externalizing, delinquent, violent, and physically aggressive behaviors and smaller for internalizing behaviors and mental health problems, as well as minor forms of delinquency (but see Wakefield & Wildeman, 2011). Thus, there is not only evidence that the effects of paternal incarceration on children do not go in opposite directions, tending to be either negative or null (and never positive on average), but also specific knowledge about the types of outcomes most consistently associated with negative effects and null or weak effects.

This point is also relevant in light of Johnson and Easterling's (2012) conclusion that there is “accumulating evidence that parental incarceration may have some benefits for children” (p. 353). Unfortunately, the one study they cite to support this claim that considered paternal incarceration (Wildeman, 2010) found no evidence of average beneficial effects (and used all four analytic strategies shown in Tables 1 and 2, a point missed in their review). The conclusions drawn in that study are that

paternal incarceration is associated with increased physical aggression for boys, and that effects are concentrated among boys whose fathers were neither incarcerated for a violent offense nor abusive to the boys' mother. Results also suggest that paternal incarceration may decrease girls' physical aggression, although this finding is not robust. (Wildeman, 2010, p. 285)

Thus, the key finding is not that paternal incarceration is on average protective but that in certain special situations, especially those in which the father had engaged in domestic violence, it is unclear whether paternal incarceration helps or harms child well-being.

Conclusion

Although Johnson and Easterling (2012) identified why it is important to think about selection bias—especially when studying parental incarceration—and provide guidance in terms of theoretical frameworks for understanding the unique effects of parental incarceration on children, the studies included in their review are so limited that they provide an inaccurate picture of the state of research in this field, leading the authors to reach erroneous conclusions about both the rigor of prior research and the unique effects of parental incarceration on child well-being. By writing this comment, we hope to have provided a more accurate view of research in this area without disparaging the other components of Johnson and Easterling's article.