Do Immigrants Answer Questions about Their Legal Status?
Our first research question is whether evidence of non-response emerges when people are asked about their migration status. As hypothesized, this does not appear to be the case in the LAFANS data. Table 1 reports the prevalence of missing data for the series of questions asked of immigrants in the LAFANS. Of the 1,949 immigrant respondents who were asked the naturalization question, 69, or 3.7 percent, had missing data. The percentage missing increases to 5.7, 10.5, and 12.4 for the green card, refugee, and temporary visa questions, respectively. The bulk of this increase is due to the fact that the same 69 respondents with missing data on the citizenship question also had missing data on all of the subsequent questions. Of all of the foreign-born respondents in LAFANS, only 4.3 percent (N = 84) have an ambiguous immigration status due to non-response to the series of questions that determine legal status. We assume that these ambiguous cases are unauthorized, but the profile of the unauthorized population (presented below) does not change when we delete these cases from the sample.
Table 1. Missing Data on Citizenship and Immigration Status Questions among Adult Immigrants in Los Angeles, 2001
|Temporary Visa Still Valid?d||94||23||0||0.0|
Also as hypothesized, non-response to the legal status items in SIPP is much higher than in the LAFANS (Table 2). Of the 9,178 foreign-born persons asked the question about arrival status, 27.2 percent of the responses were allocated by the Census Bureau. And of the 2,445 respondents asked the question about adjusting to LPR status, 17.8 percent had allocated responses. Both of these percentages exceed the 15 percent threshold beyond which Allison (2002) suggests multiple imputation techniques be used for the handling of missing data. However, it is important to note that legal status in the SIPP is indeterminable only for those non-naturalized (or missing), non-LPR arrivals (or missing) who also have missing data on the status adjustment question. Of the 2,494 immigrants with missing arrival status data, 1,552 (62%) were reported as naturalized citizens, so it can be inferred that these respondents either arrived as LPRs or arrived illegally and subsequently adjusted to permanent status before naturalizing. And among the 2,973 non-citizen immigrants with unknown or valid non-LPR arrival status, 1,168 did not provide a valid response to the question about adjustment of status. Thus, in total, 1,168 of 9,178 immigrant adults in the SIPP (12.7%) had an ambiguous legal status when considering the three questions jointly.
Table 2. Allocation Rates for Citizenship and Immigration Status Questions among Foreign-Born Respondents Aged 15 and Older, SIPP, wave 2, 2004
|Not in Universe||0||0||6,733|
|Total N in Universe||9,178||9,178||2,445|
Additional evidence suggests that the relatively high item-allocation rates for the legal status questions in SIPP may derive from issues related to the survey design or field operations rather than from the content of the questions per se. First, the percentage of missing values on the question about year of entry is also very high, 25.3. And among those who answered the question on year of entry, only 8.5 percent failed to answer the questions on legal status at entry. Thus, non-response to the legal status questions is concentrated among a relatively small percentage of immigrants who also did not answer other questions about their migration experience. This point is further illustrated in Tables 3 and 4. In Table 3 we compare the range of non-response/allocation to legal status items in LAFANS and SIPP to the range in five other publicly available surveys that have also asked legal status questions (more information about these data sources can be found in the Appendix). Relative to the other six surveys listed in Table 3, SIPP easily has the highest rate of non-response to the legal status indicators, consistent with the notion that something particular to the SIPP data collection is responsible for the high allocation rate, rather than the sensitive nature of the legal status questions.
Table 3. Range of Non-Response to Survey Questions on Immigrants' Citizenship and Legal Status in Selected, Publicly Available U.S. Surveys
|Survey of Income and Program Participation (SIPP)||In-Person||Public||1996–2008||4||4.6–25.3|
|The Los Angeles Family and Neighborhood Survey (LAFANS)||In-Person||Private||2001 and 2008||6||1.4–6.8|
|National Agricultural Workers Survey (NAWS)||In-Person||Public||1988–2009||1||0.5–5.5|
|Immigration and Intergenerational Mobility in Metropolitan Los Angeles (IIMMLA)||Telephone||Private||2004–2005||7||0.3–13.1|
|The Immigrant Second Generation in Metropolitan New York (ISGMNY)||Telephone||Private||1999–2000||3||0.1–1.6|
|National Asian American Survey (NAAS)||Telephone||Private||2008||2||0.2–5.2|
|Multi-City Study of Urban Inequality (MCSUI)||In-Person||Private||1992–1994||3||0.1–0.5|
Table 4. Missing or Allocated Data (Percentages) for Immigration-Related Questions in Selected U.S. Surveys
|Los Angeles Family and Neighborhood Survey (LAFANS), 2001||0.23||3.95||3.54||4.31|
|Survey of Income and Program Participation (SIPP), 2004||0.08||25.35||0.84||13.62|
|National Agricultural Workers Survey (NAWS), 2004–2005||0.25||0.13|| b ||0.41|
|National Asian American Survey (NAAS), 2008||0.31||6.61|| b ||6.23|
|Immigrant Second Generation in Metropolitan New York (ISGMNY), 1999–2000||0.00||0.62||0.09||1.59|
|Immigrant Integration and Mobility in Metropolitan Los Angeles (IIMMLA), 2004–2005||0.61||0.73||0.30||6.90|
|Multi-City Study of Urban Inequality (MCSUI), 1992–1994||1.47||2.16||4.84||6.99|
|American Community Survey (ACS), 2005||2.69||6.15||3.35|| d |
|Current Population Survey (CPS), March 2004||0.44||10.47|| c || d |
|National Health Interview Survey (NHIS), 2004||0.20||2.59||2.59|| d |
In Table 4 we present allocation rates for other immigration-related questions in the seven surveys with legal status indicators as well as three large government-sponsored surveys that are often used to study the immigrant population but do not measure legal status. Relative to other immigration-related variables, especially year of immigration, non-response/allocation for legal status is not appreciably higher in most surveys that measure it. To put these allocation rates into perspective, the average allocation rate of person-level variables in the 2005 American Community Survey was 3.7 percent. Thus, immigration-related variables tend to have higher than average rates of missing data, although the results here suggest that legal status variables are not more prone to non-response than other immigration-related variables such as year of immigration. Finally, it is also worth noting that other variables commonly used in social science research, such as income, are also allocated at relatively high rates. For example, the Census Bureau reports that 18 percent of the persons asked income questions in the 2005 ACS had their responses allocated.
Do Questions about Legal Status Produce a Chilling Effect?
Our second research question is whether asking about immigrants' legal status will have a “chilling effect” that leads subsequently to high unit non-response, or to refusal to participate in the survey altogether. We examine this question by comparing two outcomes: (1) non-response to the survey item immediately following the SIPP questions on legal status and (2) attrition from the SIPP panel between wave 2, when the immigration status questions were asked, and wave 3. If legal status questions exerted a chilling effect, we would expect to find higher rates of subsequent non-response and panel attrition among the unauthorized compared with legal non-citizens and naturalized citizens. The results are shown in Table 5 using the three status assignment methods described earlier. Because unauthorized status is positively correlated with other characteristics that have been found to be associated with non-response (Groves, Cialdini, and Couper, 1992), Table 5 presents, in addition to zero-order comparisons, adjusted probabilities based on logit models controlling for age, sex, marital status, education, home ownership, English-language proficiency, and duration of U.S. residence. The logit models are weighted using the SIPP person weights, and thus, the predicted probabilities can be interpreted as the percentage of the wave 2 population not represented in the third wave.
Table 5. Predicted Probabilities (Percentages) from Logit Models Predicting Non-Response to the Survey Question Immediately Following Legal Status Questions (A) and Panel Attrition Between Waves 2 and 3 (B), Foreign-Born Respondents ages 15 and Older, SIPP, 2004
|A. Subsequent Non-Response|
|B. Attrition, Wave 2–3|
We begin by focusing on comparative probabilities of non-response to the question immediately following the SIPP legal status questions, which in this case is the first in a series of questions about relationships to other household members (Panel A). Overall, non-response to this question is relatively low, although in the unadjusted case significantly higher among the unauthorized. For example, when legal status is assigned using either the hot deck or logical imputation method, the unauthorized are about twice as likely not to respond to the subsequent survey item as their legal non-citizen counterparts and 33 percent more likely when legal status is made using the multiple imputation method. Although the unauthorized are significantly more likely not to respond to the item following the legal status questions, their overall rate of non-response – 7.2 percent, 7.2 percent, and 5.7 percent for the hot deck, logical, and multiple imputation methods, respectively – is not so high as to suggest that their non-response derives from the chilling effects of the legal status questions. If this were the case, one would expect substantially higher rates of non-response among the unauthorized, and for legal status alone to be a stronger determinant of non-response than indicated by the very small pseudo-R-squared statistics reported for the unadjusted models in panel A of Table 5. Rather, we see that non-response is determined to a far greater extent by other characteristics also associated with legal status, as evidenced by the fact that predicted probabilities of non-response among the unauthorized are reduced in the unadjusted models, that the gap in non-response between unauthorized and legal immigrants decreases when controls are introduced, and that the pseudo-R-squared statistic is roughly three times larger in the adjusted versus the unadjusted model.
In Panel B of Table 5, we test a second type of chilling effect, attrition from the SIPP panel between waves 2 and 3. As would be expected, the overall rate of attrition is higher than the rate of non-response to subsequent survey items. Unauthorized migrants, regardless of the method used to assign legal status, are significantly less likely than legal migrants to be observed in the subsequent wave of the survey, but the difference is not so large to provide evidence that unauthorized immigrants are disproportionately dissuaded from subsequent participation in the survey due to being asked about their legal status. Moreover, the gap in rates of attrition across legal statuses diminishes somewhat with the introduction of the control variables, but the pseudo-R-squared statistics strongly suggest that attrition is driven largely by unobserved variables. While we cannot conclude with certainty the reason behind the relatively higher rates of panel attrition among the unauthorized, it is not surprising to find somewhat higher rates of attrition among the unauthorized given that more recently arrived immigrants live in more complex household arrangements and experience more turnover (Van Hook and Glick, 2007), but the magnitude of the difference between the rates of attrition for the unauthorized and other groups does not appear to reflect a chilling effect.
Comparative Profiles of the Unauthorized Foreign-Born Population
One reasonable concern over the use of measures of immigrants' legal status in surveys is that unauthorized immigrants may see no reason to report their status honestly, especially to representatives collecting data on behalf of the U.S. government. For this concern to be warranted, we would expect profiles of the unauthorized population that are based on self-reported data to deviate from those based on independent estimates derived largely from administrative and non-self-reported data. Thus, we turn now to comparisons of the unauthorized population estimated in LAFANS and SIPP to independent estimates derived from residual methods.
Residual methods estimate the size and demographic characteristics of the unauthorized population by subtracting a demographic estimate of the legally resident foreign-born population from an estimate based on a population survey (such as the CPS or ACS) or census count of the total foreign-born population. In short, the estimated number of unauthorized immigrants is the difference between the population estimate of legal immigrants and the survey-based estimate of all foreign-born. The two most commonly cited estimates of the characteristics of the unauthorized population are those of the Pew Hispanic Center (Pew) and the Department of Homeland Security (DHS), each of which uses variations of a residual-based method. The DHS estimates are based on the ACS and use administrative records on legal admissions to remove from the ACS the legalized population by age, sex, state of residence, and country of birth. The Pew estimates are based on the Current Population Survey, but use a very similar methodology as DHS to produce residual-based estimates of the unauthorized population. The Pew Hispanic Center also produces estimates of detailed characteristics of the unauthorized population based on imputed measures of legal status in the Current Population Survey. This methodology uses an assignment algorithm developed by Passel (Passel and Clark, 1998; Passel, Van Hook, and Bean, 2006), which identifies all the foreign-born individuals in the Current Population Survey who have a very low probability of being unauthorized (e.g., persons arriving before 1980, persons from major refugee sending countries, persons reporting as naturalized, veterans, public assistance recipients, persons in specialty occupations, etc.). Among the remaining potentially unauthorized, the method assigns unauthorized status probabilistically based on already established percentages unauthorized within occupation, state, and sex cells.
It is important to note that we are not here evaluating the accuracy of one set of estimates versus another. Rather, our goal here is to seek evidence that might give researchers pause in utilizing LAFANS and SIPP to study the unauthorized population. We propose that such evidence for concern would be present if the survey-based estimates varied dramatically from the Pew and DHS estimates, especially in instances when the latter two estimates are similar to each other.
Table 6 compares the LAFANS-based estimates (thus weighted using the adult person weights provided in the data) of the unauthorized foreign-born population in Los Angeles to estimates derived using the Passel algorithm (as published in an Urban Institute report by Fortuny, Capps, and Passel (2007)). Comparisons are made with respect to the percentage of the foreign-born population that is estimated to be unauthorized and the following characteristics of the unauthorized population: duration of U.S. residence (ten or fewer years), country of birth, and sex and age composition. It should be noted that all of the LAFANS-based estimates are for the adult unauthorized population, whereas the residual-based estimate is for the entire population with the exception of the sex and age distribution estimates, which are limited to the adult unauthorized population.3 About 12% of unauthorized persons are children younger than 18 (Hoefer, Rytina, and Baker, 2008). In addition, the LAFANS estimates are, of course, for the year 2001, while the residual estimates are based on March CPS data from 2004.
Table 6. Comparative Profiles of the Adult Unauthorized Foreign-Born Population in Los Angeles County, by Estimation Source
|% Unauthorized (of foreign-born)||26.3||26.2b|
|% in U.S. < 10 Years, Unauthorized||52.1||51.0c|
|Birthplace, Unauthorized (%)|
|Other Latin America||30.1||28.0c|
|% Male, Unauthorized||52.9||53.9d|
|Age Distribution, Unauthorized (%)|
The two estimates are similar with respect to the share of the foreign-born population that is unauthorized and the percentage of recent arrivals among the unauthorized. Both the LAFANS and the Fortuny et al. estimates indicate that unauthorized immigrants comprise about 26 percent of the foreign-born population and that among the unauthorized, 51–52 percent arrived in the United States within the previous 10 years. The estimates diverge, however, with respect to the country/region of birth distribution of the unauthorized. In particular, LAFANS estimates that 65 percent of the Los Angeles County unauthorized population is Mexican-born compared to 57 percent estimated by Fortuny, Capps, and Passel (2007). And, the LAFANS estimate of the unauthorized population that is Asian-born (4%) is substantially lower than the residual-based estimate (12%). We can only speculate about the reason(s) for this discrepancy, but one possibility is that it stems from the fact that the LAFANS was administered only in English and Spanish, which could have led to undercoverage among the Asian-born unauthorized population.
Turning finally to comparisons with respect to sex and age distributions of the unauthorized population over age 17, we find that LAFANS and the residual estimate are comparable in terms of the percentage of unauthorized adults estimated to be male (53% in LAFANS and 54% in Fortuny et al.), but the LAFANS estimates a relatively younger adult unauthorized population than the residual-based estimate. Again, the source of the age discrepancy is unclear. One possibility is that the focus by LAFANS on development outcomes among young children may have led to a sample biased toward young adult parents that is not fully accounted for in the person weights.
Table 7 presents comparisons between estimated (weighted using the SIPP person weights for wave 2) characteristics of the unauthorized population from the 2004 SIPP and estimates published by Pew and DHS, respectively. Table 7 reports three SIPP-based estimates, one for each of the three legal status assignment methods described earlier. The SIPP estimates are for the adult (age 18+) population; unless otherwise noted, the Pew and DHS estimates are for the entire unauthorized population. In addition, as noted in Table 7, while most of the Pew and DHS estimates are based on 2005 CPS/ACS data, estimates for some characteristics were published more recently and thus are based on subsequent years of data.
Table 7. Comparative Profiles of the Adult Unauthorized Foreign-Born Population in the United States, by Estimation Method/Source
|Legal Status of Foreign-Born (%)|
|Unauthorized, Years in the U.S. (%)|
|Unauthorized, Country of Birth (%)|
|Other Latin America||20.3||19.5||20.1||24.0a||n/a|
|Europe & Canada||6.1||5.9||6.5||6.0a||n/a|
|Africa & Other||4.3||6.0||6.1||4.0a||n/a|
|Unauthorized, State of Residence (%)|
|Unauthorized, Male (%)||53.6||53.8||54.7||56.0a||56.6d|
|Unauthorized, Age Distribution (%)|
With respect to the share of the foreign-born population that is unauthorized, all three SIPP estimates are lower compared to the Pew estimate, but the extent to which this is true varies across the three methods. When legal status is assigned using the Census Bureau's hot deck allocation method, just 18 percent of the foreign-born population is estimated to be unauthorized in the SIPP, compared to 29 percent estimated by Pew. Conversely, when legal status is assigned in the SIPP by the hot deck method, nearly 46 percent of the immigrant population is estimated to consist of naturalized citizens, nearly 18 percentage points higher than the Pew estimate of 32 percent. The multiple imputation and logical allocation methods (and especially the latter) yield estimated legal status distributions that more closely align with the Pew estimate, relative to the hot deck allocation method. For example, based on the logical allocation of unknown legal statuses in the SIPP, about 26 percent of the foreign-born population is estimated to be unauthorized compared to 29 percent by Pew; 35 percent of the SIPP foreign-born population is estimated to be legal non-citizens, versus 39 percent by Pew; and 39 percent are estimated to be naturalized citizens compared to 32 percent in the Pew estimate.
Thus, regardless of the method used in the SIPP, the foreign-born population is estimated to have a disproportionately larger share of naturalized citizens than is true for the Pew estimate. We suspect that this may have to do with the tendency of some immigrants, especially those from Mexico, to misreport as naturalized citizens in Census surveys (Van Hook and Bachmeier, 2013). In the residual-based Pew estimates, adjustments are made in an attempt to account for this tendency (Passel, Van Hook, and Bean, 2006), whereas the only such adjustment we make in the SIPP data is to recode persons reporting as naturalized citizens that have not been in the United States at least 5 years (unless they have a U.S. citizen spouse), as non-citizens. This methodological difference in the handling of naturalization reports, thus, may in part explain differences in the estimated legal status distribution of the foreign-born population.
Variation also exists with respect to the duration of residence of the estimated unauthorized population across the three assignment methods used in the SIPP.4 Using the hot deck and multiple imputation methods, 73 percent and 75 percent, respectively, of the unauthorized foreign-born population are estimated to have lived in the United States ten years or less. Using the logical allocation method, the estimated percentage of 64.7 closely aligns with the Pew estimate, 65 percent, both of which are higher than the DHS estimate of 59 percent.
Turning to country/region of birth of the unauthorized population, Mexicans predominate in all of the estimates presented in Table 7, although some variation does exist across the SIPP estimates. Mexicans are relatively less numerous when legal status is assigned using the hot deck allocation method (51.6% Mexican), compared to the multiple imputation method (56.4%) and the logical allocation method (54.8%), both of which are more in line with the identical Pew and DHS estimates of 57 percent. Conversely, the hot deck estimate of the share of the unauthorized population consisting of persons born in Asia (17.7%) is significantly higher than the other four estimates. And with respect to the state of residence of the unauthorized foreign-born population, there is general agreement across the set of five estimates reported in Table 7, which are listed for the five states with the largest concentrations of unauthorized residents.
There are modest differences between the three SIPP estimates and the Pew and DHS estimates with respect to the gender composition of the unauthorized population, with the estimated SIPP unauthorized population being slightly less male. Greater variation exists across the five estimates with respect to the age distribution of the unauthorized population. Sixty-one, 63, and 65 percent of the adult unauthorized population in the hot deck, multiple imputation, and logical allocation SIPP estimates, respectively, are between the ages of 18 and 34. These estimates are substantially higher than the Pew estimate of 51.6 percent, but relatively comparable to the DHS estimate, 60 percent.
In summary, SIPP-based estimates of the characteristics of the unauthorized population compare favorably to estimates derived from other data sources and using other methods. In instances where SIPP-based estimated characteristics of the unauthorized population diverge from residual-based estimates, there also tends to be little agreement between the two residual-based estimates (e.g., duration of U.S. residence and the age distribution of the unauthorized population). Most importantly, we find little in Table 7 to suggest that misreporting of legal status is so widespread in the SIPP to lead to substantially biased estimates of the unauthorized immigrant population. This conclusion varies somewhat depending on the method used to handle missing data for the legal status measures, as the multiple imputation and logical allocation methods tend to produce profiles of the unauthorized population that are more in line with those published by the Pew Hispanic Center and Department of Homeland Security.