SEARCH

SEARCH BY CITATION

Summary

  1. Top of page
  2. Summary
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Supporting Information

Background  The validity of International Classification of Diseases-9 codes for liver disease has not been determined.

Aim  To examine the accuracy of International Classification of Diseases-9 codes for cirrhosis with hepatitis C virus or alcoholic liver disease and HIV or hepatitis B virus coinfection with hepatitis C virus in Veterans Affairs data.

Methods  We conducted a retrospective study comparing the Veterans Affairs administrative data with abstracted data from the Michael E. DeBakey VA Medical Center’s medical records. We calculated the positive predictive value, negative predictive value, per cent agreement and kappa.

Results  For cirrhosis codes, the positive predictive value (probability that cirrhosis is present among those with a code) and negative predictive value (probability that cirrhosis is absent among those without a code) were 90% and 87% with 88% agreement and kappa = 0.70. For hepatitis C virus codes, the positive predictive value and negative predictive value were 93% and 92%, yielding 92% agreement and kappa = 0.78. For alcoholic liver disease codes, the positive predictive value and negative predictive value were 71% and 98%, with 89% agreement and kappa = 0.74. All parameters for HIV coinfection with hepatitis C virus were >89%; however, the codes for hepatitis B virus coinfection had a positive predictive value of 43–67%.

Conclusion  These diagnostic codes (except hepatitis B virus) in Veterans Affairs administrative data are highly predictive of the presence of these conditions in medical records and can be reliably used for research.


Introduction

  1. Top of page
  2. Summary
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Supporting Information

Administrative databases can be valuable sources of information for epidemiological, health services and outcomes research. These databases usually employ codes such as the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9 CM) codes to identify specific clinical conditions. The accuracy of these codes in correctly identifying the actual presence of these conditions is a potentially limiting factor for studies that use administrative databases. Therefore, validation of administrative data is important to ensure the dissemination of precise results.

The Department of Veterans Affairs (VA) is the largest integrated healthcare provider in the United States. The VA maintains electronic in-patient and out-patient administrative records from all VA facilities (currently 154 medical centres and 875 ambulatory care and community-based out-patient clinics) in the Patient Treatment File (PTF) and the Outpatient Care File (OPC) databases, respectively.

Several researchers have conducted validation studies for a variety of conditions by comparing administrative data to medical chart review in both VA1–3 and non-VA sources.4–6 In addition, our group validated the codes for hepatitis C virus (HCV) and HIV in a prior study.7 However, ICD-9 codes for cirrhosis and alcoholic liver disease (ALD) as well as hepatitis B virus (HBV) and HIV coinfection with HCV have not been validated in any administrative database to our knowledge. These are common disorders with high morbidity and mortality, and they have been the subject of numerous epidemiology and outcomes studies.7–9 In addition, these are important conditions for studies related to screening, diagnosis and treatment of patients with viral hepatitis and liver tumours. Therefore, the primary aim of this study was to validate the ICD-9 codes for cirrhosis among high-risk patients with liver disease, with a secondary aim to validate several associated conditions. To accomplish this, we evaluated the accuracy of diagnostic codes compared with the medical record for HCV and ALD among a population with ICD-9 codes for one or both of those conditions. Within that population, we validated codes for cirrhosis, HBV infection and HIV infection.

Methods

  1. Top of page
  2. Summary
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Supporting Information

Study setting

We conducted a retrospective study comparing the VA administrative data with data abstracted from the medical record in the Michael E. DeBakey VA Medical Center (MEDVAMC). The MEDVAMC, located in Houston, TX, is one of the largest facilities in the national VA Medical Center system. It serves 28 counties in south-east Texas and is the primary healthcare provider for more than 116 000 veterans.

Data sources

We tested the validity of ICD-9 codes in national administrative data obtained from the VA PTF and OPC databases. Since 1981, the VA PTF has recorded discharge diagnoses, as well as in-patient medical and surgical procedures, according to ICD-9.10 For each hospitalization, there are up to 10 discharge diagnosis codes, as well as demographic information.11 The codes are entered into the PTF by professional coders who base them on findings in the clinicians’ progress notes and laboratory test results of the electronic medical record.

The OPC contains information on each out-patient visit including up to 10 ICD-9 diagnosis codes beginning on 1 October 1996.11 In most cases, the providers code the out-patient encounters using an electronic encounter form, and approximately 20% are validated by professional coders.

We estimated the validity of select ICD-9 codes by comparing these codes with diagnoses derived from a comprehensive review of the patients’ medical records. The VA uses an electronic medical record system called the Computerized Patient Record System (CPRS), which resides at each facility. It is similar to a traditional paper chart and contains a problem list, pharmacy data, orders, laboratory results, progress notes, vital signs, radiology results, transcribed documents and reports from various studies such as endoscopy.12 In May 1995, there was a facility mandate for all progress notes to be in electronic form. Since 1999, CPRS was implemented and virtually all in-patient and out-patient medical records are available in this single electronic medical record for patients treated at MEDVAMC including electronic progress notes beginning in 1995.

Study population and definitions

We identified 5498 patients in VA PTF and OPC databases with at least one ICD-9 code for HCV or ALD at the MEDVAMC between September 1998 and July 2004 (see Table 1 for ICD-9 codes). We randomly selected 331 patients from this group (see Figure 1). Among patients with HCV or ALD, patients were further classified as having cirrhosis, HIV or HBV according to ICD-9 codes in the administrative data. This resulted in 12 ICD-9 code-defined strata, of which nine of the strata had more than 10 patients per strata and a random sample of a predetermined size was selected (see Table S1). The stratum sample sizes were chosen to ensure that each available combination of HCV, ALD and cirrhosis as well as HIV and HBV diagnosis codes was adequately represented in the data.

Table 1.   Conditions and corresponding ICD-9 codes
DiagnosisICD-9 codeMeaning
  1. * Older codes for HIV (043 and 044) were no longer in use by September 1998 and therefore were not included.

  2. ICD, International Classification of Diseases; HBV, hepatitis B virus; ALD, alcoholic liver disease; HCV, hepatitis C virus.

Cirrhosis571.2Alcoholic cirrhosis of liver
571.5Cirrhosis of liver without mention of alcohol
571.6Biliary cirrhosis
HCV070.41Acute or unspecified HCV with hepatic coma
070.44Chronic HCV with hepatic coma
070.51Acute or unspecified HCV without hepatic coma
070.54Chronic HCV without hepatic coma
V02.62Hepatitis C carrier
ALD571.0xAlcoholic fatty liver
571.1xAcute alcoholic hepatitis
571.3xAlcoholic liver damage, unspecified
571.2Alcoholic cirrhosis of liver
HIV*042.xxHIV disease
V08Asymptomatic HIV infection status
HBV070.2 (0 or 1)Acute viral hepatitis B with hepatic coma
070.2 (2 or 3)Chronic viral hepatitis B with hepatic coma
070.3 (0 or 1)Acute viral hepatitis B without hepatic coma
070.3 (2 or 3)Chronic viral hepatitis B without hepatic coma
V02.61Hepatitis B carrier
image

Figure 1.  Sampling flow charts for patients with an International Classification of Diseases-9 code for hepatitis C virus and/or alcoholic liver disease selected for validation study from the Michael E. DeBakey VA Medical Center in Houston, TX. Samples are not mutually exclusive.

Download figure to PowerPoint

Typically, the gold standard for diagnosing cirrhosis is the liver biopsy; however, not everyone in our study underwent this procedure. Given that there are clinical signs and symptoms frequently associated with cirrhosis, for the purposes of this study, we defined the gold standard as follows: either stage 4 cirrhosis on liver biopsy; any two of the following on ultrasound, computerized tomography scan or magnetic resonance imaging: cirrhosis, ascites, hepatocellular carcinoma (HCC) or portal hypertension; or the presence of at least two of the following conditions in the progress notes or on imaging: cirrhosis, ascites or peritonitis, varices with or without bleeding, HCC, hepatorenal syndrome, hepatic encephalopathy or two of the following laboratories in conjunction with one of the above markers: albumin <3.0, total bilirubin >2.0 or INR >1.2. HCV infection was defined as a positive HCV antibody test, detectable HCV-RNA in the blood or documentation of an HCV diagnosis in the progress notes of the medical record in CPRS. ALD was defined as reports in the progress note of ALD or heavy alcohol consumption and evidence of liver injury. The gold standard for HIV was a positive HIV antibody test, Western blot or detectable viral load. Finally, HBV infection was defined in two ways: active HBV infection denoted by a positive hepatitis B surface antigen (HBsAg) test, or any marker of HBV infection indicative of past infection or immunization such as a positive HBsAg, hepatitis B surface antibody, hepatitis B core antibody or documentation of any HBV infection in the progress notes. All conditions were assessed in both CPRS and the administrative databases from September 1998 to July 2004.

A trained clinician (EM) abstracted the medical record using a standardized, detailed abstraction form to determine the presence/absence of the above conditions. The abstractor was blinded to the OPC or PTF coding of these conditions. For quality control, a second clinician re-abstracted a 20% random sample, and a 10% random sample was reviewed by a senior clinician (HE). The study team discussed and resolved any discrepancies by consensus. Collected data included demographic information, presence of imaging tests and diagnoses, laboratory values and conditions documented in the progress notes.

Statistical analysis

The data collection forms were scanned using teleform Software (Cherry Valley, CA, USA) into a Microsoft Access 2000 database (Redmond, WA, USA), verified and cleaned, and subsequently analysed using statistical analysis system version 9.1 (SAS institute, Cary, NC, USA).

We calculated the positive predictive value (PPV) and negative predictive value (NPV) for ICD-9 codes in correctly identifying cirrhosis, HCV, ALD and HIV or HBV coinfection with HCV in the medical records. PPV indicates the probability that a patient with the ICD-9 code actually has that condition as defined by the medical record, or how well the code predicts the presence of the disease. NPV means the probability that a patient without the code actually is without the disease as defined by the medical record, or how well the absence of the code predicts the absence of the disease. We used patients without the condition of interest as determined by the medical record to calculate the NPV. A weighting scheme was used to correct for the skewed sampling. To obtain unbiased estimates of PPV and NPV, each subject was weighted by the inverse of the probability of being sampled under the stratified sampling scheme.

We also calculated the per cent agreement and kappa for each condition. Per cent agreement is the total of the two concordant cells (e.g. the proportion without both the code and condition plus the proportion with the code and the condition). Kappa was calculated in the standard manner, as the observational probability of agreement minus the hypothetical expected probability of agreement by chance, divided by one minus the expected probability of agreement by chance. According to Landis and Koch, a kappa >0.60 indicates substantial agreement and a kappa >0.80 indicates almost perfect agreement.13 We also examined the validity of the use of a single occurrence of an ICD-9 code as well as a coding algorithm (either one in-patient or two out-patient codes) for each condition.

Results

  1. Top of page
  2. Summary
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Supporting Information

Cirrhosis

The sample of 331 veterans comprised 98% males with a mean age as of the beginning of the study period (20 September 1998) of 49.3 years (s.d. = 8.6). By race/ethnicity, they were 52% white, 36% African-American, 11% Hispanic and 1% other. Table 2 highlights the diagnostic test characteristics of each group of codes. ICD-9 codes for cirrhosis were present in the administrative database in 25% of the total source population. In the entire sample (N = 331), the PPV (probability that cirrhosis recorded in administrative data is actually present according to gold standard) for any cirrhosis code was 90% and the NPV (probability that cirrhosis absent in administrative data is actually absent according to gold standard) was 87%. This yielded 88% agreement and kappa = 0.70, indicating substantial agreement. Among 257 patients with an ICD-9 code for HCV, the PPV and NPV of a cirrhosis code were 92% and 88%, with 89% agreement and kappa = 0.63. The values for the cirrhosis codes were virtually identical among 236 patients with confirmed HCV defined by positive enzyme-linked immunosorbent assay (ELISA) or polymerase chain reaction (PCR) or documentation in the medical chart (data not shown). In a subset of 183 patients with an ICD-9 code for ALD, the PPV and NPV for cirrhosis were 90% and 77%, respectively, with 86% agreement and kappa = 0.67. When restricting the population to 109 patients with codes for both HCV and ALD codes, the PPV was 93% and NPV was 86% with a kappa of 0.79, signifying that among patients with two major risk factors for liver disease, the presence/absence of the codes for cirrhosis is more likely to agree with what is in the medical record than patients with just one of the risk factors.

Table 2.   The PPV, NPV, per cent agreement and kappa of ICD-9 CM codes for identifying various liver disease conditions in VA administrative data compared to the medical records of 300 patients from the Michael E. DeBakey VA Medical Center
ICD-9 code definition sample populationMedical record definition PPV (%) NPV (%)Per cent agreementKappa
  1. PPV, positive predictive value; NPV, negative predictive value; ICD-9 CM, International Classification of Diseases, 9th revision, Clinical Modification; VA, Veterans Affairs; HCV, hepatitis C virus; ALD, alcoholic liver disease; HBV, hepatitis B virus; Ab, antibody; PCR, polymerase chain reaction; HBsAg, hepatitis B surface antigen; anti-HBs, hepatitis B surface antibody; HBc, hepatitis B core antibody.

Cirrhosis
 Total (n = 331)See Methods section for complete definition9087880.70
 Any code in patients with an  ICD-9 code for HCV (n = 257)9288890.63
 Any code in patients with an  ALD code (n = 183)9077860.67
 Any code in patients with both  ALD and HCV codes (n = 109)9386910.79
HCV
 Total (n = 331)HCV + Ab and/or PCR or HCV in notes9392920.78
ALD
 Total (n = 331)ALD in medical record7198890.74
 Total (using our coding algorithm,  see Methods)8396930.80
HIV coinfection
 Any code in patients with an ICD-9  code for HCV (n = 257)HIV + Ab or PCR 89100990.92
HBV coinfection
 Any code in patients with an ICD-9  code for HCV (n = 257)Active HBV (+HBsAg)4384810.17
Any marker of HBV infection (+HBsAg, +anti-HBs, +HBc or documentation of any HBV infection in the progress notes)6778770.23

More than half of the patients (57%) with a code for cirrhosis also had one of the complication codes (oesophageal varices with and without bleeding, ascites, hepatorenal syndrome and hepatic coma), while virtually no one (0.3%) without a cirrhosis code had a complication code. Therefore, the presence of codes for complications of cirrhosis only slightly increased the PPV of cirrhosis codes. Finally, we examined the accuracy of cirrhosis codes using a coding algorithm of requiring the occurrence of at least two out-patient or one in-patient ICD-9 code and there was little change in the parameters (data not shown).

HCV

Of the 331 patients in the study, 257 had at least one code for HCV in the VA administrative databases. Of these, 225 (88%) also had a positive ELISA or PCR test for HCV and an additional 12 (5%) had documentation of HCV in the progress notes. Of the remaining 20 (8%), three had no ELISA and PCR was negative, 12 had a negative ELISA and no PCR test and five had no documented testing with either ELISA or PCR. Among those with a code for HCV who received an ELISA test (n = 231, 90% of 257), 216 (94%) were HCV positive by ELISA and 15 were HCV negative. Of the 216 that were positive by ELISA, 121 (56%) received a PCR test of which 107 (88%) were HCV positive. Of the 15 that were negative by ELISA, two (13%) received a PCR test all of which were HCV positive.

Compared to the gold standard for HCV defined by the medical records, the PPV (probability that HCV infection is present among those with a code) for any HCV code was 93% and the NPV (probability that HCV infection is absent among those without a code) was 92%. This yielded 92% agreement and kappa = 0.78. We also examined using a coding algorithm (one in-patient or two out-patient codes) for HCV and all parameters, including the kappa, were lower (data not shown).

ALD

Of the 331 patients, 183 were selected to have at least one ALD code. When using one code to define ALD, the PPV was 71% and the NPV was 98% (see Table 2). This indicates that 71% of patients with a code for ALD actually have the disease while almost all of the patients without a code also did not have ALD. Using the coding algorithm requiring at least one in-patient or two out-patient ALD codes improved the PPV to 83% with 93% agreement, while the NPV stayed above 95% (see Table 2). The agreement was >89% and the kappa denotes substantial agreement for both comparisons.

HIV or HBV coinfection with HCV

When restricting the population to the 257 patients with HCV defined by ICD-9 codes, all the validity parameters for HIV codes were >89%, indicating almost perfect agreement of the code and the presence of HIV documented in the medical record. For HBV coinfection with HCV, the validity parameters were low. For example, compared to a gold standard of a positive HBsAg test indicating active HBV infection, the PPV for having any HBV code was 43% and NPV was 84%. The PPV improved only to 67% when examining the validity of predicting any positive marker, active or cleared, of HBV infection, despite the fact that the ICD-9 codes specify active infection (see Table 1). The kappa for both these comparisons was quite low at 0.17 for positive HBsAg test and 0.23 for any marker of HBV infection as the gold standard.

Discussion

  1. Top of page
  2. Summary
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Supporting Information

To our knowledge, this is the first study to validate systematically the VA’s in-patient and out-patient databases for several different codes relating to chronic liver disease including cirrhosis, HCV, ALD and HIV or HBV coinfection with HCV. The cirrhosis and HCV codes demonstrated excellent PPV (>90%) and NPV ranging from 77% to 92%. In the case of ALD, for which the PPV was lower, we used the coding algorithm that raised it to an acceptable value (83%). The agreement was >86% for these conditions; the kappa values indicated moderate to substantial agreement for cirrhosis, HCV and ALD, with almost perfect agreement for HIV coinfection with HCV. Finally, PPV for HBV coinfection with HCV was low at 43% and improved some when using any positive marker for HBV as the gold standard.

Results of this study are useful in several ways. First, they clarify any uncertainty concerning the validity of these codes, which will help researchers interested in using administrative databases to examine questions regarding patients with viral hepatitis and chronic liver disease. Secondly, we examined the validity of the cirrhosis codes in the HCV population defined in two different ways, which may be helpful for different groups of researchers. Defining HCV with ICD-9 codes applies to researchers using the PTF and OPC databases to define their HCV cohort, whereas using the laboratory test results to define the HCV population is particularly relevant to researchers interested in using the new Clinical Case Registry, a National VA HCV registry in which laboratory test results are available to confirm HCV infection.

Results for HCV and HIV were similar to those we obtained in a previous validation study for these two conditions using a different random sample.7 All patients in this study had to have codes for liver disease, whereas the previous study chose a random sample with HCV, HIV or neither virus. In that study, we found the presence of an HIV ICD-9 code was 98% predictive of a positive HIV laboratory test, while the absence of the code was 100% predictive of the absence of a positive HIV test. The presence of an HCV code was 94% predictive of a positive HCV laboratory test, while the absence of a code was 90% predictive of the absence of a positive test. The PPV and NPV for both HCV and HIV are comparable to this study.

While the absence of a code for HBV was highly predictive of the absence of infection, the presence of a code did not accurately predict infection. In particular, patients with active HBV infection were poorly identified by the ICD-9 codes for HBV. To give coders and practitioners the benefit of the doubt, we included all markers of HBV in the gold standard definition, thus allowing them to code for active HBV infection even when the laboratory test only indicates prior immunization (i.e. positive hepatitis B surface antibody). With this scenario, the PPV increased slightly, but it was still low (<70%) with the kappa indicating poor agreement. Caution should be exercised before using these codes to identify patients with HBV, but they may be used to identify accurately patients without the condition.

Several studies have been conducted to identify risk factors for cirrhosis in patients with HCV. Two of these specifically examined the role of HIV coinfection, using VA administrative databases.7, 9 The results of this validation study support the conclusions drawn in these previous studies.

There are several limitations of this study. First, it was conducted in a single centre of the VA healthcare system and may not reflect the coding practices of other VA sites or non-VA databases. However, MEDVAMC in Houston, TX is one of the largest medical centres of the VA, and the study sample included out-patient clinics in Lufkin and Beaumont, TX. Also, the agreement between the PTF and medical records for in-patient diagnoses was previously established nationally for other conditions, thus minimizing the likelihood of this bias.2 Secondly, we assessed the presence/absence of the codes and clinical findings during the study period, but not the sequence in which they occurred. Some conditions may have been improperly coded before the documentation of the relevant clinical findings, resulting in a higher estimate of the PPV for some of the codes. However, this bias was likely minimal, as ICD-9 codes generally are derived from documentation of clinical findings. Thirdly, we used patients with liver disease (defined as HCV or ALD) without the condition of interest as the comparison group for validating HCV and ALD codes. Finally, it is important to note that this study validated how well the assigned ICD-9 codes matched the diagnosis indicated by the medical record, but we did not assess the reliability of the information provided in the medical record itself, a bias inherent to any study using medical records.

This study demonstrates that the diagnostic codes for cirrhosis, HCV, ALD and HIV coinfection with HCV in VA databases are highly predictive of the presence of these conditions in the medical record and the absence of theses codes is highly predictive of the absence of the conditions. Agreement and kappa were high for all four conditions. Therefore, these ICD-9 codes can be reliably used in VA administrative databases for research, especially for accurately identifying patients with these conditions; however, other means of identifying patients with HBV coinfection should be explored.

Acknowledgements

  1. Top of page
  2. Summary
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Supporting Information

Declaration of personal interests: Dr El-Serag has received funding from Schering-Plough Corporation. Dr Giordano has received funding from Tiobotec Pharmaceuticals and Pfizer. Declaration of funding interests: This study was funded in part by Health Services Research and Development Service, Office of Research and Development, Department of Veterans Affairs, grant MRP05-315 to Dr Kramer, MRPO3-134-1 to Dr Davila, and grant RCD00-013-2 to Dr El-Serag. In addition, part of this research was supported with funding from Schering-Plough Corporation (Kenilworth, NJ, USA).

References

  1. Top of page
  2. Summary
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Supporting Information
  • 1
    Abraham NS, Cohen DC, Rivers B, Richardson P. Validation of administrative data used for the diagnosis of upper gastrointestinal events following nonsteroidal anti-inflammatory drug prescription. Aliment Pharmacol Ther 2006; 24: 299306.
  • 2
    Kashner TMPJD. Agreement between administrative files and written medical records: a case of the Department of Veterans Affairs. Med Care 1998; 36: 132436.
  • 3
    Petersen LA, Wright S, Normand SL, Daley J. Positive predictive value of the diagnosis of acute myocardial infarction in an administrative database. J Gen Intern Med 1999; 14: 5558.
  • 4
    Andrade SE, Gurwitz JH, Chan KA, et al. Validation of diagnoses of peptic ulcers and bleeding from administrative databases: a multi-health maintenance organization study. J Clin Epidemiol 2002; 55: 3103.
  • 5
    Cattaruzzi C, Troncon MG, Agostinis L, Garcia Rodriguez LA. Positive predictive value of ICD-9th codes for upper gastrointestinal bleeding and perforation in the Sistema Informativo Sanitario Regionale database. J Clin Epidemiol 1999; 52: 499502.
  • 6
    Van WC, Wang B, Ugnat AM, Naylor CD. False-positive coding for acute myocardial infarction on hospital discharge records: chart audit results from a tertiary centre. Can J Cardiol 1990; 6: 3836.
  • 7
    Kramer JR, Giordano TP, Richardson P, Souchek J, Hwang LY, El-Serag HB. The effect of HIV coinfection on the risk of cirrhosis and hepatocellular carcinoma in US veterans with hepatitis C. Am J Gastroenterol 2005; 100: 5663.
    Direct Link:
  • 8
    Butt AA, Fultz SL, Kwoh CK, Kelley D, Skanderson M, Justice AC. Risk of diabetes in HIV infected veterans pre- and post-HAART and the role of HCV coinfection. Hepatology 2004; 40: 1159.
  • 9
    Giordano TP, Kramer JR, Souchek J, Richardson P, El Serag HB. Cirrhosis and hepatocellular carcinoma in HIV-infected veterans with and without the hepatitis C virus: a cohort study, 1992-2001. Arch Intern Med 2004; 164: 234954.
  • 10
    Medicode. The International Classification of Diseases, 9th Revision, Clinical Modification. Salt Lake City, UT: Medicode, 2001.
  • 11
    Boyko EJ, Koepsell TD, Gaziano JM, Horner RD, Feussner JR. US Department of Veterans Affairs medical care system as a resource to epidemiologists. Am J Epidemiol 2000; 151: 30714.
  • 12
    Brown SH, Lincoln MJ, Groen PJ, Kolodner RM VistA – U.S. Department of Veterans Affairs national-scale HIS. Int J Med Inform 2003; 69:13556.
  • 13
    Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 15974.

Supporting Information

  1. Top of page
  2. Summary
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Supporting Information

Table S1. Sixteen strata used to determine validation sample of 331 subjects from MEDVAMC in Houston, TX

Please note: Blackwell Publishing are not responsible for the content or functionality of any supplementary materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.

FilenameFormatSizeDescription
APT_3572_sm_TableS1.doc43KSupporting info item
APT_tables1.doc43KSupporting info item

Please note: Wiley Blackwell is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.