The low rate of response to antidepressants in treatment resistant depression (TRD) justifies studies of next-step therapies following a treatment failure. In TRD clinical trials, it is important to verify the accurate diagnosis of treatment resistance for all enrolled subjects using a reliable and valid instrument. Self-rated scales can reduce the impact of investigator bias and reduce the time burden for clinical researchers. The Massachusetts General Hospital (MGH) Antidepressant Treatment Response Questionnaire (ATRQ) is a self-rated scale used to determine treatment resistance in major depressive disorder (MDD). The ADAPT-A study is a multi-center double-blind, placebo-controlled study of low-dose aripiprazole adjunctive to ADT among outpatients with TRD. At the screening assessment, potential subjects completed the MGH ATRQ. The ADAPT-A medical monitors subsequently performed remote patient interviews and obtained detailed medication histories. The data obtained from the MGH ATRQ and by the medical monitors were compared for congruency. Of the 186 patients enrolled by the local sites, no subjects deemed treatment resistant by the MGH ATRQ were found to be nonresistant by the medical monitors. In 76.3% (n = 142) of the subjects, the number of failed adequate antidepressant trials reported by the MGH ATRQ was concordant with the data collected by medical monitors. In 16.1% (n = 30) of all cases, the medical monitors found a greater number of failed trials; in 7.5% (n = 14) of cases, the medical monitors found fewer failed medication trials. The discrepancy was by more than one medication trial in only 4.0% (n = 7) of cases. We found the MGH ATRQ to be relatively concordant in its assessment of treatment resistance in depression compared with independent clinical researchers. Although the MGH ATRQ tended to underreport the number of unsuccessful treatment trials relative to the clinical interviews, its accuracy in cases it detected was confirmed by raters.