Mental retardation (MR) is a group of common and complex disabilities affecting the central nervous system and appears before the period of brain developmental maturity. Recently, only 40% of genetic MR has been identified, however 60% remains unexplained. In this study, we applied exome sequencing to identify the mutation p.R430X in UPF3B gene in an MR pedigree, which was validated by Sanger sequencing and completely cosegregated within this family. UPF3B gene encodes a protein involved in nonsense-mediated mRNA decay (NMD). By real-time quantitative PCR, we detected the significant difference in the mRNA expression levels of the UPF3B and the classical NMD pathway target growth arrest and DNA-damage-inducible-beta (GADD45B) between the patients and the controls. Our results directly implicated that the mutation p.R430X in UPF3B gene was the genetic etiology of the MR pedigree.