While ultrasound guided prostate brachytherapy has gained wide acceptance as a primary treatment tool for prostate cancer, quality assurance of the ultrasound guidance system has received very little attention. Task Group 128 of the American Association of Physicists in Medicine was created to address quality assurance requirements specific to transrectal ultrasound used for guidance of prostate brachytherapy. Accurate imaging guidance and dosimetry calculation depend upon the quality and accuracy of the ultrasound image. Therefore, a robust quality assurance program for the ultrasound system is essential. A brief review of prostate brachytherapy and ultrasound physics is provided, followed by a recommendation for elements to be included in a comprehensive test phantom. Specific test recommendations are presented, covering grayscale visibility, depth of penetration, axial and lateral resolution, distance measurement, area measurement, volume measurement, needle template/electronic grid alignment, and geometric consistency with the treatment planning computer.