This paper describes the matrix effect during the analysis of ten antibiotic compounds in water by SPE followed by HPLC-ESI-MS/MS. The target analytes were tetracycline, oxytetracycline (tetracyclines), sulfathiazole, sulfamethazine, sulfadiazine (sulfonamides), erythromycin-H2O, roxithromycine, spiramycin (macrolides), ofloxacin, and norfloxacin (quinolones). The matrix effect was examined for internal standards and the target analytes in five different water matrixes, with signal suppression being increased in the order: ultrapure water, tap water, river water, sewage effluent, and sewage influent. A combined application of the internal standards and matrix-matched extract calibration was shown to be successful in compensating the matrix effect for the analytes. The procedural recovery of the target compounds in sewage effluents and influents was higher than in river water samples, which was further enhanced by sample acidification to pH 2. The validity of the internal standard based matrix-matched calibration approach was verified by the standard addition method.