Poly(vinyl alcohol) (PVA) composites with incorporation of Ag nanoparticle–reduced graphene oxide (Ag–rGO) hybrids as fillers were prepared and the thermal conductivity of the resulting Ag–rGO/PVA composites was investigated. Ag–rGO hybrids were fabricated by a chemical reduction method, and their morphology, crystal, and chemistry were characterized. Benefiting from the unique characteristic of Ag–rGO hybrids, the Ag–rGO/PVA composites showed a 153% increase in thermal conductivity with addition of 1.0 vol% Ag–rGO, which was much superior to the composites with bare rGO. The significant thermal conductivity enhancement was probably attributed to the combined factors of homogeneous filler dispersion, enhanced interfacial interaction, and “bridge” effect of Ag NPs for establishing thermally conductive networks. Therefore, our study shows that the Ag–rGO hybrids have the potential to be successfully used as the nanofillers of polymer composites for thermal management applications.