Cross-linked rather than non-covalently bonded graphitic carbon nitride (g-C3N4)/reduced graphene oxide (rGO) nanocomposites with tunable band structures have been successfully fabricated by thermal treatment of a mixture of cyanamide and graphene oxide with different weight ratios. The experimental results indicate that compared to pure g-C3N4, the fabricated CN/rGO nanocomposites show narrowed bandgaps with an increased in the rGO ratio. Furthermore, the band structure of the CN/rGO nanocomposites can be readily tuned by simply controlling the weight ratio of the rGO. It is found that an appropriate rGO ratio in nanocomposite leads to a noticeable positively shifted valence band edge potential, meaning an increased oxidation power. The tunable band structure of the CN/rGO nanocomposites can be ascribed to the formation of C−O−C covalent bonding between the rGO and g-C3N4 layers, which is experimentally confirmed by Fourier transform infrared (FT-IR) and X-ray photoelectron (XPS) data. The resulting nanocomposites are evaluated as photocatalysts by photocatalytic degradation of rhodamine B (RhB) and 4-nitrophenol under visible light irradiation (λ > 400 nm). The results demonstrate that the photocatalytic activities of the CN/rGO nanocomposites are strongly influenced by rGO ratio. With a rGO ratio of 2.5%, the CN/rGO-2.5% nanocomposite exhibits the highest photocatalytic efficiency, which is almost 3.0 and 2.7 times that of pure g-C3N4 toward photocatalytic degradation of RhB and 4-nitrophenol, respectively. This improved photocatalytic activity could be attributed to the improved visible light utilization, oxidation power, and electron transport property, due to the significantly narrowed bandgap, positively shifted valence band-edge potential, and enhanced electronic conductivity.