We present an all-optical method for structure assignment of individual single-walled carbon nanotubes (SWNTs) based on combined measurements of the elastically (Rayleigh) and inelastically (Raman) scattered light by individual freestanding specimens. The optical resonances of a single SWNT are identified from the Rayleigh spectra. These measurements are complemented with an independent estimation of the SWNT diameter and chiral angle from its radial breathing and G Raman modes, respectively. An assignment of the (n,m) chiral indices can thus be provided. In the most simple cases, a reliable structure assignment can be inferred from the Rayleigh spectra alone. However, we also show that SWNTs with different structures may display similar Rayleigh spectra within a given photon energy window. In these situations, Raman measurements provide unambiguous criteria for a proper structure assignment. This approach is particularly helpful in order to identify the high-symmetry armchair and zig-zag structures.