We describe a technique to model the radiative properties of mineral aerosols which accounts for their composition. We compile a data set of refractive indices of major minerals and employ it, along with data on mineralogical composition of dust from various locations, to calculate spectral optical and radiative properties of mineral aerosol mixtures. Such radiative properties are needed for climate modeling and remote sensing applications. We consider external mixtures of individual minerals, as well as mixtures of aggregates. We demonstrate that an external mixture of individual minerals must contain unrealistically high amounts of hematite to have a single scattering albedo lower than 0.9 at 500 nm wavelength. In contrast, aggregation of hematite with quartz or clays can strongly enhance absorption by dust at solar wavelengths. We also simulate the daily mean net (solar + infrared) forcing by dust of varying compositions. We found that, for a given composition and under similar atmospheric conditions, a mixture of aggregates can cause the positive radiative forcing while a mixture of individual minerals gives the negative forcing.