The photoluminescence of Si nanocrystals is often assigned to exciton recombination confined within the nanocrystals. However, localized radiative or non-radiative defects at the nanocrystal (NC) interface can drastically influence the NCs ensemble and reduce the luminescence output. In addition, size of the NCs as well as embedding host are playing a major role and have a significant influence on the optical properties. Here we summarize our work on size controlled Si NCs based on the SiOx/SiO2 superlattice approach. We will discuss how to distinguish between defects at the NC-Si/SiO2 interface and defects in the matrix and demonstrate ways for effective NC surface passivation. The influence of different annealing ambient used for crystallization will be shown. Such perfectly defect passivated and size controlled samples will be used as a model system to discuss size dependent quantum properties such as the temperature dependent blue shift of the Si NC band gap.