We use an individual-based simulation model to investigate factors influencing progress toward ecological speciation. We find that environmental differences can quickly lead to the evolution of substantial reproductive barriers between a population colonizing a new environment and the ancestral population in the old environment. Natural selection against immigrants and hybrids was a major contributor to this isolation, but the evolution of sexual preference was also important. Increasing dispersal had both positive and negative effects on population size in the new environment and had positive effects on natural selection against immigrants and hybrids. Genetic divergence at unlinked, neutral genetic markers was low, except when environmental differences were large and sexual preference was present. Our results highlight the importance of divergent selection and adaptive divergence for ecological speciation. At the same time, they reveal several interesting nonlinearities in interactions between environmental differences, sexual preference, dispersal and population size.