Alzheimer’s disease (AD) is an age-related, progressive degenerative disorder that is characterized by synapse and neuron loss in the brain and the accumulation of protein-containing deposits (referred to as ‘senile plaques’) and neurofibrillary tangles. Insoluble amyloid β-peptide (Aβ) fibrillar aggregates found in extracellular plaques have long been thought to cause the neurodegenerative cascades of AD. However, accumulating evidence suggests that prefibrillar soluble Aβ oligomers induce AD-related synaptic dysfunction. The size of Aβ oligomers is distributed over a wide molecular weight range (from < 10 kDa to > 100 kDa), with structural polymorphism in Aβ oligomers of similar sizes. Recent studies have demonstrated that Aβ can accumulate in living cells, as well as in extracellular spaces. This review summarizes current research on Aβ oligomers, focusing on their structures and toxicity mechanism. We also discuss possible formation mechanisms of intracellular and extracellular Aβ oligomers.