The aim of this work is to understand how face processing develops from birth to 4 months by modelling some face processing abilities of infants between these ages. The neural model reproduces experimental data regarding both visual preferences at birth and right hemisphere behaviour in 4-month-old infants. The model shows that two basic properties may be sufficient to simulate face preference at birth: (a) tuning selectivity of visual neurons for spatial frequencies and (b) limited vision (CSF) of newborns. Three additional properties are sufficient to match experimental data on face processing at 4 months: (c) plasticity of lateral connections as modelled by Bayesian networks, (d) amplification of eye importance and (e) mother bias induced by the high probability of seeing mother’s face. These conditions suggest that the neural substratum for face processing could be, at birth, the retina-V1/V2-colliculus system, and at 4 months, the activation of the RH face-Fusiform area, with configural representations stored in lateral connections.