For the detection of a moving object, segregating the object from the background is a necessary first step. This segregation can be achieved by detection of differences in the spatial, temporal and spatio-temporal properties of the object and background. Here we investigate how flicker influences the perception of a moving object in man and monkey, and we examine the neuronal responses in extrastriate medial temporal and medial superior temporal areas (MT and MST) of two rhesus monkeys. The performance of humans and monkeys in a direction discrimination task was impaired in the presence of flicker in the background compared to the static background condition. A similar effect was found in recordings from 155 single units in areas MT and MST during the discrimination task. The discriminability (d′) of the neuronal responses in preferred and nonpreferred directions was reduced by 33% on average in the presence of a flicker background compared to the static background. This reduction in discriminability was not caused by differences in variance of the neuronal activity for the two background conditions, but was due to a reduction of the difference between the activities in preferred and nonpreferred direction. This reduction in directional selectivity could be traced back to two different mechanisms: in 32 out of 155 neurons (21%), the decrease resulted from an increase in the response to the stimulus moving in the nonpreferred direction; in 62 out of 155 neurons (40%), the reduction in directional selectivity was due to a decrease in the response to the preferred direction. These results give deeper insights into how moving stimuli are processed in the presence of background flicker as present in natural visual scenes.