Sensory systems continually adjust the way stimuli are processed. What are the circuit mechanisms underlying this plasticity? We investigated how synapses in the retina of zebrafish adjust to changes in the temporal contrast of a visual stimulus by imaging activity in vivo. Following an increase in contrast, bipolar cell synapses with strong initial responses depressed, whereas synapses with weak initial responses facilitated. Depression and facilitation predominated in different strata of the inner retina, where bipolar cell output was anticorrelated with the activity of amacrine cell synapses providing inhibitory feedback. Pharmacological block of GABAergic feedback converted facilitating bipolar cell synapses into depressing ones. These results indicate that depression intrinsic to bipolar cell synapses causes adaptation of the ganglion cell response to contrast, whereas depression in amacrine cell synapses causes sensitization. Distinct microcircuits segregating to different layers of the retina can cause simultaneous increases or decreases in the gain of neural responses.