Translation inhibition is a major but poorly understood mode of action of microRNAs (miRNAs) in plants and animals. In particular, the subcellular location where this process takes place is unknown. Here, we show that the translation inhibition, but not the mRNA cleavage activity, of Arabidopsis miRNAs requires ALTERED MERISTEM PROGRAM1 (AMP1). AMP1 encodes an integral membrane protein associated with endoplasmic reticulum (ER) and ARGONAUTE1, the miRNA effector and a peripheral ER membrane protein. Large differences in polysome association of miRNA target RNAs are found between wild-type and the amp1 mutant for membrane-bound, but not total, polysomes. This, together with AMP1-independent recruitment of miRNA target transcripts to membrane fractions, shows that miRNAs inhibit the translation of target RNAs on the ER. This study demonstrates that translation inhibition is an important activity of plant miRNAs, reveals the subcellular location of this activity, and uncovers a previously unknown function of the ER.