Epigenetic inheritance is more widespread in plants than in mammals, in part because mammals erase epigenetic information by germline reprogramming. We sequenced the methylome of three haploid cell types from developing pollen: the sperm cell, the vegetative cell, and their precursor, the postmeiotic microspore, and found that unlike in mammals the plant germline retains CG and CHG DNA methylation. However, CHH methylation is lost from retrotransposons in microspores and sperm cells and restored by de novo DNA methyltransferase guided by 24 nt small interfering RNA, both in the vegetative nucleus and in the embryo after fertilization. In the vegetative nucleus, CG methylation is lost from targets of DEMETER (DME), REPRESSOR OF SILENCING 1 (ROS1), and their homologs, which include imprinted loci and recurrent epialleles that accumulate corresponding small RNA and are premethylated in sperm. Thus genome reprogramming in pollen contributes to epigenetic inheritance, transposon silencing, and imprinting, guided by small RNA.