In tropical forests, insect herbivores exert significant pressure on plant populations. Adaptation to such pressure is hypothesized to be a driver of high tropical diversity, but direct evidence for local adaptation to herbivory in tropical forests is sparse. At the same time, herbivore pressure has been hypothesized to increase with rainfall in the tropics, which could lead to differences among sites in the degree of local adaptation. To assess the presence of local adaptation and its interaction with rainfall, we compared herbivore damage on seedlings of local vs. nonlocal populations at sites differing in moisture availability in a reciprocal transplant experiment spanning a rainfall gradient in Panama. For 13 native tree species, seeds collected from multiple populations along the rainfall gradient were germinated in a shadehouse and then transplanted to experimental sites within the species range. We tracked the proportion of seedlings attacked over 1.5 yr and quantified the percentage of leaf area damaged at the end of the study. Seedlings originating from local populations were less likely to be attacked and experienced lower amounts of herbivore damage than those from nonlocal populations, but only on the wetter end of the rainfall gradient. However, overall herbivore damage was higher at the drier site compared to wetter sites, contrary to expectation. Taken together, these findings support the idea that herbivory can result in local adaptation within tropical tree species; however, the likelihood of local adaptation varies among sites because of environmentally driven differences in investment in defense or herbivore specialization or both.