Phosphorus (P) is a vital nutrient for all living organisms and may control the growth of bacteria in the ocean. Bacteria induce alkaline phosphatases when inorganic phosphate (P(i)) is insufficient to meet their P-requirements, and therefore bulk alkaline phosphatase activity measurements have been used to assess the P-status of microbial assemblages. In this study, the molecular basis of marine bacterial phosphatases and their potential role in the environment were investigated. We found that only a limited number of homologs to the classical Escherichia coli alkaline phosphatase (PhoA) were present in marine isolates in the Bacteroidetes and gamma-proteobacteria lineages. In contrast, PhoX, a recently described phosphatase, was widely distributed among diverse bacterial taxa, including Cyanobacteria, and frequently found in the marine metagenomic Global Ocean Survey database. These taxa included ecologically important groups such as Roseobacter and Trichodesmium. PhoX was induced solely upon P-starvation and accounted for approximately 90% of the phosphatase activity in the model marine bacterium Silicibacter pomeroyi. Analysis of the available transcriptomic datasets and their corresponding metagenomes indicated that PhoX is more abundant than PhoA in oligotrophic marine environments such as the North Pacific Subtropical Gyre. Those analyses also revealed that PhoA may be important when Bacteroidetes are abundant, such as in algal bloom episodes. However, PhoX appears to be much more widespread. Its identification as a gene that mediates organic P acquisition in ecologically important groups, and as a marker of P(i)-stress, constitutes an important step toward a better understanding of the marine P cycle.