Formate dehydrogenase H, FDH(Se), from Escherichia coli contains a molybdopterin guanine dinucleotide cofactor and a selenocysteine residue in the polypeptide. Oxidation of 13C-labeled formate in 18O-enriched water catalyzed by FDH(Se) produces 13CO2 gas that contains no 18O-label, establishing that the enzyme is not a member of the large class of Mo-pterin-containing oxotransferases which incorporate oxygen from water into product. An unusual Mo center of the active site is coordinated in the reduced Mo(IV) state in a square pyramidal geometry to the four equatorial dithiolene sulfur atoms from a pair of pterin cofactors and a Se atom of the selenocysteine-140 residue [Boyington, J. C., Gladyshev, V. N., Khangulov, S. V., Stadtman, T. C., and Sun, P. D. (1997) Science 275, 1305-1308]. EPR spectroscopy of the Mo(V) state indicates a square pyramidal geometry analogous to that of the Mo(IV) center. The strongest ligand field component is likely the single axial Se atom producing a ground orbital configuration Mo(dxy). The Mo-Se bond was estimated to be covalent to the extent of 17-27% of the unpaired electron spin density residing in the valence 4s and 4p selenium orbitals, based on comparison of the scalar and dipolar hyperfine components to atomic 77Se. Two electron oxidation of formate by the Mo(VI) state converts Mo to the reduced Mo(IV) state with the formate proton, Hf+, transferring to a nearby base Y-. Transfer of one electron to the Fe4S4 center converts Mo(IV) to the EPR detectable Mo(V) state. The Y- is located within magnetic contact to the [Mo-Se] center, as shown by its strong dipolar 1Hf hyperfine couplings. Photolysis of the formate-induced Mo(V) state abolishes the 1Hf hyperfine splitting from YHf, suggesting photoisomerizaton of this group or phototransfer of the proton to a more distant proton acceptor group A-. The minor effect of photolysis on the 77Se-hyperfine interaction with [77Se] selenocysteine suggests that the Y- group is not the Se atom, but instead might be the imidazole ring of the His141 residue which is located in the putative substrate-binding pocket close to the [Mo-Se] center. We propose that the transfer of Hf+ from formate to the active site base Y- is thermodynamically coupled to two-electron oxidation of the formate molecule, thereby facilitating formation of CO2. Under normal physiological conditions, when electron flow is not limited by the terminal acceptor of electrons, the energy released upon oxidation of Mo(IV) centers by the Fe4S4 is used for deprotonation of YHf and transfer of Hf+ against the thermodynamic potential.