Small water clusters containing a single hydroxyl radical are synthesized in liquid helium droplets. The OH-H2O and OH(D2O)n clusters (n = 1-3) are probed with infrared laser spectroscopy in the vicinity of the hydroxyl radical OH stretch vibration. Experimental band origins are qualitatively consistent with ab initio calculations of the global minimum structures; however, frequency shifts from isolated OH are significantly over-predicted by both B3LYP and MP2 methods. An effective Hamiltonian that accounts for partial quenching of electronic angular momentum is used to analyze Stark spectra of the OH-H2O and OH-D2O binary complexes, revealing a 3.70(5) D permanent electric dipole moment. Computations of the dipole moment are in good agreement with experiment when large-amplitude vibrational averaging is taken into account. Polarization spectroscopy is employed to characterize two vibrational bands assigned to OH(D2O)2, revealing two nearly isoenergetic cyclic isomers that differ in the orientation of the non-hydrogen-bonded deuterium atoms relative to the plane of the three oxygen atoms. The dipole moments for these clusters are determined to be approximately 2.5 and 1.8 D for "up-up" and "up-down" structures, respectively. Hydroxyl stretching bands of larger clusters containing three or more D2O molecules are observed shifted approximately 300 cm(-1) to the red of the isolated OH radical. Pressure dependence studies and ab initio calculations imply the presence of multiple cyclic isomers of OH(D2O)3.