Modified nucleoside triphosphates (NTPs) are invaluable tools to probe bacterial enzymatic mechanisms, develop novel genetic material, and engineer drugs and proteins with new functionalities. Although the impact of nucleobase alterations has predominantly been studied due to their importance for protein recognition, sugar and phosphate modifications have also been investigated. However, NTPs are cell impermeable due to their negatively charged phosphate tail, a major hurdle to achieving live bacterial studies. Herein, we review the recent advances made to investigate and evolve bacteria and their processes with the use of modified NTPs by exploring alterations in one of the three moieties: the nucleobase, the sugar and the phosphate tail. We also present the innovative methods that have been devised to internalize NTPs into bacteria for in vivo applications.