We proposed a method for polarization ghost imaging based on single photon counting. With the time-correlated single-photon-counting technique, we can construct photon time distribution histograms and select a distance gate to accurately estimate the light intensity. Experiments are performed to realize discrimination of the object from the background of different materials in weak illumination. In the situation that ambient noise is significantly stronger than the signal, our method still can retrieve an image as ambient noise is mostly filtered out through distance gate selection. We suppose that our method may facilitate applications in remote target discrimination and biological imaging.