Computational methods to study protein-ligand interactions at a molecular level have been successful to a certain extent in predicting the pose, atomic interactions, and so forth, but poor efficiency in estimating a protein-ligand binding affinity is still a crucial problem to be solved. Analyzing the protein-ligand interactions quantitatively is one primary concern for understanding. Qualitative analysis of these interactions may lead to better insights about protein-ligand interactions. To perform such an analysis, the macroscopic molecular properties of the protein and ligand can be studied in detail and should be correlated with the ligand-binding affinity. This detailed study can be helpful in designing the ligands and the ligand-binding site as well. In this study, we attempted to identify the hydrophobic/hydrophilic features of a ligand and ligand-binding site and check their correlation with the experimental affinity of the protein-ligand complexes. This combinatorial analysis of ligand log P and binding site hydrophobicity on data set distribution and binding affinity suggested two critical findings. The hydrophobic ligands bind to hydrophilic and hydrophobic pockets equally, whereas hydrophilic ligands are specific to hydrophilic pockets. The combination of the hydrophobic ligand-hydrophobic pocket prefers high-affinity values compared to other combinations. Although these results cannot be used for atomic-level design of ligands or binding sites, they are expected to be used as a reference for screening the ligands for a given target binding site.