Previous studies have focused on the subpopulations of tumor-infiltrating lymphocytes (TILs) in tumors. This study focuses only on the concentration of TILs in the tumor irrespective of type and elucidates its prognostic value. We used 315 HCC patients as the discovery phase and another 343 HCC patients as the validation phase. By following the standardized guideline, density of TILs were categorized into low (TILs < 10%), intermediate (10% ≦ TILs < 50%), and high (TILs ≧ 50%) levels. Associations of TILs with prognostic, immune-related, and genetic variables were examined. We observed a dose-response relation of TILs with overall survival (intermediate: HR, 0.58; 95% confidence interval (CI), 0.36-0.93; high: HR, 0.37; 95% CI, 0.15-0.93) and disease-free survival (intermediate: HR, 0.35; 95% CI, 0.22-0.58; high: HR, 0.23; 95% CI, 0.09-0.58). The prognostic value of TILs was validated in the TCGA set. Mutation burden or the number of neoantigens were not associated with TILs intensity. However, hepatitis B or C virus infection patients had higher TILs intensity in the para-tumor tissue. The TILs intensity was associated with patients' survival. If confirmed, this would suggest that clinical routine assessment of TILs could provide prognostic information in HCC.