Contraction of blood clots plays an important role in blood clotting, a natural process that restores hemostasis and regulates thrombosis in the body. Upon injury, a chain of events culminate in the formation of a soft plug of cells and fibrin fibers attaching to wound edges. Platelets become activated and apply contractile forces to shrink the overall clot size, modify clot structure, and mechanically stabilize the clot. Impaired blood clot contraction results in unhealthy volumetric, mechanical, and structural properties of blood clots associated with a range of severe medical conditions for patients with bleeding and thrombotic disorders. Due to the inherent mechanical complexity of blood clots and a confluence of multiple interdependent factors governing clot contraction, the mechanics and dynamics of clot contraction and the interactions with red blood cells (RBCs) remain elusive. Using an experimentally informed, physics-based mesoscale computational model, we probe the dynamic interactions among platelets, fibrin polymers, and RBCs, and examine the properties of contracted blood clots. Our simulations confirm that RBCs strongly affect clot contraction. We find that RBC retention and compaction in thrombi can be solely a result of mechanistic contraction of fibrin mesh due to platelet activity. Retention of RBCs hinders clot contraction and reduces clot contractility. Expulsion of RBCs located closer to clot outer surface results in the development of a dense fibrin shell in thrombus clots commonly observed in experiments. Our simulations identify the essential parameters and interactions that control blood clot contraction process, highlighting its dependence on platelet concentration and the initial clot size. Furthermore, our computational model can serve as a useful tool in clinically relevant studies of hemostasis and thrombosis disorders, and post thrombotic clot lysis, deformation, and breaking.