Extracellular vesicles (EVs) play major roles in intracellular communication and participate in several biological functions in both normal and pathological conditions. Surface modification of EVs via various ligands, such as proteins, peptides, or aptamers, offers great potential as a means to achieve targeted delivery of therapeutic cargo, i.e., in drug delivery systems (DDS). This review summarizes recent studies pertaining to the development of EV-based DDS and its advantages compared to conventional nano drug delivery systems (NDDS). First, we compare liposomes and exosomes in terms of their distinct benefits in DDS. Second, we analyze what to consider for achieving better isolation, yield, and characterization of EVs for DDS. Third, we summarize different methods for the modification of surface of EVs, followed by discussion about different origins of EVs and their role in developing DDS. Next, several major methods for encapsulating therapeutic cargos in EVs have been summarized. Finally, we discuss key challenges and pose important open questions which warrant further investigation to develop more effective EV-based DDS.