A man in his late 70s presented to the emergency department endorsing a week of malaise. He was recently hospitalised for 2 days for new back pain and was discharged with non-opioid pain medications but continued to seek care as he felt unwell. On presentation, he was afebrile with a leukocytosis. Physical examination revealed a painful left knee with no evidence of trauma. Arthrocentesis revealed purulent fluid with elevated white blood cell consistent with septic arthritis. He was started on broad-spectrum antibiotics and underwent irrigation and synovectomy of the left knee. Aspirate and blood cultures grew Streptococcus agalactiae Transthoracic echocardiogram showed no vegetations; however, an MRI of lumbar spine showed L2-L3 and L4-L5 osteomyelitis. He was treated with intravenous ceftriaxone for 3 weeks and then oral levofloxacin for 3 weeks, for a total 6 week course of antibiotics.