The aim of this study is to develop a non-targeted approach for the authentication of extra virgin olive oil (EVOO) using vibrational spectroscopy signatures combined with pattern recognition analysis. Olive oil samples (n = 151) were grouped as EVOO, virgin olive oil (VOO)/olive oil (OO), and EVOO adulterated with vegetable oils. Spectral data was collected using a compact benchtop Raman (1064 nm) and a portable ATR-IR (5-reflections) units. Oils were characterized by their fatty acid profile, free fatty acids (FFA), peroxide value (PV), pyropheophytins (PPP), and total polar compounds (TPC) through the official methods. The soft independent model of class analogy analysis using ATR-IR spectra showed excellent sensitivity (100%) and specificity (89%) for detection of EVOO. Both techniques identified EVOO adulteration with vegetable oils, but Raman showed limited resolution detecting VOO/OO tampering. Partial least squares regression models showed excellent correlation (Rval ≥ 0.92) with reference tests and standard errors of prediction that would allow for quality control applications.