The construct of illness severity serves many scientific and clinical functions. This study tested the performance as severity scales of three systems for diagnosing drug dependence--DSM-III, DSM-IV and ICD-10--in a multisite regional sample of 370 clinical subjects. Both lifetime and current severity of four drug problems--alcohol, cannabis, cocaine and opiate dependence--was studied in three stages: (a) item difficulty and internal consistency analysis; (b) probabilistic modeling of distribution behavior; and (c) concurrent validation against a set of independent measures. All three systems, for most drugs correlated with most test variables, had good to excellent concurrent validity. Unexpectedly, DSM-III showed in some instances better item behavior, composite score behavior and concurrent validity than the other systems, though DSM-IV and ICD-10 are based on slimmer generic algorithms, and may represent a good balance between simplicity and concurrent validity. Results suggest that the design of future diagnostic algorithms start at the item level and strive for moderate levels of both internal consistency and difficulty. Composite score distributions can then be modeled in field research, and necessary item corrections can be made before the algorithm is widely promulgated.