A method exists for quantifying lexical diversity in a text by examining the relationship between the number of unique words (types) and the total number of words (tokens). The result of this calculation provides a standardized measure applicable across different text lengths. For example, a text with 100 total words but only 50 unique words would exhibit less diversity than a text of equal length containing 75 unique words.
This measurement offers valuable insights into writing style, language development, and potential cognitive processes. Lower ratios may indicate repetitive language use, limited vocabulary, or potentially, cognitive constraints. Higher ratios typically suggest more varied and complex vocabulary usage. Historically, such metrics have been applied in linguistic research, educational assessments, and clinical analyses of speech and writing.