Sen Yan

The psychometric approach in IS offers a foundational framework for a broad spectrum of research endeavors, which typically rely on construct validation to confirm that a series of indicators accurately measures the intended construct. However, a longstanding issue with construct validity, unaddressed since its introduction by Cronbach and Meehl in 1955, is that it is evaluated using study-specific response data without comparison to constructs outside the study. This oversight (or, rather, incapability) has significant implications. We introduce a large language model combined with principal components analysis (PCA) and develop the Validity Lodestar application. This approach lays the groundwork for developing more accurate and reliable theoretical models, marking a significant leap forward in the IS discipline’s methodological capabilities, making IS the first psychometric discipline with the capability to properly evaluate construct validity.