Drawing Valid Inferences

The purpose of research is to investigate specific relations between independent and dependent variables. The value of experiments derives from their capacity to simplify the situation in which variables may operate so that the influence of many variables can be separated from the variable of interest. Stated another way, an experiment helps rule out the influence of many alternative variables that might explain changes on the dependent measures.

The extent to which an experiment rules out as explanations those factors that otherwise might account for the results is referred to as internal validity. Factors or sources of influence other than the independent variables are referred to as threats to internal validity and include history, maturation, testing, instrumentation, statistical regression, selection biases, attrition, diffusion of treatment, and special treatment or reactions of controls.

Aside from evaluating the internal validity of an experiment, it is important to understand the extent to which the findings can be generalized to populations, settings, measurement devices, and experimenters other than those used in the original experiment. The generality of the results is referred to as the external validity of the experiment. Although the findings of an experiment could be limited to any particular condition or arrangement unique to the demonstration, a number of potential limitations on the generality of the results can be identified. These potential limitations are referred to as threats to external validity and include characteristics of the sample, the stimulus conditions of the experiment, the contextual conditions such as reactivity of the experimental arrangement, multiple-treatment interference, and novelty effects, and the conditions of assessment such as reactivity, test sensitization, and timing.

As a priority, the internal validity of an experiment is to be regarded as more important, or at least logically prior in importance, than external validity. One must first have an unambiguous finding before one can ask about its generality. Given the priority of internal validity, initial considerations of an experiment pertain to devising those conditions that will facilitate demonstrating the relation between the independent and dependent variables. A well-designed experiment maximizes the opportunity to draw inferences about the intervention.

Copyright © 2003 PCG Evaluations, All rights reserved.