Assessment Threats to External Validity
Reactivity of assessment
For most measures used in psychological experiments, subjects or clients are aware that some facet of their functioning is assessed. The measures may include a variety of questionnaires or tests that subjects complete. If subjects are aware that their performance is being assessed, the measures are said to be obtrusive. Obtrusive measures are of concern in relation to external validity because awareness that performance is being assessed can alter performance from what it would otherwise be. If awareness of assessment leads people to respond differently from how they would usually respond, the measures are said to be reactive.
Test sensitization
In many experiments, particularly in therapy research, pretests are administered routinely. The purpose is to measure the client's standing on a particular variable before receiving the experimental manipulation or treatment. Administration of the pretest may in some way sensitize the subjects so that they are affected differently by the intervention, a phenomenon referred to as pretest sensitization. Individuals who are pretested might be more or less amenable or responsive to an intervention than are individuals who are not exposed to a pretest merely because of the initial assessment.
Even when a pretest is not used, it is possible that assessment may influence the results. The posttest might sensitize subjects to the previous intervention that they have received and yield results that would not have been evident without the assessment. This effect, referred to as posttest sensitization, is very similar to pretest sensitization where test administration may crystallize a particular reaction on the part of the subject. With posttest sensitization, assessment constitutes a necessary condition for treatment to show its effect. Essentially, treatment effects may be latent of incomplete and appear only when a reactive assessment device is administered. As a threat to external validity, posttest sensitization raises the question of whether the results would extend to measures that subjects could not associate with intervention or measures that were completely out of their awareness. The effect of posttest sensitization is slightly more difficult to assess than is pretest sensitization because it requires nonreactive assessment of treatment effects and a comparison of the results across measures varying in reactivity.
Time of measurement and treatment effects
The results of an experiment may depend on the point in time that assessment devices are administered. For example, an investigation may reveal that a particular type of psychotherapy surpassed no treatment or that one therapy was superior to another immediately after completion of treatment. An external validity question that can be raised is whether the same result would have been obtained had measurement been taken at another time, say, several months after treatment.
