Professor Fisher has enumerated three principles of experimental design. 01. The Principle of Replication, 02. The Principle of Randomization, and 03. The Principle of Local Control.
An experiment’s quality is judged by two types of validity. These are known as internal and external validity.
Description of Internal and external validity:
Internal validity extent that an experimental variable is truly responsible for any variance in the dependent variable. In different words, will the experimental manipulation actually cause changes within the specific outcome of interest?
If the observed results were influenced or confounded by extraneous factors, the researcher will have problems making valid conclusion about the relationship between the experimental treatment and the dependent variable.
01. Manipulation Checks:
The validity of manipulations can often be determined with manipulation check. If a drug is administered in different dosages that should affect blood sugar levels, the researcher could actually measure blood sugar level after administering the during to make sure that the dosages were different enough to produce a change in blood sugar.
Extraneous variables can jeopardize internal validity. The six major ones history, maturation, testing, instrumentation, selection, and mortality.
A history effect occurs when some change other than the experimental treatment occurs during the course of an experiment that affects the dependent variable. A common history effect occurs when competitors change their marketing strategies during a test marketing experiment.
Maturation effects are effects that a function of time and the naturally occurring events that coincide with growth and experience. Experiments happening over longer time spans may even see lower internal validity as subjects merely get older or suffered.
Testing effects are also called pretesting effects because of the initial measurement or test alerts or primes subjects in a way that affects their response to the experimental treatments. Testing effects only occur in a before and after study.
A before and after study is one requiring an initial baseline measure be taken before an experimental treatment is administered. So, before and after experiments are a special case of a repeated measures design.
For example, students taking standardized achievement and intelligence tests for the second time usually do better than those taking the tests for the first time.
The effect of testing may increase awareness of socially appropriate answer, increase attention to experimental conditions (that is, the subject may watch more closely,) or make the subject more conscious than usual of the dimensions of a problem.
A change in the wording of questions, a change in interviewers, or a change in other procedures used to measure the dependent variable cause an instrumentation effect, which may jeopardize internal validity.
Sometimes instrumentation effects are difficult to control. For example, if measurement, some problems may arise.
The selection effect is a sample bias that results from differential selection of respondents for the comparison groups, or sample selection error, discussed earlier. Researchers must make sure the characteristics of the research subjects accurately reflect the population of relevance.
If an experiment is conducted over a period of few weeks or more, some sample bias may occur due to the mortality effect experiment before it is completed.
Mortality effects may occur if subjects drop from one experimental treatment group disproportionately that from other groups. Consider a sales training experiment investigation the effects of close supervision of sales people (high pressure) versus low supervision (low pressure).
External validity is the accuracy with experimental results can be generalized beyond the experimental subjects. External validity is increased when the subjects comprising the sample truly represent the population of interest and when the results extend to other market segments or groups of people.
The higher the external validity, the more researchers and managers can count on the fact that any results observed in an experiment will also be seen in the “real world”