Re-visiting the Quasi-Experimental Design
In this lecture we will consider a type of evaluation design, like a normal experiment, includes independent and dependent variables, but involves a situation in which program participants cannot be randomly assigned to groups. Because of the absence of random assignment means that causal conclusions cannot be made, whereas they can be made with some degree of confidence in a purely experimental study, this design is called quasi-experimental (“almost” experimental). When you finish this week in the course, you should be able to:
Identify the defining features of a quasi-experimental design.
Describe the features of a nonequivalent control group design and understand why this design is necessarily confounded.
Understand why matching nonequivalent groups on pretest scores can introduce a regression effect.
Describe the features of interrupted time series designs and understand how they can be used to evaluate trends.
Describe several variations on the basic time series design.
Explain why most archival research is quasi-experimental.
Describe the advantages and limitation of archival research.
As a point of reference for a discussion about quasi-experimental evaluation designs, let’s go back to the “true” experimental design. The true experimental design involves the manipulation of independent variables and either equivalent groups for between-subjects design or appropriate counterbalancing for within-subjects design. Anything less than this true experimental design is quasi-experimental—meaning “almost” experimental.
Generally speaking, a quasi-experiment exists whenever causal conclusions cannot be drawn because there is less than complete control over the variables in the evaluation. These designs are actually quite important in the evaluation of public programs because they allow for a degree of control. They serve evaluators quite well when ethical or practical problems make random assignment impossible, and they often produce results that can have clear benefits for people’s lives. To date, we have encountered a number of designs examples that could be considered quasi-experimental.
Single-factor nonequivalent group designs, with two or more levels
Nonequivalent group factorial designs
P x E Factorial designs
Cook & Campbell (1979) provide examples of the two designs typically found in most textbooks: nonequivalent control group designs and interrupted times series designs. We discussed those examples last week. This week will discuss the measurement of impacts in Quasi-experimental evaluations along with a type of evaluation called archival. Archival evaluations involves answering empirical questions by using information that has already been collected for another purpose, rather than collecting new data, and it often includes non-manipulated independent variables.
In the most common quasi-experimental design, control or comparison groups are constructed in an attempt to approximate a randomized design. This is typically done by either matching participating and nonparticipating targets or by statistical adjustment of participants and non-participants in an attempt to make the equivalent on relevant variables. In both cases, the goal is to be able to compare the program participants with non-participants that resemble them on those characteristics and experiences related to the evaluation’s outcome measures.
The basic formula for impact assessment in quasi-experiments is similar to the one for experimental designs. The key difference is that it contains an additional term representing uncontrolled pre-intervention differences between the intervention and control groups. In this way, the central problem of how to handle selection bias is highlighted.
Net effect = [Gross outcome for an intervention group] – [Gross outcome for a constructed control group] + or – [Uncontrolled difference between intervention and control groups] + or – [Design effects and stochastic error].
Whether a specific quasi-experiment will yield unbiased estimates of net effects, therefore, depends to a great degree on the extent to which the design minimizes critical differences between the intervention and control groups. When there is a possibility that one or more relevant differences exists between the members of the intervention and comparison groups, as there typically is in quasi-experiments, then it is also a possibility that these differences – not the intervention- cause all or part of the observed effects.
Research Using Archival Data
Archival data refers to evaluations that use secondary data analysis. This means that the data was collected for a purpose other than the evaluation at hand. There is as much variation in the sources of such data as one can imagine. Public sources of such data include census data, court records, genealogical data, corporate annual reports and patient office records. Private sources of data include credit histories, health history data, educational records, personal correspondence and diaries. The records themselves and the places where the records are stored is the source of the term “archives.” Archives are located in university libraries, government offices, and computerized databases.
The greatest strength of archival data is the fact that the amount of information is available is unlimited. This is also the source of a significant weakness since not all data is amenable to study in it current form.
It is clear that by it very nature, archival research does not allow for random assignment in between-subjects designs, and this is what makes it quasi-experimental. Like other quasi-experimental evaluations, this type of design often involves sophisticated attempts to control for potential threats to internal validity.
Chapter 9- Peter Rossi, Mark Lipsey & Howard Freeman, Evaluation: A Systematic Approach (7th edition), 2004.
Jason et al., Effects of Enforcement of Youth Access Laws on Smoking Practices
Ballart & Riba: Impact of Legislation Requiring Moped and Motorbike Riders to Wear Helmets (Time Series)
Avery-Leaf, et al., Efficacy of a Dating Violence Program on Attitudes Justifying Aggression
Rotheram-Borus, at al., Efficacy of a Preventative Intervention for Yourths Living with AIDS
Bobcock & Steiner: The Relationship between Treatment, Incarceration
Our Service Charter
Excellent Quality / 100% Plagiarism-FreeWe employ a number of measures to ensure top quality essays. The papers go through a system of quality control prior to delivery. We run plagiarism checks on each paper to ensure that they will be 100% plagiarism-free. So, only clean copies hit customers’ emails. We also never resell the papers completed by our writers. So, once it is checked using a plagiarism checker, the paper will be unique. Speaking of the academic writing standards, we will stick to the assignment brief given by the customer and assign the perfect writer. By saying “the perfect writer” we mean the one having an academic degree in the customer’s study field and positive feedback from other customers.
Free RevisionsWe keep the quality bar of all papers high. But in case you need some extra brilliance to the paper, here’s what to do. First of all, you can choose a top writer. It means that we will assign an expert with a degree in your subject. And secondly, you can rely on our editing services. Our editors will revise your papers, checking whether or not they comply with high standards of academic writing. In addition, editing entails adjusting content if it’s off the topic, adding more sources, refining the language style, and making sure the referencing style is followed.
Confidentiality / 100% No DisclosureWe make sure that clients’ personal data remains confidential and is not exploited for any purposes beyond those related to our services. We only ask you to provide us with the information that is required to produce the paper according to your writing needs. Please note that the payment info is protected as well. Feel free to refer to the support team for more information about our payment methods. The fact that you used our service is kept secret due to the advanced security standards. So, you can be sure that no one will find out that you got a paper from our writing service.
Money Back GuaranteeIf the writer doesn’t address all the questions on your assignment brief or the delivered paper appears to be off the topic, you can ask for a refund. Or, if it is applicable, you can opt in for free revision within 14-30 days, depending on your paper’s length. The revision or refund request should be sent within 14 days after delivery. The customer gets 100% money-back in case they haven't downloaded the paper. All approved refunds will be returned to the customer’s credit card or Bonus Balance in a form of store credit. Take a note that we will send an extra compensation if the customers goes with a store credit.
24/7 Customer SupportWe have a support team working 24/7 ready to give your issue concerning the order their immediate attention. If you have any questions about the ordering process, communication with the writer, payment options, feel free to join live chat. Be sure to get a fast response. They can also give you the exact price quote, taking into account the timing, desired academic level of the paper, and the number of pages.