by Michael Korcok
copyright 1997, Michael Korcok, all rights reserved
Chapter One: Introduction
· Problem Statement
· Purpose Statement
· Background
· Proposed Methods
Chapter Two: Literature Review
· Overview of the Debate-Critical Thinking Studies
· Howell (1943)
· Brembeck (1947)
· Williams (1951)
· Beckman (1955)
· Jackson (1961)
· Cross (1971)
· Colbert (1986)
· Debate
· Critical Thinking
· Hypotheses and Rationale
Chapter Three: Proposed Methods
References
"Developing critical thinking ability has long been assumed to be one of the primary educational outcomes a student might receive by participating in competitive debate. While that outcome is presumptively important, the debate community has not generated sufficient research to demonstrate that participating in competitive debate promotes development of critical thinking ability to any significant degree…. We can no longer assume that our educational mission is apparent to those outside our community, or that others value its importance. Nor can we assume that our history and traditions are sufficient to ensure a prosperous future. Educational accountability is a very real demand the debate community will face. As we prepare to meet educational accountability demands, we should keep in mind one fundamental principle we try to teach our students: 'They who assert must prove.' We have the burden of proof to demonstrate the educational value of our activity." (Hill, 1993, p.22).
Because of the importance of critical thinking abilities to our common future, debate educators have looked to justify debate programs via the salutary effect of participation in competitive intercollegiate debate upon critical thinking skills. The debate community nearly uniformly believes that participation in competitive intercollegiate debate increases critical thinking abilities, and there are many prima facia reasons to believe that debate participation does increase critical thinking ability: students are expected to construct understandings of multiple subject domains by researching complex issues, to synthesize diverse information into coherent positions, to formulate arguments from conflicting viewpoints, and finally to advocate their arguments in a competitive atmosphere. This "debate teaches critical thinking" belief is expressed in a widely-used high school debate textbook, Lynn Goodnight's (1993, p.3) Getting Started in Debate:
"Debate is a great way to develop your critical thinking skills. Debaters need to know how to choose the best type of approach, method of organization, or presentation style for a debate. They also must know how to find the best evidence. They must evaluate the evidence being used by the other debate team and understand exactly what it does for their arguments. They must think about the consequences of each argument and evaluate its worth.… As a debater, you will find that the critical thinking skills you develop as you answer such questions prove invaluable."
The view that debating increases critical thinking skills is sometimes buttressed with reference to the empirical research in this area. The most widely-used undergraduate debate textbook, Austin Freeley's (1990, p.21) Argumentation and Debate refers to studies conducted in 1949 and 1961 respectively:
"Through study of argumentation and practice in debate, students participate in an educational process specifically designed to develop their proficiency in critical thinking. Winston Brembeck investigated whether a college course in argumentation and debate, as taught at eleven different colleges, improved critical-thinking scores on a standardized test and found that the argumentation students outgained the control students by a statistically significant amount. Ted R. Jackson found that the critical-thinking ability of 100 college debaters (in comparison with 147 nondebaters) improved by a statistically significant amount after one season of intercollegiate debating. The debater learns to apply the principles of critical thinking not only to problems that emerge in the relative leisure and comfort of research or the briefing session but also to problems that arise in the stress of debate."
The research, however, has not uniformly found that debating increases critical thinking ability. Several studies have found no statistically significant differences between the control group and the debaters and in one recent study, students scored lower on the critical thinking test after participating in debate. Despite the debate community's nearly unanimous belief that debating increases critical thinking ability, there is, as Hill argues, an disturbing lack of empirical support for this claim.
The research question is deceptively simple: does participation in competitive intercollegiate debate increase critical thinking ability? Furthermore, the subsidiary question of whether higher participation in intercollegiate debate results in greater critical thinking gains is also important. Participation in intercollegiate debate is a fairly straightforward concept: does a college student compete at the hundreds of intercollegiate debate tournaments held throughout the United States over the course of the academic school year. The question of what is meant by higher participation in intercollegiate debate is somewhat more complex, but it is roughly the notion of how much time is spent researching, attending tournaments, and participating in practice rounds throughout the academic school year. Critical thinking is a much more complex concept. Initially, little more is meant by this concept than the definition advanced by Robert Ennis (1987, p.10): "reasonable reflective thinking that is focused on deciding what to believe or do."
As many as 500 American colleges and universities have competitive intercollegiate debate programs with at least one faculty member serving as a director of debate: these programs have anywhere from two to over a hundred students actively traveling to compete at debate tournaments across the nation. It is a conservative estimate that millions of dollars and hundreds of thousands of hours annually are given to competitive intercollegiate debate in the United States. Numerous justifications are offered regarding the educational value of such expenditures and "debate teaches critical thinking skills" is the classical rationale for intercollegiate debate. In an era of educational accountability, empirical support for this rationale could be of especial importance.
The "debate teaches critical thinking" rationale is potentially very important in part because the importance of critical thinking skills has been repeatedly recognized by educators and policymakers. Forty-five years ago, the American Council on Education concluded that critical thinking ability was of fundamental importance to individuals: "The good life in a democratic society … seems to rest fundamentally on one's ability to think critically about the problems with which he (or she) is confronted." (Dressel and Mayhew, 1954, p. 35) Those concerned with the evaluation and reform of education have consistently argued that improvement of students' critical thinking abilities is a national priority. The Department of Education's National Assessment of College Student Learning identified critical thinking as an essential skills area with wide-ranging implications. They concluded that:
"Ideally an effective undergraduate education helps students become better thinkers, communicators, and decision makers in the real world contexts of work and society that extend beyond the traditional classroom. Halpern (1993, p.242) asserts that "ideally, the students who have become better thinkers will demonstrate critical thinking skills that range from more reasoned consumerism to improved problem solving." Students need to acquire basic communication skills and develop cognitive abilities to understand principles, concepts or ideas. However, students must move beyond being simply receivers or transmitters of information. In order to reach advanced skills in writing and speech as well as listening, college students need to develop their critical thinking skills in order to evaluate, analyze, and make judgments about the multitudes of messages or interactions they encounter in their daily lives. Ideally, college graduates will learn to assume responsibility for their own intellectual development that will continue beyond the formal education they receive in college. The improvement of these skills should help students to become better citizens and employees in real world contexts." (Jones et al., 1995, p.168)
At the state level, individual governors have gone so far as to issue executive orders regarding the importance of teaching critical thinking skills. California is an instructive example:
"The executive order establishing California's requirement states: 'Instruction in critical thinking is designed to achieve an understanding of the relationship of language to logic, which would lead to the ability to analyze, criticize, and advocate ideas, to reason inductively and deductively, and to reach factual or judgmental conclusions based on sound inferences drawn from unambiguous statements of knowledge or belief. The minimal competence to be expected at the successful conclusion of instruction in critical thinking should be the ability to distinguish fact from judgment, belief from knowledge, and skills in elementary inductive and deductive processes, including an understanding of the formal and informal fallacies of language and thought." (Freeley, 1990, p.1)
The classical rationale for competitive intercollegiate debate programs is grounded in important and recognized values. Critical thinking ability is understood as of fundamental importance to both individuals and to democratic society. The question remains, however: does debate teach critical thinking skills?
I propose to examine whether participation in competitive intercollegiate debate increases critical thinking ability via a type III quasi-experimental design, i.e. a pre-test/post-test design with a control group. The "treatment" will be one year of participation in competitive intercollegiate debate. A simple questionnaire will survey college student participants to determine whether they are currently or have previously competed in intercollegiate debate and thus divide them into an experimental group of debaters and a control group of nondebaters. Furthermore, the extent of participation in intercollegiate debating will be surveyed for the primary experimental group, the debaters to create an ordinal-level measure. This design has been the traditional design of studies addressing these issues for the past 50 years.
The Watson-Glaser Critical Thinking Appraisal will be the instrument used to measure critical thinking ability, the dependent variable. The WGCTA has been the instrument of choice for every substantial study of these issues to date.
The mean difference in critical thinking gains between the pre-test/post-test means of the control group and the treatment group will be subjected to tests of statistical significance. Tests of correlation between the degree of participation and the extent of critical thinking gains for debaters will be conducted.
Participation in debate has been studied a number of times over the past 50 years with regard to the effects on participants' critical thinking abilities. These studies have used similar pre-test/post-test quasi-experimental designs with debate competition as the experimental treatment and have measured the dependent variable with the same instrument, the Watson-Glaser Critical Thinking Appraisal. The studies have examined different populations, have examined different sample sizes, have operationalized "competition" differently, and have used different analyses of the data collected. The results have been mixed.
Evaluations of the corpus of studies in this area have differed widely, with some authors taking the results to indicate an unambiguously favorable relationship between debate participation and critical thinking ability while other authors conclude that no such relationship has been demonstrated. Huseman, Ware, and Gruner (1972, p.262) concluded of the corpus of these studies that:
"In brief, attempts to examine the relationship between critical thinking ability and debate ability have been numerous, and researchers have many times established that debate ability and critical thinking ability are positively correlated."
It was noted in the introduction that debate coaches and textbooks claim that debate participation increases critical thinking ability. Reference to only one or two of the favorable studies is typically made to support that contention, but overall assessments of the research in this area similarly conclude that debate participation increases critical thinking ability. An example of such an assessment is contained in Keefe, Harte, and Norton’s (1982, pp.33-34) introductory debate text:
"Many researchers over the past four decades have come to the same general conclusions. Critical thinking ability is significantly improved by courses in argumentation and debate and by debate experience."
More recent meta-analyses and assessments do not share the view that the body of research supports an unambiguously positive relationship between debate participation and critical thinking ability. The ground-breaking meta-analysis of the corpus of studies examining the relationship between debate participation and critical thinking conducted by Follert and Colbert (1983) cast substantial doubt on the quality of the empirical support available for the connection between debate participation and critical thinking ability. After examining the 47 paired comparisons between debaters and nondebaters with respect to pre-test/post-test critical thinking gains as measured by the Watson-Glaser Critical Thinking Appraisal contained in the Howell (1943), Brembeck (1947), Williams (1951), and Jackson (1961) studies, Follert and Colbert (1983, pp. 10-11) concluded:
"While this research may not prove that there is not a relationship between critical thinking skills and debate training, it shakes the foundation upon which this long-standing assumption has existed. If this relationship is not firmly established, a radical re-evaluation of our purpose is required. This is not to say that the debate activity does not offer many educational benefits and skills to individual participants. However, additional research which statistically demonstrates critical thinking benefits is clearly warranted."
Kent Colbert’s 1986 Ph. D. Dissertation constituted the additional research which Follert and Colbert called for in 1983. Colbert (1986) found a statistically significant relationship between debate participation and critical thinking ability but cautioned (Colbert, 1986, p.136) that:
"Replication of these results with larger samples are needed to firmly establish a case for contemporary debate practices; however the findings that debaters’ critical thinking test scores are significantly higher than those of nondebaters is consistent with all previous research cited in this dissertation."
Furthermore, a smaller study of critical thinking gains comparing students in an argumentation course which had a co-curricular debating requirement with students in a basic speech class and students in an argumentation class without a debating requirement undertaken by Shawn Whalen (1991) concluded that there was no effect of debate participation on critical thinking ability. Taking into consideration the more direct and much larger sample of the Colbert study, Hill (1993, pp. 7-8) nonetheless concluded that the relationship between debate participation and critical thinking remains undemonstrated:
"Considered jointly, the results of these two studies do not provide the evidence Follert and Colbert so clearly demonstrated that we need…. We have not substantially improved our position since Follert and Colbert sounded the first notice that the results of our research are, at best, inconclusive. Taken as a whole, the available evidence neither demonstrates that debate does not affect development of critical thinking ability, nor that it does. Even though some evidence suggests that there is a relationship between debate training and development of critical thinking ability, we are not able to demonstrate convincingly that the relationship is significant. As a result, it is clear that we are ill-prepared to meet educational accountability demands by claiming that our activity promotes development of critical thinking ability."
The seven principal studies to date directly relevant to issues of the relationship between debate participation and critical thinking ability are surveyed below. The studies differ in how they operationalize debate participation, the independent variable, and in their results. All use the Watson-Glaser Critical Thinking Appraisal to measure critical thinking ability, the independent variable. All use a type III quasi-experimental design: a pre-test/post-test control group design with debate participation as the treatment. Table One lists the principal studies and their principal results.
Study |
debate studied |
principal result |
| Howell (1942) | inter-scholastic high school: one season (6 months) |
no significant difference in gains: (debaters outgain at 85% significance) |
| Brembeck (1947) | college argumentation course: one semester |
course students outgained significantly |
| Williams (1951) | inter-collegiate competition: one season (6 months) |
no significant difference in gains: (debaters outgain) |
| Beckman (1955) | college argumentation and discussion courses: one semester | no significant difference |
| Jackson (1961) | inter-collegiate competition: one season (6 months) |
debaters outgained significantly at 5 colleges but nondebaters outgained at 4 colleges |
| Cross (1971) | inter-scholastic high school: one semester |
debaters outgained significantly on only 2 sub-tests of the WGCTA |
| Colbert (1986) | inter-collegiate competition: one season (6 to 7 months) |
all four nulls rejected: significant difference for debater outgains |
Howell (1942)
The first empirical examination of the effect of debate participation on critical thinking ability was William Howell’s Ph.D. dissertation at the University of Wisconsin. The Howell study principally examined whether a season of inter-scholastic high school debate competition in Wisconsin improved students’ critical thinking abilities, but it also examined a host of secondary effects and differences. As Howell (1943, p.96) notes in his introduction, "The present experiment was undertaken to determine whether a typical program of high school debating, which allegedly teaches participating students to ‘think straight,’ was succeeding in this objective." Howell set the pattern for future debate participation/critical thinking studies by utilizing a type III quasi-experimental pre-test/post-test control group design. The mean gains between the pre-test and the post-test for the experimental and control groups were compared. This was also one of the first uses of the newly developed Watson-Glaser Critical Thinking Appraisal which was initially published in 1939 and helped to establish the WGCTA as the instrument of choice when measuring critical thinking abilities. The study obtained preliminary data from 218 high school debaters from 25 Wisconsin high schools and examined more than 200 students from 24 Wisconsin high schools in the main experiment over a 6 month competition season.
Howell could not conclude that there was a significant difference between the critical thinking gains made by the debaters and the nondebaters over the course of his study. Nonetheless, the Howell results were suggestive: at the 85% level of confidence, the mean difference between debaters and nondebaters became significant. Some of the more important of the host of findings reported by Howell (1943, p.100-102) are:
"1. Considering all experimental and control groups, debaters outgained non-debaters in critical thinking scores over the experimental period of six months. The critical ratio of the difference in mean gains is 1.04. Since a minimum critical ratio of 2 is required for significance we cannot conclude that high school debaters are certain to outgain non-debaters.
2. When the experimental and control groups were equated on I.Q. scores the debaters again outgained the non-debaters, but not significantly. The critical ratio of this difference in mean gains was .14. When the experimental and control groups were matched on both I.Q. scores and pretest scores on the Watson-Glaser tests the critical ratio of the difference in mean gains was 1.26, again in favor of the debaters. There are 89 chances in 100 that this difference exists.
3. Both debaters and non-debaters showed significant gains in critical thinking scores over one debate season of approximately six months.
4. Even though the debaters and non-debaters were closely matched on I.Q. scores the debaters showed a significant superiority on both pretests and posttests of critical thinking. This difference is affected but little by amount of debating experience.
5. College students with high school debating experience scored consistently higher on these critical thinking tests than did those who had not debated….
1`. The coefficient of correlation between I.Q. scores and scores on the Watson-Glaser tests was found to be .63. This shows a substantial over-lapping of the abilities measured, yet proves that they are not identical. Critical thinking scores cannot be predicted accurately from I.Q. scores if this coefficient is correct.
2`. The mean I.Q. of the debaters tested in both the preliminary investigation and in the controlled experiment was 119. Evidently the high school debater in Wisconsin is considerably above the average in intelligence.
3`. Students with moderately high I.Q. scores (115-125) improved slightly more in terms of net gain in critical thinking scores than did the superior IQ group.
4`. Scholarship accomplishment is more directly related to critical thinking scores than is I.Q., or debating skill, or amount of debate experience. The evidence suggests that the abilities measured by the Watson-Glaser tests are essential to the getting of good grades in high school.
5`. High school debaters are superior in scholarship. 50% of all the debaters in the preliminary investigation were "A" students, and 35.6% were "B" students."
Although the main finding of the Howell study failed to achieve significance, this first empirical examination of the relationship between debate participation and critical thinking ability was sufficiently suggestive of a relationship to motivate further research.
Brembeck (1947)
The second empirical examination of the effect of debate participation on critical thinking ability was Winston Brembeck’s Ph.D. dissertation, also at the University of Wisconsin. The Brembeck study principally examined whether a one semester university-level argumentation course improved students’ critical thinking skills. Brembeck (1949, p.177) points out that he considers his work an extension of the research begun by Howell: "This is the second of a series of studies designed to investigate experimentally the effects of training in argumentation and debating on critical thinking ability." Brembeck follows Howell’s by using an identical type III quasi-experimental pre-test/post-test control group design. He too compares the mean gains between the pre-test and the post-test for the experimental and control groups. The Watson-Glaser Critical Thinking Appraisal is also the instrument used to measure the dependent variable. The experimental group in this study was comprised of 202 students attending Argumentation classes at 11 institutions: control groups were set up at each institution.
Brembeck concluded that there was a significant difference between the critical thinking gains made by the argumentation students and the control group over the course of the semester. Several of the more important of the dozen major conclusions reported by Brembeck (1949, p.187-188) are:
"1. The argumentation students, as a whole, significantly outgained the control students in critical thinking scores. The critical ratio of the difference in battery mean gains was 2.56. There is approximately one time in a hundred that this difference could occur by chance.
2. In ten of the eleven schools, the students of argumentation had higher average pretest scores than students in control groups. Even though this pretest advantage narrowed their range for improvement, the argumentation students outgained those in control groups….
3. Gains made by the argumentation students in critical thinking ability varied widely among the participating schools, ranging from none at all to highly significant gains. Because of the great variation in the size of the individual argumentation classes, no valid statistical comparisons can be made.
8. Argumentation students with high school and/or college debate training made significantly higher pre-test scores than those without debating experience. Whether this is due to the debating, or whether these students would have made higher scores anyway, cannot be determined by this experiment."
The findings of Brembeck whetted the appetites of researchers looking into the relationship between training in argumentation and critical thinking, but they were hardly dispositive of the issue with respect to either the value of Argumentation classes or competitive debating.
Williams (1951)
The third empirical examination of the effect of debate participation on critical thinking ability was Donald William’s 1951 Master’s Thesis at Purdue University. The Williams study principally examined whether a season of inter-collegiate debate improved students’ critical thinking skills. This was the first study to examine the effects of intercollegiate debating on critical thinking ability. Williams followed the now-established form for experimental design by using a quasi-experimental pretest/posttest control group design and utilized the WGCTA as the measuring instrument of the dependent variable.
Williams could not conclude that there was a significant difference between the critical thinking gains made by the debaters and the nondebaters over the course of the competition season. Several of the major conclusions reported by Williams (1951) are:
"1. Debaters did make significant gains in critical thinking, as measured by the Watson-Glaser Critical Thinking Test, but their gains were not significantly greater than the gains of a similar group of non-debaters.
2. There was a slight suggestion that those college students who have previous experience in debating in high school and in college may have greater critical thinking ability as measured by this test than those who had one year or less of such experience.
3. Those students who have more than one year of experience in debating did not have greater critical thinking ability as measured by this test than those students who had one year or less of such experience.
4. There was some indication that those students who are rated as better debaters by their coaches will have greater critical thinking ability than those who are rated as having less skill in debating."
Williams’ findings continued the emerging pattern of results which hinted at the ability of argumentation and debate to improve critical thinking skills but which stopped short of achieving statistical significance. The first test of intercollegiate debatings effect upon critical thinking ability was merely suggestive for future research.
Beckman (1955)
Vernon Beckman’s Ph.D. dissertation re-examined the issues raised by Brembeck’s research into the effects of an argumentation course on critical thinking. The Beckman study principally examined whether either a one semester university-level argumentation course or a one semester university-level discussion course improved students’ critical thinking skills. A type III quasi-experimental pre-test/post-test control group design using the WGCTA as a measure of critical thinking ability was utilized by Beckman. He too compared the mean gains between the pre-test and the post-test for the experimental and control groups.
Beckman could not conclude that there was a significant difference between the critical thinking gains made by either the argumentation students or the discussion students and the control group over the course of the semester, but he did find significant differences between colleges and universities. Several of the major conclusions reported by Beckman (1955) are:
"1. It cannot be concluded from the present study that there is a significant difference in critical thinking ability, as measured by the Watson-Glaser Critical Thinking Appraisal, between students in college argumentation courses, discussion courses, and other courses of the type used as controls in this investigation….
2. The conclusion can be drawn that there are statistically significant differences in gains between schools. There is less than one chance in 200 that these observed differences are due to chance….
6. Students who score high on the pretest make relatively smaller gains than those with lower pretest scores."
Beckman’s findings continued the pattern of suggestive results which fail to achieve statistical significance. The important theoretical finding from this study with respect to the effect of intercollegiate debating on critical thinking was the observation that those who score higher on the WGCTA have more difficulty making gains than those who score lower: given that previous studies had unambiguously indicated that debaters score much higher on the WGCTA pretest, one might now expect critical thinking gains to be more difficult to achieve by whatever means. Further research was clearly in order.
Jackson (1961)
Teddy Jackson’s Ph.D. dissertation at the University of Wisconsin was the second of the three major studies to examine directly the effect of intercollegiate debate competition upon critical thinking ability and completed the troika of studies conducted at the University of Wisconsin examining the effects of high school debating, college argumentation courses, and collegiate debate competition upon critical thinking ability. Jackson follows the lead established by Howell and Brembeck and uses a type II quasi-experimental control group design, measures the dependent variable with the Watson-Glaser Critical Thinking Appraisal, and examines the mean difference in gains between the experimental and control groups. Jackson observes 100 debaters in the experimental group and 147 students in the control group distributed over 9 different colleges.
Jackson concluded that the debaters significantly outgained the nondebaters on critical thinking improvements over the course of this study. Some of the more important of the ten major conclusions reported by Jackson (1961) are:
"1. The debaters in this study outgained the non-debaters. The difference between gains was statistically significant between the .05 and .01 level of confidence….
2. There were considerable differences among schools in the sample. Pretest total sample means ranged from 66.82 to 79.29. Debater pretests ranged from 70.64 to 81.50. All of the schools, with one exception, had debater pretest mean advantage over the non-debaters. Post-test means for the debaters at all schools were higher.
3. Previous debating experience gave the students in this study a definite advantage. Those with previous debating experience, either in high school and/or college had a mean pretest advantage of 5.43… Students with high grade point average, made significantly higher critical thinking scores….
4. There was no significant relationship between success at winning debate contests and gain in critical thinking ability. Those who won over 80 per cent of their debates have a slightly higher gain, but it was not statistically significant.
9. Freshmen with high school debating experience had a great advantage on pretest scores over those without such experience. The means were 76.63 and 70.10 respectively.
10. Students who scored low on the pretest significantly outgained those who scored high."
The Jackson findings appeared to be unambiguous with respect to the value of intercollegiate debating for critical thinking improvements. Furthermore, the study replicated the results first noted by Beckman: it is easier to improve initially low WGCTA scores than it is to improve initially higher WGCTA scores. Together with the commonly and consistently replicated finding that debaters score significantly higher on critical thinking pretests, this finding might explain the failure of the Williams study to reach statistical significance. As Hill (1993, p.4) observes, however, the Jackson findings were somewhat less spectacular than they appear to be at first glance:
"Jackson tested debaters at 9 different colleges and found that in 5 colleges, debaters made significantly higher gains in critical thinking than non-debaters, while in 4 colleges the non-debaters out-gained the debaters."
Cross (1971)
Returning to examine the effect of inter-scholastic high school debating on critical thinking, Gary Cross’s 1971 University of Utah Ph. D. dissertation continued the then quarter-century tradition established by William Howell.. The Cross study examines both the effect of debate participation upon critical thinking skills and the effect of differing levels of participation upon critical thinking ability. The methodology of the Cross study is identical to the methods used by the other studies in this tradition. The participants in the Cross study were novice debaters participating in their first semester of debate.
Cross concluded that those who participated highly in debate achieved statistically greater critical thinking gains than those who did not debate, but he also concluded that low participation in debate did not generate statistically significant differences in critical thinking gains in comparison to the nondebating control group. The two immediately relevant conclusions from the Cross study are:
"1) Those who are drawn to competitive debate, low and high participants, and continue for one academic year have greater thinking facilities than those who are not attracted to debate.
2) High participation in competitive debate accelerates debaters’ capacity in critical thinking while low participation may not enhance critical thinking beyond the normal improvement in an academic year."
The Cross findings further refined the developing picture of the effects of debate participation upon critical thinking ability: high participation generated statistically significant improvements in critical thinking ability which were not statistically significant for low participation in debate.
Colbert (1986)
The most extensive and the third study examining directly the effect of intercollegiate debating on critical thinking to date was Kent R. Colbert’s Ph. D. Dissertation at the Florida State University. The Colbert study examined whether collegiate debate participation increases critical thinking ability and whether there was a significant difference in the effect of two distinct forms of debate participation, CEDA and NDT. As Colbert (1987, p. 194) notes, "The purpose of this study was to determine if CEDA debaters, NDT debaters, and nondebaters differ significantly on the Watson-Glaser Critical Thinking Appraisal." Like his predecessors, Colbert used a traditional quasi-experimental pre-test/post-test control group design. The Watson-Glaser Critical Thinking Appraisal was used to measure critical thinking ability, the dependent variable, while a questionnaire and tournament attendance determined the independent variable. The study examined 285 participants with 146 nondebaters in the control group and 138 debaters through an experimental period of 6 to 7 months during the 1984-1985 intercollegiate debate season.
Colbert concluded by rejecting all 4 of his null hypotheses that there was no significant difference in the critical thinking gains made by the combined group of collegiate debaters and the nondebaters, no significant difference in the critical thinking gains between the NDT debaters and the nondebaters, no significant difference between the CEDA debaters and the nondebaters, and no significant difference between the NDT debaters and the CEDA debaters.
Colbert (1987, p.199) interpreted the first and directly relevant of the four conclusions in this manner:
"The hypothesis that debaters and nondebaters do not perform differently to a significant degree on critical thinking tests is rejected by the data in this study. These results constitute preliminary proof that some criticisms of the educational benefits of college debating are unwarranted if critical thinking ability is used as a criterion. (Replication of these results with larger samples are later needed to firmly establish a case for contemporary debate practices, however). The findings that debaters’ critical thinking test scores are significantly higher than those of nondebaters is consistent with the majority of previous research cited in this study."
<the literature relevant to the connection between debate theory and practice and critical thinking>
Critical thinking is notoriously resistant to definition. The notion is intended to capture the idea of reasoned, reflective thought. Robert Ennis (1987, p.10) defines critical thinking as "reasonable reflective thinking that is focused on deciding what to believe or do." Often, the notion of critical thinking is situated as a trait of persons identified as "critical thinkers". Thus, Siegel (1988, p.23) argues that "a critical thinker is one who is appropriately moved by reasons: she has a propensity and disposition to believe and act in accordance with reasons; she has the ability to assess the force of reasons in the many contexts in which reasons play a role." This specification of critical thinking has been operationalized in several measures of critical thinking skill, the most thoroughly studied of which is the Watson-Glaser Critical Thinking Survey. Critical thinking as the "ability to reason well and reflectively" has obvious social and educational importance and provides a natural focus for attempts to justify intercollegiate debate programs.
Although critical thinking, the dependent variable, is controversial and discussion of it has generated a substantial literature, many of these concerns have been incorporated into discussion of the Watson-Glaser Critical Thinking Appraisal (WGCTA) as a measuring instrument. If the measuring instrument is either unreliable or invalid as a measure of critical thinking ability, then both the design and results of the outstanding series of studies are problematic. The internal validity concerns of the research design largely center on questions of self-selection bias of various sorts. This is of primary concern because previous studies of debate participation and critical thinking ability indicate that those who choose to debate begin with significantly greater critical thinking ability than those who do not debate. There are also other observed self-selection differences such as the 20 point IQ difference and significantly higher grades of debaters observed in the original Howell study. Since there is observed self-selection at work, the ability of the research design to control for self-selection bias becomes a critical issue.
The reliability and validity of the WGCTA as a measure of persons’ ability to think well and reflectively has been contested for nearly 50 years: the entire time that this instrument has existed. The first recorded application of this measuring instrument in research was William Howell’s previously mentioned PhD dissertation (Howell, 1943), conducted three years after the Watson-Glaser was first available. Since then, there have been hundreds of published uses of the WGCTA as a measure of critical thinking ability and a number of published articles assessing the reliability and validity of this instrument.
The now half-century tradition of examination into the relationship between debate participation and critical thinking ability suggests that intercollegiate debating increases critical thinking ability. Furthermore, there is reason to believe that higher levels of participation in debate increase critical thinking more than lower levels of debate participation do. This study will thus examine two hypotheses:
H1: participants in intercollegiate debate make greater gains in critical thinking ability than do nonparticipants.
H2: greater participation in intercollegiate debate results in greater improvements in critical thinking ability
The first hypothesis is merely the same hypothesis examined directly by Williams (1951), Jackson (1961), and Colbert (1986) which posits that an activity in large measure designed to teach critical thinking abilities does so successfully. The hypothesis is a simple statement of the still controversial relationship between participation in intercollegiate debate and gains in critical thinking ability with which this proposal opens. The hypothesis is also a specific instance of the general claim that debate participation increases critical thinking ability which is addressed by Howell (1942) and Cross (1971). Following the
The second hypothesis has been tested, and even then only indirectly, by Cross (1971): his finding that high participation in debate by high school students resulted in statistically significant differences in critical thinking ability gains but that low participation in debate did not, suggests that negative findings by Williams (1951) may be the result of a failure to distinguish between high and low participation in intercollegiate debate. Colbert (1986) found a statistically significant difference between the critical thinking gains obtained by CEDA debaters and those obtrained by NDT debaters. Colbert furthermore suggests that these differences might be due to the higher levels of participation found in the NDT during the middle 1980s in comparison to the participation levels present in CEDA during that same time. Finally, with regard to the second hypothesis, it is on-face plausible that greater participation in an activity designed to raise critical thinking abilities would produce greater gains in critical thinking ability.
The questions addressed by this study are not new: they are, in fact, now ancient questions, but they have been addressed in a quantitative and experimental manner only during the past 50 years. These questions have furthermore been directly addressed in only three notable studies. Furthermore, the latest study of this issue was undertaken more than a decade ago and there has been at least one confounding study (Whalen, 1991) since the Colbert (1986) research was accomplished. A fresh look at these issues are in order. Finally, Colbert concludes that at the least an attempt to replicate his results with a higher sample size is in order. In the context of new calls for intercollegiate academic debate to justify the expenditures it makes, it is time to re-examine whether intercollegiate debate participation increases critical thinking ability.
The use of intact groups rather than attempting a randomization of subjects between the experimental and control groups is what marks this design as a quasi-experiment rather than a true experiment. This has been necessitated by the intensity of the time and effort commitment demanded by participation in debate: it is unlikely that there will ever be a substantial study of the relationship between debate participation and critical thinking ability which is a true experiment. Colbert (1986, p.90) explains that:
"It would be difficult, if not, impossible to recruit participants if random assignment was used. The researcher is forced to sample intact groups. While this approach limits the inferential statistics appropriate for analysis, it tests the experimental treatment as it actually occurs."
The pre-test/post-test control group design was chosen because of internal validity concerns. Intact groups are self-selected and this challenge to internal validity cannot be ignored. The pre-test/post-test control group design handles self-selection bias by examining the mean differences between pre-test and post-test scores between the experimental and control group. The logic here is that even if there are self-selected differences between the groups, examining pre-test/post-test differences rather than just differences should neutralize simple self-selection biases. As regarding internal validity concerns generally, Colbert (1987, p.196) observes:
"Pretest posttest control group design has the advantage of helping to control for internal validity factors (such as maturation, history, testing, instrumentation, regression, selection, selection-maturation, and mortality). The reasoning is that when extraneous variables are present it may be logically deduced that the treatment is responsible for the differences in dependent measures between the experimental and control groups."
Beckman, Vernon. (1955). An Investigation of the Contributions to Critical Thinking Made by Courses in Argumentation and Discussion in Selected Colleges. Ph. D. Dissertation, University of Minnesota.
Brembeck, William. (1947). The Effects of a Course in Argumentation on Critical Thinking Ability. Ph.D. Dissertation, University of Wisconsin.
Brembeck, William. (1949). The effects of a course in argumentation on critical thinking ability. Speech Monographs, 16, 172-189.
Colbert, Kent. (1986). The Effects of CEDA and NDT Debate Training on Critical Thinking Ability. Ph. D. Dissertation, Florida State University.
Colbert, Kent. (1987). The effects of CEDA and NDT debate training on critical thinking ability. Journal of the American Forensic Association. 23, Spring, 194-201.
Colbert, Kent & Biggers, Thompson. (1985). Why should we support debate? Journal of the American Forensic Association. 21, Spring, 237-240.
Cross, G. (1971). The Effects of Belief Systems and the Amount of Debate Experience on the Acquisition of Critical Thinking. Ph. D. Dissertation, University of Utah.
Dressel, P., & Mayhew, L. (1954). General education: Exploration in evaluation. Final Report of the Cooperative Study of Evaluation in General Education. Washington, DC: American Council on Education.
Ennis, Robert. (1987). Critical thinking and the curriculum. In Marcia Heiman and Joshua Slomianko (Eds.), Thinking skills instruction: Concepts and techniques. Washington, DC: National Education Association.)
Follert, Vincent, & Colbert, Kent. (1983). An analysis of the research concerning debate training and critical thinking improvements. (ERIC Document Reproduction Service No. ED 238 058)
Freeley, Austin J. (1990). Argumentation and Debate. 7th ed., Belmont, CA: Wadsworth.
Goodnight, Lynn. (1993). Getting Started in Debate. 2nd ed.. Lincolnwood, IL. :National Textbook Co.
Hill, Bill. (1993). The value of competitive debate as a vehicle for promoting development of critical thinking ability. CEDA Yearbook 14. 1-22.
Howell, William. (1942). The Effects of High School Debating on Critical Thinking Ability. Ph.D. Dissertation, University of Wisconsin.
Howell, William. (1943). The effects of high school debating on critical thinking. Speech Monographs, 10, 96-103.
Huseman, Richard, Ware, Glenn, & Gruner, Charles. (1972). Critical thinking, reflective thinking, and the ability to organize ideas: a multi-variate approach. Journal of the American Forensic Association. 9, 261-265
Jackson, Teddy. (1961). The Effects of Intercollegiate Debating on Critical Thinking. Ph. D. Dissertation, University of Wisconsin.
Jones, Elizabeth, Hoffman, Steven, Moore, Lynn, Ratcliff, Gary, Tibbetts, Stacy, & Click, Benjamin. (1995). National Assessment of College Student Learning: Identifying College Graduates' Essential Skills in Writing, Speech and Listening, and Critical Thinking. National Center for Education Statistics, US Department of Education, Office of Educational Research and Improvement, NCES 95-001)
Keefe, C., Harte, T., & Norton, L. (1982). Introduction to Debate. New York: Macmillan.
Siegel, Harvey. (1988). Educating Reasoning: Rationality, critical thinking, and education. New York: Routledge Chapman and Hall, Inc.
Whalen, Shawn. (1991). Intercollegiate debate as a co-curricular activity: effects on critical thinking. Argument in Controversy: Proceedings of the Seventh SCA/AFAa Conference on Argumentation. Ed. Donn W. Parson. Annandale, VA: Speech Communication Association. 391-397.
Williams, Donald. (1951). The Effects of Training in College Debating on Critical Thinking Ability. M.S. Thesis, Purdue University.
Return to Proposals Return to Course Notes Return to Syllabus