Research Article
The Impact of Multimedia on Critical Thinking and Writing of Saudi Secondary School Students
Department of Foreign Languages, College of Education, King Faisal University, Al-Ahsa, Kingdom of Saudi Arabia
Technology oriented students carry their laptops to classes to facilitate note taking and to add reflections during classes in the most efficient manner. Presently due to technology assisted learning, students are not only familiar with the use of computer but are also quite knowledgeable about the variety of tasks that can be accomplished through computer applications. Additionally, the students use internet to solve problems and to find necessary research information to support their papers and assignments. Schools, which provide access to computers, often conduct trips to computer laboratories or provide computer facilities in the classroom as a part of the writing process so that students can write their essays in school as regularly scheduled parts of classes. Those, who promote computer use, assume that the computer use is natural and does not impede the thinking process. Rather, it is seen as a fluid way to express thinking in a format that is easy to read and edit and gives more time for critical thinking (Hartley, 1993). It is also important to consider whether ready access to computers actually enhances critical thinking or it merely provides students with a tool that helps them finish tasks quickly in a more acceptable, finished form without additional editing and revision. Whether this efficiency attenuates students critical thought processes that lead to revisions that ultimately promote quality writing is a major consideration of this research.
Research on writing with computers has been conducted on special needs populations, demonstrating a positive impact on the writing development of children with learning disabilities (MacArthur, 1996). However, no clear body of research has explored the effects of using computers in the writing process for high-ability learners. In addition, the literature on the use of computers to assist high-ability students in composing essays features that their critical thinking is not plentiful. An important consideration is whether the gender has any impact on high-ability performance. Considering the interests of high-ability boys, Kerr and Cohn (2001) cited the classic studies with regard to play interests and career goals. They found that high-ability boys were more like an average boys than like average girls. Their interests in both play and careers scored high on his masculinity indices.
Similarly, Coleman and Cross (2001) wrote that the interests of high-ability boys are more similar to those of non-gifted boys than the interests of high-ability girls or to those of normal girls. In terms of ability, they stated that girls tend to do better in English while boys achieve significantly better in science and math.
Teachers advocate critical thinking and writing as a fundamental goal of education, but limited evidence demonstrates the success of implementing critical thinking in the classroom (Keeley et al., 1995; Seshechari, 1994). Keeley et al. (1995) and Perkins (1985) indicated that most of the classrooms greatly lack in critical thinking activities, Seshachari (1994) argued that while many teachers pride themselves on their critical thinking and writing assignments, but the results indicate that writing by students did not necessarily guarantee better grades.
Many scholars agree that critical thinking involves skill (ability) and disposition (attitude) (Beyer, 1997; Garside, 1996; Kennedy et al., 1991; Wilen and Phillips, 1995). In general, critical thinking means to make effective use of our thinking skills.
Students need to realize that the goal of learning is thinking (Costa, 1983). French and Rhoder (1992) mentioned that the learner must construct learning and meaning and not be the recipient of learning. The students, who are given instructions on thinking generally, have higher scores in outcome measures than their counterparts (Idol and Jones, 1991; Nickerson et al., 1985; Perkins, 1985). Many educators observed that useful thinking skills include those associated with acquiring, interpreting, organizing and communicating information; processing data to investigate questions, solving problems and making decisions; and interacting with others (Bloom et al., 1956; Wilen and Phillips, 1995; Garside, 1996).
Earlierly, some leading scholars have paved the way for critical thinking pedagogy. In the 1930's, Dewey (1933) created the term reflective thinking to refer to the thinking in which a person turns a subject over in the mind giving it serious consideration. Watson and Glaser (1939) were also pioneers in critical thinking and argued that critical thinking was a persistent effort in looking at knowledge, weighing evidence, interpreting data, creating logical relationships between propositions and drawing justified conclusions about the material at hand. In other words, critical thinking is demonstrating clear argumentation structure.
Piaget (1967) stage theory served as the explanation for an individual's development throughout childhood. He argued that by formal-stage learning (age from 11-12 to adulthood), an individual has the ability to coordinate and apply abstract reasoning and solve problems through systematic hypothesis testing.
Research indicated that not all the students reach the formal stage of learning by the time they reach college as may be typically assumed. In fact, many younger college-aged students have not attained all of the formal operations of thinking (Hester, 1994). Lehmann and Dressel (1962) studied the changes in students' critical thinking, attitudes and values. He discovered that there was a significant change from freshmen to seniors and that most of the change occurred in the freshmen and sophomore years.
Several scholars agreed that writing is fundamental to learning knowledge and communicating that knowledge (Brent and Felder, 1992; Kloss, 1996; Sublett, 1993). Others further indicated that effective writing includes clarity, consistency, variety and logic (Boyd, 1995; Caprariis, 1996; Hester, 1994; Leahy, 1995; O'Flahavan and Trierney, 1991; White, 1993). O'Flahavan and Tierney (1991) further proposed three essential writing abilities to learners to demonstrate critical thinking i.e., planning, translating and reviewing. These skills should also be promoted in the basic course when students are creating presentations. The process of writing is closely related to the teaching of critical thinking and problem solving. Medhurst (1989) mentioned that students need to learn how to share their findings in clearly structures and argumentative prose writing. By giving these arguments, it is evident that critical thinking encompasses the reading and writing of effective arguments.
Critical thinking, as opposed to rote memorization, involves active and skillful demonstration of higher-order thinking skills (analysis, synthesis and evaluation) among learners (Brown, 1998). Sumner (1940) posits that critical faculty, being a product of education and training that guarantees mental habit and power, is the only defense against delusion, deception, superstition and misapprehension of our earthly circumstances and ourselves. Elder and Paul (1998) believed that if students can take charge of their own minds, they can take charge of their own lives; they can improve them, bring them under their command and direction. Teaching critical thinking or higher-order thinking skills improves the quality of students 'mode of thinking about any subject, content, or problem by skillfully analyzing, assessing and reconstructing it. Thus, the demand to teach critical thinking skills or higher-order thinking skills reaches an insurmountable height (Black, 2005; Brown, 1998; Elder and Paul, 1998; Gonzalez, 1999; Van Gelder, 2005).
Scholars cited Socrates as the initiator of the art of critical thinking because of the importance he attributed to ideas and their role in directing the conduct of everyday life. (Capossela, 1996). Continuing with the idea of reflection as a dimension of critical thinking, Ennis (1989) defined critical thinking as reasonable and reflective thinking that is focused on deciding what to believe and do. Later, Paul (1996) argued that, to become a critical thinker, is to practice skills that enable one to start to take charge of the ideas that run ones life. It is to think consciously and deliberately and skillfully in ways that transform oneself.
Both Ennis (1989) and Paul and Elder (2001) emphasized that a major facet of critical thinking involves examining assumptions that underlie thought and action. In considering the process of critical thinking, Yanchar and Slife (2003) suggested that it has two parts: the first requires knowledge of the assumptions and underlying worldviews of a particular discipline or field of inquiry and the second involves developing ideas and assumptions that are alternatives to present views. Halpern (1984) defined critical thinking as directed thinking; that is, critical thinking has a purpose or a goal toward which it is directed.
McKeachie et al. (1994) argued that learning to think critically requires contemplation and communicating the thinking through talking, writing, or doing so that others can react to it. Dixon (1996) found that evidence of critical thinking in writing increased after students were trained to use a strategy based on the hegelian dialectic. The hegelian dialectic is a strategy that focuses on a thesis, a statement of a major idea that directs examination. Similarly, Dlugos (2003) has suggested modifying course content to explore how conventional student assignments can be expanded to include critical thinking and writing about ones experiences, attitudes and values relative to the main concepts of the course.
In one recent analysis, this disjointed pattern connected to computer use in writing led researchers to conclude that the effects were essentially random, indicating that the use of computers had no meaningful influence on performance (Dybdahl et al., 1997).
Writing quality is one aspect of the writing process that has been studied in relation to the use of computers. In a counterbalanced repeated measures design examining within-subject differences in writing process and product, eighth-grade students received higher ratings on the quality of their writing when using computers to compose their essays (Owston et al., 1992; Owston and Wideman, 1997). The research appears to support the proposition that computers do allow for greater writing fluency, provided that the level of computer experience and student motivation are controlled (Reed, 1996).
Petersons (1993) analysis of the fluency provided two notable effects supporting the benefits of computers in writing for high school seniors. Similar patterns of superior writing fluency were reported for students in the fifth (Dybdahl et al., 1997) and sixth grades (Nichols, 1996). In these studies, the number of words and/or sentences were the primary differences noted in the repeated measures designs. The main objective of this study was to explore the impact of word processing technologies on writing in response to guided prompts.
The study was carried out in private Saudi secondary schools in Al-Ahsa during 2008-2009.
Questions of the study: The study involved two questions. First one is the rubric used to evaluate critical thinking scores both valid and useful in measuring the construct and Second one deals with the differences that are noted in student writing samples based on the gender of the writer and the mode of creating the written work (i.e., technology-supported vs. handwritten).
Selection of students: A total of 99 students (39 males and 60 females), who attend two private Saudi secondary schools for high-ability learners were selected for study. The mean age of students was 16 years at the time of entry to school. The criterion used was that the students willing to study at the private school must submit an application including standardized achievements or ability test scores, teacher or counselor recommendations, transcripts and essays to indicate their desire for admittance to the school. Also, the students should agree to an interview with one or more representatives of the school. The admitted students must complete their junior and senior years in the school. The two schools are referred to as Saudi private secondary schools for high-ability learners.
Procedures of the study: The critical thinking was assessed using the Watson-Glaser critical thinking Appraisal (Watson and Glaser, 1980). The Watson-Glaser Critical Thinking Appraisal (CTA) is a multiple-choice test of reasoning skills that is widely used in studies at the high school and college level.
The critical thinking appraisal tests skills of arguments, specifically drawing inferences, recognizing assumptions, evaluating conclusions and assessing the strength of reasons offered in support of a claim (Kurfiss, 1988).
The students took this test two times in a calendar year i.e., Fall of their junior year and at the end of their junior year.
Form A of the Watson-Glaser is composed of 80 test items following 16 scenarios. All of the items are objective questions in which the test taker selects the answer. There are five content areas namely inference, recognition of assumptions, deduction, interpretation and evaluation of arguments. A student received a single score based on the completion of the test.
The internal consistency reliability (coefficient alpha) for the measure was 0.85; test-retest at a 3 month interval was 0.73 and alternate forms reliability was 0.75.
In analysis, the raw scores for each of the five content areas were examined to provide more fine-grained examination of critical thinking skills.
The critical thinking was also measured in essays gathered from the participants at 2 different administrations. The first essay prompt was administered to all juniors as an entry essay at the beginning of their junior year. The second essay prompt was administered during the fall semester of their senior year.
Table 1: | Rubric for scoring essays |
All the students in the study composed their first essay in handwritten form. Whereas, for the second essay, some students were randomly assigned a computer for composing their essays. Both prompts were based on an essay by Katherine Anne Porter. English teachers at the school selected the initial reading.
The students were instructed to read the passage carefully, taking notes if they desired. They were also told that it was acceptable to make changes to their text as they worked and that their writing would be assessed based on the following characteristics:
• | A response that addressed the topic and the assignment |
• | A thesis statement |
• | Specific development of the thesis, using details from the passage as evidence |
• | Analysis (discussion) of the evidence |
• | Logical organization |
• | Coherence of thought |
• | Clarity of expression and |
• | Observance of the rules of grammar and mechanics |
The second prompt included the writing guidelines, referred to and included the initial Porter essay and extended it.
The essays were scored for critical thinking using a rubric adapted from the AP English composition rubric (Dixon, 1996). The major focus of this rubric was on critical thinking (analysis, synthesis and evaluation of ideas) expressed in the essay rather than the writing mechanics (Table 1).
Statistical analysis of data: The main question of the study was the impact of word processing on students writing performance, while maintaining focus on potential gender effects. To analyze the effects of gender and word processing, a 2 (male, female) by 2 (word process, handwrite) repeated measures multivariate analysis of variance was employed, examining four dependent variables at two points in time (WS-1, WS-2): writing sample quality rating, number of words, number of sentences and number of paragraphs. Given the unequal sample sizes, threats to homogeneity of covariance and small sample size in this study, Pillais Trace was used to interpret the MANOVA.
In addition to the rubric ratings of quality in writing, basic features of the writing samples provided by the students were examined. These simple features included number of words, sentences and paragraphs offered in each writing sample.
Raters: Two raters were trained to score both administrations of the essays. Raters were two English instructors at the school who were interested in working on the study and were experts in writing and in assessing writing. Training occurred on four occasions. At the first session, the rubric was explained and examples of each level of the rubric were presented to the raters. They read the essays and asked questions to clarify their notions of what each level represented. Then they practiced coding essays. Each rater coded 10 essays and scores were compared. They practiced on three different occasions after the initial explanatory session. The goal was to establish inter-rater reliability at 0.70 on the practice essays before beginning to rate the sample essays. Subsequent training sessions involved practicing coding essays, checking scores for congruence and discussing the reasons for lack of agreement. When training was completed and inter-rater reliability was established at 0.70, then the raters independently scored the essays for each administration naive to the identity of the students creating the essay. For any essay where the two primary raters did not reach initial agreement, the rubric creator and trainer for the two raters coded the essays. In this way, all essays were assessed with the same rating by two raters. Initial inter-rater reliability estimates for the two primary raters for the data in this study was 0.60. To overcome the reduced consistency found in initial rating, the rubric creator and trainer coded all essays that did not receive matching scores. In all cases, this expert rating matched one of the initial codes; therefore, all reported values are based on converging values from two independent raters.
The results examined two primary issues. First, the study evaluated the utility of the writing rubric by comparing critical thinking scores and basic writing indicators. Second, it explored the scores generated through this rating scheme and investigated differences based on gender, as well as the impact of using computers to write the posttest essays.
Essay analysis: The inter-correlation matrix in Table 2 shows that the writing sample score was correlated with number of words and the subset of Watson-Glaser items determining inference. The Watson-Glaser Critical Thinking Total Score approached a level of statistical significance (p<0.03), but with the given number of variables in the analysis, a significant relationship could not be established. In the second essay, the score on the essay rubric was significantly related to (a) score on the first essay, r = 0.32, (b) Watson-Glaser total score, r = 0.29 and (c) number of words produced, r = 0.34 (p<0.005). The correlation between rubric rating scores and deduction and interpretation approached significance in this sample.
Overall, the results suggested that the method for scoring the writing sample is not a direct measure of critical thinking but is mainly related to components of critical thinking. The significant relationship with the number of written words was not a feature of the rubric. It is possible that students completing the writing samples would be able to perform more satisfactorily on the total writing rubric scoring system.
Factors influencing writing performance: The results of the MANOVA revealed significant main effects for gender, method of writing at WS-2 and the repeated factor (time). Also, the interaction between method of writing at WS-2 and time was significant (Table 3). As such, Table 4 presents the gender comparison for this writing sample without disaggregation based on the method of writing for WS-2, which was not a relevant factor for this initial writing condition.
For WS-1 comparison, the single discriminant function represented a reliable relationship between the predictors and gender, χ2 (4) = 12.636, p<0.01. Examination of the structure matrix revealed that the gender differentiation was a product of writing production and not writing quality scores. The structure coefficients for the 4-measures taken during WS-1 revealed that the number of sentences was the strongest predictor of gender differentiation (structure coefficient = 0.99), with number of words (0.78) and number of paragraphs (0.67) exceeding the 0.63 criterion established by Comrey and Lee (1992) to indicate very good coefficient loading. The writing quality score coefficient (0.24) did not reveal any reliable difference between girls and boys in ratings on the scoring rubric during the initial writing sample. To examine the effects of gender and use of word processors on the students writing performances across the two conditions, a second discriminant analysis was conducted. To simultaneously examine the effects of gender, method of writing at WS-2 and the repeated factor, the discriminant function analysis was designed to predict membership in one of four contrived groups: males/word processing, males/handwriting, females/word processing and females/handwriting.
Table 2: | Intercorrelation matrix for writing variables |
aWatson-Glaser total score is a combined score derived from all other WG subscores. *p<0.005 |
Table 3: | Repeated measures analysis of variance |
Table 4: | Gender and writing condition performance averages |
aAll participants completed WS-1 as a handwritten exercise, so only gender differences are displayed in this table |
Table 5: | Change score averages for gender and writing condition groups |
To capture the effect of the repeated factor, the predictor variables were the participants change scores on the four ratings derived from the writing samples (rubric rating, number of words, number of sentences, number of paragraphs) Table 4. With 4 contrived groups, there were three computed discriminant functions. Only the first function was a reliable predictor, accounting for 79% of variance between groups, χ2 (12) = 22.45, p<0.03. The second function accounted for 20% of the variance, χ2 (6) = 4.95, p>0.50, while the third function accounted for only 0.5% of the variance, χ2 (2) = 0.13, p>0.50. As such, only the first function is interpreted, which revealed that the differentiation among the groups was determined by writing productivity. Specifically, the discriminant function differentiated between those using word processors and those handwriting the second essay with two clearly meaningful variables: change in the number of words produced (structure coefficient = 0.93) and change in the number of sentences produced (structure coefficient = 0.65). The change in scores on the writing rating rubric (structure coefficient = 0.31) met the base minimum criterion for coefficient strength (Tabachnick and Fidell, 2001), while change in number of paragraphs was not a reliable predictor. No gender differences were revealed in the comparison of the change scores.
Examination of data shows a trend in which the use of computer (available only in WS-2) helped boys generate more text in response to the writing prompts, bringing the amount of written work in line with their female counterparts (Table 4, 5). Review of Table 4 also illustrates a dramatic difference on the essay rubric scores when compared with boys in the handwritten and word processing conditions. The difference in the essay rubric scores was statistically significant and in favor of the word processing group, t (36) = 2.50, p<0.01.
The present study examined the impact of technology on writing in a high-ability population. Although, critical thinking is a term widely used in education literature but only few studies are available that measure critical thinking. So, this study is a contribution to the field of high-bility learners' education. The results yielded some interesting considerations. First, the Dixon (1996) rubric used for coding and scoring essays was useful in revealing limited aspects of critical thinking.
The study indicated that on essay 1, when the students were required to handwrite their study, the boys produced an average of 229 words. On the other hand, on essay 2, those with access to computers produced an average of 420 words on their essays compared to 265 words from boys who composed handwritten essays on essay 2. This 83% increase in word production in the word processing condition may be attributed to the ease and speed in writing that they were able to achieve in the same amount of time. It is quite possible that the computer-experienced students in the population simply type faster than handwriting and were inclined to stop writing after a period of time rather than bringing their open-ended essays to a conclusion. Therefore, the benefit of use of computers to high-ability students (particularly males) appears to be a simple matter of speed and efficiency.
Kerr and Cohn (2001) stated that the high-ability student may be continually frustrated by his inability to make his hands and body do what his brain insists to do.
Indeed, computer technology seems to bridge the writing gap for high-ability boys. In addition to an increase in words, the boys in the computer group also increased the number of sentences and paragraphs in their writing, as well. Their mean scores on the dixon rubric (measuring critical thinking) on essay 2 averaged 4.1 as compared to 3.1 for boys in the handwritten group. The males outperformed their earlier attempt at writing using critical thinking with the use of computers. This suggests that computers may have a remediating effect on males.
The female participants in this study performed more consistently on both essays (average rating of 4.1 on essay 1 and 4.0 on essay 2). Their performance did not seem to change regardless of the use of format. They were overall more reflective and generated more words, sentences and paragraphs. The findings were similar to those of Li and Adamson (1995), who reported that high-ability girls in secondary school showed higher levels of interest (motivation) and confidence in their skills for English (language arts) than boys. Overall, on all measures-critical thinking, number of words, sentences and paragraphs generated-females scored higher than males in the handwritten condition. The most significant finding of this research was the improvement shown by the males when they were able to compose essays on computers.
CONCLUSION AND RECOMMENDATIONS
The boys produced significantly more words, sentences and paragraphs by using computers than those who did not use computer to write and received higher ratings on a structured rubric. Girls scored identical grades in both conditions (handwritten and computer) and performed consistently at par with the boys using computers. The female participants performed more consistently on both essays (average rating of 4.1 on essay 1 and 4.0 on essay 2). Their performance did not seem to change regardless of the use of format. They were overall more reflective and generated more words, sentences and paragraphs. The most significant finding of this research was the improvement shown by the males when they were able to compose essays on computers. Explicit instructions about clear statement of an argument or a proposition to ensure effective essays, importance of evidence to support and develop a line of thinking or conclusion and identification of a response to counter arguments should be given in all stages of writing. Teachers should devote some class hours to oral discussion of an issue prior to actual writing. Because in this way, the students will have opportunity to discuss all possible evidence, counter-arguments and refutations. This activity should be a part of the brain storming or planning stage of the writing process. Provision of well-defined and specific writing prompts is a way out to respond to lack of instruction. A student audience awareness program should be introduced to overcome the poor response to opposite views and to interviews. Further investigations should be extended beyond the descriptive statistics and content analysis. A strategy for online teaching critical thinking in advanced teaching and learning in electronic forums should be established.