Increasing science FCAT scores using

Part 1 Introduction

Overview

Each and every year the results of the Florida Comprehensive Assessment Test (FCAT) science proves that the students who attend the Florida Schools are far below their grade level.  The controversy surrounding science education over the past 90 years has centered on what information should be included in the science curriculum, who should receive this information, and how science information should be taught.

There is a growing concern about how to teach science in order to produce higher science standardized test scores.  Both educators and the general public are aware that critics blame the educational system for poor science literacy and low science concepts comprehension skills among students in secondary schools (Bower, 2002).

In the early part of the century, science learning was a low priority in education, mostly consisting of practical topics such as health and hygiene (Scruggs, Mastropieri, Bakken, & Brigham, 2003) by the late 1950’s however, science education drastically changed because of the Soviet Union’s launching of the Sputnik satellites. The United States felt inferior in their scientific and technological pursuits and responded by initialing reforms to produce better science curriculum and materials (Scruggs et al., 2003). New science programs quickly emerged. Within 15 years of Sputnik dozens of science curricula were developed that stressed the process of how to learn science (Shymansky, Kyle, & Alport, 1993) rather than the memorization of facts, laws, and theories (Klopfer, 1991). In the mid 1980’s these instructional approaches lost their appeal, in part, because of the “back to basics” movement (Shymansky et al., 1989).

            Within the last decade, the United States, threatened by the technological strength of Japan, has reexamined its Science education at all levels (Blough & Schwartz, 1990).  Various professionals in the field of science education called for reform as United States Secretary of Education, William Bennett (1986), states in his report on secondary  Education, “we need a revolution in elementary-school Science. There is probably no other subject whose teaching is so at odds with its true nature” (p.27). President Bush’s administration, composed a document containing six educational goals to be accomplished by the Year 2000. Goal number four states that by the year 2000 United States students will be first in the world in Math, Science and Technology. To further the efforts of reform, the National Council on science and technology Education, a distinguished group of scientists and educators appointed by the American Association for the Advancement of science (AAAS), authored Project 2061, a set of recommendations promoting scientific literacy. Science for all Americans (Rutherford & Ahlgren, 1990), the first report of Project   2061, recommends basic learning goals for all American children. This reform of science education seeks to produce citizens who are scientifically literate (American Association for the Advancement of Science, 1989). Questions have arisen, however, concerning which citizens are included in the phrase “all American children.” In the original document the AAAS defined “all” as 90% or more of the student body. In a revised statement (Rutherford, 1992) AAAS interpreted “all” to mean “all students, in all subject, all grades, and all levels of ability” (Cawley, 1994,p.68).

            Not only does Science for all American emphasize what students should learn and which students learn, it also recognizes the importance of how science is taught and how students learn scientific information. Rutherford and Ahlgren (1992) content that “people have to construct their own meaning regardless of how clearly teacher or books tell them things. Mostly, a person does this by connecting new information and concepts to what he or she already believes”(p. 186). This view of learning draws on a growing body of research about the nature of learning. The focus has shifted away from examining the external factors of learning, towards examining cognitive factors.

            Today cognitive scientists are investigating “the mental process underlying activities such as perceiving, thinking, and learning” (Bransford & Vye, 2006,p.173), a perspective acknowledging that “individuals construct ‘models’ or ‘schemes’ which are used to interpret their experiences” (Driver & Oldham, 2006,p.107). This philosophical stance is the foundation of a theory about learning called schema theory. Schema theory focuses on the development and structure of knowledge. The central proposition is that prior knowledge interacts with new knowledge and influences the students’ interpretation and recall of new information.

            Most theorists trace the term “schema” to Bartlett’s (1992) idea involving interacting cognitive structures called “schemata” (Ausubel, 1993; Holley & Dansereau, 2001; Rumelhart & Ortony, 1997). Rumelhart (2000) calls schemata “…. The building blocks of cognition. They are the fundamental elements upon which all information processing begins” (p.33). Schema theory has proven to be useful in providing a framework for studying the activation of appropriate existing schemata that facilitates comprehension and recall (Holley & Dansereau, 2001). Therefore, research pertaining to schema theory has mostly been conducted in the area of reading comprehension (Anderson, 1994; Rumelhart, 2000; Rumelhart & Ortony, 1997). Schema proposes an explanation for reading comprehension that emphasizes an interaction between the reader’s prior knowledge and the information provided in the text. This interactive model differs from the earlier linear model of reading comprehension (Adams & Collins, 1995).

            Another closely related theory, meaning reception learning, was developed by Ausubel (1993, 1998). Ausubel explores how “individuals comprehend, learn, organize, and remember the large volume of meaningful verbal materials which are presented to them by an educational agency such as the school” (1993,p. xi). Meaningful reception learning theory expands on schema theory by suggesting that the process of interaction between new information and prior knowledge occurs more readily if new information is subsumed or integrated with already existing knowledge. Furthermore, the theory hypothesizes that new meaning will be enhanced if the existing knowledge or cognitive structure is clearly and concisely organized (Ausubel, 1993, 1998).

            Schema theory and its variant, meaningful reception theory, carry profound implication for reforms in science education since they can help explain how students construct, organize and retain scientific concepts. Driver and Oldham (2006) emphasize that the development of meaningful learning theory necessitate rethinking science teaching. Researchers such as Ausable (1993) have attempted to translate meaningful reception theory into useful applications with the development of advanced organizers. Other applications emerged out of the research on advance organizers, such as structured overviews (Barron, 1996), followed by graphic organizers (Barron & Stone, 1994). Several studies focusing on these applications have been conducted in science because of the need to assist students in their comprehension and recall of information presented in expository textbooks (Griffin & Tulbert, 1995; Horton, Lovitt, & Bergerud, 2000; lovitt & Horton, 1994; Moore & Readence, 1994)

            Vocabulary graphic organizer, the focus of this study, have been operationally defined by Barron and Schwartz (1994) as learning tasks that require the learning to “manipulate vocabulary to schematically depict subordinate, parallel, and super ordinate relationship among the terminology” (p.276) however, in reviewing the literature graphic organizers have included fairly simple visual, hierarchical diagrams (Bernard, 2000) to highly complex displays of words (Hawks, 1986).In the literature graphic organizers generally fall under a larger category of graphic structure. Several terms have been used as an overall description of these structures. For example, Holley and Dansereau (1994) refer to a variety of non-linear representations as “spatial learning strategies” (p.9). Jones, Pierce, and Hunter (1999) describe various illustrations of verbal statements as “graphic representations” (p.20). Hudson, Lignugaris-Kraft, and Miller (1993) used the term “content enhancers” (p.106) to characterize several visual techniques used to help students select, understand, and retain critical information, and Crank and Bulgren (2003) used the term “visual depictions” (p.142) to refer to all structures depicting and clarifying common information patterns. “From simple time lines to complex matrices; their purpose is to organize information in a manner that makes the information easier to learn” (p.140). Crank and Bulgren further differentiate this graphic structure by classifying them into three main types of structure: central and hierarchical, directional, and comparative.

Types of Graphic Structures

Central and Hierarchical Structures

            Central structure focus on a single central idea and all other information is organized around this idea. Examples of these kinds of structure include concept maps (Novak & Gowin, 1994) and networking (Holley & Dansereau, 1994).  In contrast, a hierarchical structure ranks the information by placing supraordinate items above subordinate ones. Barron’s (1999) graphic organizers fall into this category.

Directional Structures

            A directional structure is a graphic device which places information in a sequence to convey meaning. For example, Specific processes such as photosynthesis could be displayed in a directional structure using arrows leading from one item to another.

Comparative Structures

            A third major type of visual depiction is a devise that indicates relationships between at least two concepts that are compared or contrasted. Typically a matrix is used to arrange the informational elements into columns and rows. These structure have been referred to as relationships charts ( Bos, Anders, Filip, & Jaffe, 1999).

            Researchers are promoting these graphic structures to activate in-depth learning of science concepts rather than a broad shallow understanding of scientific concepts. They can enable student to perceive relationships between and among concepts and helps students recognize common organizational patterns of information in text (Ellis, 2004). This skill is important because recognizing the structure of the text provides a framework for learning and remembering (Rice, 1994). Graphic structure assists students in developing a sense of text structure by displaying information in meaningful patterns (Chambliss, 2004).

            Students who have difficulties constructing knowledge through textbook instruction and/or lecture may find graphic structures to be an especially powerful tool. Various graphic structure have been recommended for students with learning disabilities, a group of students who generally experience difficulties when trying to understand what they read (Chambliss, 2004; Crank & Bulgren, 1993; Griffin, Simons, & Kameenui, 2001; Horton & Lovitt, 1999; Lovitt & Horton, 1994). To make science learning appropriate and relevant for all student these application have been examined in the instruction of special education as well as general education students (Crank & Bulgren; Darch & Carnine, 1996; Griffin et al,; Horton & Lovitt, 1999).

Statement of the Problem

This state of science education in the United States has brought many challenges to Florida public schools. After pilot testing in the spring of 2002, the state of Florida’s Department of Education administered the science FCAT to students in grades 5, 8 and 10 in all Florida schools in spring, 2003. The science FCAT is based on the Sunshine State Standards for science and has been developed to assess student learning of science in Florida’s public schools.

Standardized tests are considered by many to be an important means of ensuring that America’s educational system remains accountable for providing the best learning opportunities possible for all children. Although no test can perfectly measure what a student has learned, standardized testing may provide a quantifiable and visible estimate of what a student knows. This numerical estimate may then provide a reference point against which future achievement can be measured. These achievement reference points can be valuable in different ways to teachers, administrators, schools, districts and states, as well as to policymakers at all levels of government. Data from standardized tests can be used to identify trends at the national level, and by organizations to study various curricular and testing issues.

For teachers, standardized test data, such as FCAT, can be used to determine patterns in student performance. Teachers can determine which students are not working to their ability by identifying students with high test scores and mediocre or inconsistent work in class. Teachers can also identify students who seem to be working above their potential, and those students who may be consistently in need of extra help. In general, teachers can use standardized test scores as a tool in determining the needs of their students. Administrators can use standardized test data to see how their school’s achievement compares to other similar schools. They can work with teachers to set goals for student achievement. In addition, administrators can use student test scores as an incentive for effective instruction on the part of teachers. Teachers who are aware that student scores will be carefully considered may be more likely to do everything they can to help their students succeed. Finally, student test scores may help administrators identify teachers who are ineffective at helping students learn.

School districts and states also use standardized test data to compare student achievement and to look for trends. Standardized test data can help districts and states decide how best to allocate money and resources. Data obtained from standardized tests can also be helpful to organizations by giving diagnostic information needed for designing better tests. Data can also be helpful in determining what adjustments could be made to curriculum and instruction on a large scale. Particularly now that there are national standards and standards for 49 of the 50 states, standardized assessments may provide a way to determine how well teachers are incorporating standards into their teaching.

Although there is a definite role for standardized testing, there is cause for concern that test scores might be used in what some consider harsh accountability programs. The term high-stakes is used to describe standardized tests in Florida and some other states because of the important nature of the decisions based on standardized test scores. FCAT scores, and the Governor’s A+ Schools Program, (2000), for example, are used to make decisions on issues such as graduation, retention, increases in teacher salaries and school funding, and even the placement or removal of school principals.

Science instruction for students has been inadequate at best, and non-existent at worst (Cawley, 1994; Holahan et al., 1994; Patton, Pollaway, & Cronin, 1990). Ysseldyke, Thurlow, Christenson, and Weiss (1987) reported that the time devoted to science education for students in the general classroom was less than that devoted to reading. Similarly, Patton et al. found that 38% of the students in self-contained classes received less instruction in science.  Researchers have also questioned the appropriateness of science instruction. Parmar and Cawley (1993) stated “very little information is available regarding the appropriateness of the science instruction that students are receiving in general education (p. 518). It can be assumed that many students are receiving inadequate instruction in science. A study examined report card grades in science for 9th-through 12th-grade students with mild disabilities. Fifty to sixty percent of the 500 participants received grades of Ds and Fs (Cawley, Kahn, & Tedesco, 1989).

                  Another major problem for students in science is the science teacher’s dependency on textbooks. Although the primary thrust of science education today is an inquiry approach (National Research Council, 1996), a vast majority of science programs are textbook driven.  Yager (1989) points out that over 90% of all science teachers use a textbook 95% of the time. In a study sponsored by the National Science Teachers Association, it was found that while many teachers claim to use hands-on activities there is a heavy reliance on textbooks as well, particularly in grades four through six (teters, gabel, & Geary, 1984).

                  From the research, it is evident that students are experiencing difficulties in science achievement. The pervasiveness of science literacy for all Americans requires that educators cultivate techniques to help their students develop sound scientific knowledge. Selected research reveals that the use of graphic organizers can assist the student in gaining a deeper and more complete understanding of relationships between and among scientific concepts (Lovitt & Horton, 1994).

Harris and Pressley (2001) point out that “errorful knowledge construction is common among children, and can indicate either partial understanding on route to true comprehension, or incorrect knowledge and misunderstanding that will lead to further learning problems” (p.303). Graphic organizers could be used to assist students in characterizing relationship, and comparing, contrasting, and sequencing the concepts they are attempting to understand. Although information presented in textbooks and lectures typically contains super ordinate and subordinate concepts, students often have difficulty identifying the relationships between them (Deshler et al., 2006). Graphic organizers may provide a tool to assist students with learning disabilities in the deep understanding of both super ordinate and subordinate concepts.

Significance of the Study

                 Research on science education for students is usually narrow in its scope “The field lacks validated curricular and any significance set of knowledge relative to effectiveness of instructional approaches in science” (Cawley, 2004, p. 67). Without effective instructional approaches, students may not acquire a sound foundation of scientific knowledge. If students develop faulty foundations in the elementary years, they may experience difficulties when they study more complex scientific concepts in secondary school.

                  Textbooks play an increasingly important role in learning these science concepts from about the fourth-grade on (Armbruster, Anderson, & Meyer, 1991). It is in the middle grades (i.e., four through nine) that students have difficulty selecting information from the text, cognitively organizing the information, and integrating that information into existing cognitive structure (Mayer, 1992). There is a need to conduct studies with this age group because of the added demands in their science instruction, the importance of establishing sound science concepts, and the necessity to acquire good learning techniques prior to high school.

                  Researchers and educators have suggested using graphic organizers with students who experience difficulties when learning science (Darch & Carnine , 1996; Griffin et al., 2001; Horton, Lovitt, & Bergerud, 2000). Recently published textbooks and manuals also suggest the use of graphic organizers for science instruction (Deshler et al., 1996;  Mastropieri & Scruggs, 2003; Salended, 2004; Wood, 2003). Additionally, current science textbooks include graphic organizers with each chapter ( Addision-Wesley, 1995; MacMillan/McGraw-Hill, 1995). If graphic organizers are being implemented in the general education science textbooks, students who receive their science education in the general classroom will be exposed to them; yet it is unclear if graphic organizers actually assist students. Many researchers have recognized the need to validate the effectiveness of graphic organizers for students (Griffin et al., Griffion & Tulbert, 1995; Horton & Lovitt, 1989; Hudson et al., 1993).

                    Based on a review of the literature, Griffin and Tulbert (1995) Make recommendations for strengthening the graphic organizer literature. Two of their suggestions relate to the present study: a) further examination of the use of graphic organizers (Alvermann, 2001; Balajthy & Weisberg, 2000), and b) careful examination of the design features of graphic organizers.

Limitations of Study

                     The limitations of this study included problems with the reliability of the instruments and the sampling. As part of the study, the researcher developed a pre-and post-science mastery test. Reliability was determined by the internal consistency of the test. The coefficient, calculated by using the Kruder-Richardson 20 formula, for the pretest was .54. This score falls in the moderate positive range. The coefficient of the posttest was. 36, a score that falls in the low positive range. The low reliability of both the pretest and posttest was a limitation to the study.

                The study was also limited by sampling difficulties. The sample for this study included three different schools in a school district. The researcher selected the sample based on the available number of 8th grade science students ages 13-14 yrs. of age.

               Additionally, the demographic characteristics of the sample, two groups of 8th-grade science students were significantly different in their racial composition and socio-economic level. This was problematic because the groups were determined to be non-equivalent on a particular characteristic (Almasi, 2003).

The implementation will only run for 6 weeks and all the students will be given a pre and post practice Science FCAT exam to gauge learning improvements.

Delimitations of Study

                 This study investigated the effects of graphic organizers on FCAT science scores in 8th grade science students. Because many factors are involved in the formation of a student’s cognitive structure, not all variables can be controlled when studying the learning process. Therefore, this study is limited and the following delimitations should be noted:

1. 8th grade science students will serve as the participants in this research. The results of the study, therefore, generalize only to 8th -grade students in Florida high school.
2. Since the graphic materials of the science curriculum were all at the 8th grade level, it is not possible to generalize to other grade levels of science instruction without further research.

Research Questions

                   Based on the need for further research that helps meet the learning needs of 8th- grade students in science, this study will explore the following question: Using science instruction with 8th-grade students, is there a difference in the mean scores in the FCAT among: (a) students who receive the graphic organizer treatment, (b) 8th grade students who are not receiving the treatment.

The Context, Input, Process, Product (CIPP) evaluation model, created by Daniel Stufflebeam and his colleagues (1971) to improve contribution to the decision-making process was employed in this study.

The theoretical framework and research literature that supports this dissertation falls into four sections. The first section of the literature review relates to the problems that many students face in science in light of the recent science education reform movement and the state of science education for students.  The second section of the review will examine the power of cognitive strategy instruction and cognitive science and science reform. The third section of the literature review deals with the work that has been done previously on graphic organizer. The fourth section will provide an overview of the historical evolution of educational evaluation and Daniel Stuffelbeam`s CIPP Evaluation Model.  This paper will draw directly on this work and the suggestions of earlier researchers and research on graphic organizers for students in science classes.

The State of Science Education

Reform efforts in science education have recently gained national attention. Studies indicated that science has been a neglected area of education, and many students are not prepared to face the rapid growth in scientific knowledge.  Science education has also been a neglected area; however, with the national goal to be first in the world in math, science, and technology, scientific literacy for all American has become an increased area of interest. As part of the reform effort, researchers are now examining instructional approaches to assist students in science learning.

            Influences on this reform effort include cognitive scientist’s recent in how new information is processed and stored in memory. Schema theory and its variant, meaningful reception learning theory, have emerged as possible explanations of how students construct, organize, and retain scientific concepts. The two theories emphasize the importance of existing cognitive structures in comprehension and recall.

            Based on these theories, researchers have recommended various techniques to assist student with and without disabilities in learning science. In particular, graphic organizers have been investigated to determine if they enhance science learning.

            Science education is changing because of the demands by professional organization, researchers, and educators ( Cawley, 1994; National Research Council, 1996; Rutherford & Ahlgren, 1990) to improve science achievement for all students. The American Association for the Advancement for all students. The American Association for the Advancement of Science (AAAS, 1989) characterizes the seriousness of the problem.

            ….the fact is that general scientific literacy eludes us in the United States. A cascade of recent studies has made it abundantly clear that both by national standards and world norms, United State education is failing too many students—and hence failing the nation (p.3).

            In the United State, the lack of second conceptual scientific framework is evidenced by the results of student performance on the science section of the National. Assessment of Science Progress (NAEP 1978a, 1978b,1979a, 1979b). The outcomes suggest various weaknesses in science literacy for many students. Based on these findings, Eylon and Linn (1988) concluded that, in general, student use scientific terms imprecisely, develop limited or incorrect notions of causal relationships, view scientific phenomena in isolation, and assume that science is memorized not understood. Another study indicated that among 13 industrialized countries, high school students in the United State score 9th in physics, 11th in chemistry and last places in biology (International Association for the Evaluation of Education Achievement, 1988).

            Some researchers ascribe these results to weaknesses in science instruction. For example, there have been several studies that focus on the problem of inadequate training for teachers who teach science ( Elliot, Nagle, & Woodward, 1986; Patton, Polloway, & Cronin, 1990; Shymansky, Woodsworth, Norman, Dunkhase, Matthews, & Liu, 1993).

            Shymansky et al. raised question about whether or not science teachers are ready to cover topics in greater depth. The results of their study indicated that teacher, as well as their students, harbored erroneous ideas about science topics. This is a problem because one of the basic tenets of science reform is the need for more in-depth coverage (Eylon & Linn, 1988). This notion is exemplified by the phrase “ less is more” (Shymansky et al., p.737), an idea that studying fewer topics more fully is preferable to covering many topics superficially (AAAS, 1989; Shymansky et al.).

            However, inadequate teacher training may only be part of the problem of weak science instruction since there are also cases of teachers fostering serious misconceptions about topics in which they are adequately trained (Lawrencz, 1986). The teachers in Lawrencez’s study were all enrolled in a physical science in-service training course, yet they exhibited misconceptions when were tested on various topics in physical science. Other studies have found that even adequate training does not correlate with effective teaching.

            Stepans andMcCormack (1985) found that teachers in an elementary education training program did not increase their understanding of scientific concepts simply by taking more science courses.

            Another problem related to weak science instruction is school districts tendency to treat science as less valuable than reading mathematics (Elliot et al., 1986); therefore, teacher spend less time teaching science. A study by Raizen (1988) verifies the limited time spent on science education: 18 minutes per week in grades Kindergarten through three and about 29 minutes in grades four through six. Although the teacher in Lawrenz’s (1986) study, spent more time per week (90 minutes or less), these were teachers characterized as being interested and inclined toward science education.

            Other researchers have attributed problems in science achievement to the reliance on textbooks. Wood and Wood(1988) state:

            …what students have learned in school about science will be largely related to what they have read and understood from the science textbook. Therefore, the inability to comprehend science textbook continues to be a major deterrent of student achievement in science. (p. 561)

Recent research on science textbook describes obstacles student face when trying to learn science. The results of Staver and Bay’s study (1989) indicated that the reasoning demands of primary level science textbook were above the developmental of a large segment of the students for whom the textbooks were written. Furthermore, as the body of scientific knowledge increases, textbook companies respond by adding more new concepts and vocabulary to the latest editions. Consequently, students are forced to spend an excessive amount of time engaged in lower level cognitive processes, like decoding, rather than in-depth understanding (Woodward, 1994). Some students also have a difficult time discerning the salient ideas in the text (Dee-Lucas & Larkin, 1988); therefore many smaller concepts remain fragmented. An investigation of widely used commercial textbooks ( Mastropieri & Scruggs, 1994) revealed that these books do not use proven instructional principles, such as activating prior knowledge and previewing concepts and vocabulary. This finding is particularly discouraging.

Research on Graphic Organizers

Graphic organizers are visual models that illustrate key concepts while allowing students to synthesize, organize, and later internalize the information in a particular piece of writing (Gallavan and Kottler, 2007). Essentially, graphic organizers should allow the learner to simplify the information (Gallavan and Kottler). Graphic organizers are used to help students to distinguish important ideas from those that were insignificant, as well as understand the connections and inferences within a piece of writing (DiCecco & Gleason, 2002).

Once organized, students can then process, and simplify, the information into a coherent piece of writing. With the assistance of the organizers, students are often able to write a more coherent, organized essay (Burke, 2004). For example, a certain type of graphic organizer may allow a student to distinguish between main ideas and smaller details by visualizing the cause and effect of major relationships in a text. As a result students are able to and engage in the writing process with confidence due to their understanding of how to use such organizational tools as graphic organizers (Almasi, 2003).

The most efficient graphic organizers allow students to separate information into chunks, or small clusters, so that students are able to effectively recall main ideas and key concepts (Willis, 2007). Willis asserts that related brain-based research has indicated that the brain processes information in chunks as it makes meaning through patterned information. Therefore, graphic organizers work in conjunction with the way we learn. Graphic organizers serve as a bridge between what students already know and the new information that is being processed (Edmonds &Vaughn, 2006). These learning tools assist students in organizing information by visually identifying key concepts or relationships (Fountas & Pinnell, 2006). When higher levels of comprehension are achieved, students will be more capable or organizing factual information from expository texts (DiCecco & Gleason, 2002) and which, subsequently, will lead to the improvement of their writing ability.

Types of Graphic Organizers

Graphic organizers assist in the comprehension of new material because they allow students to create links between existing information and new knowledge (Derner & Reardon, 2004). There are a number of different types of graphic organizers, each with its own purpose. Some examples of graphic organizers are concept maps, cause and effect charts, sequences charts, K-W-L charts, and compare-and-contrast diagrams (Gower & Saphier, 1997). A cause and effect chart is a type of chart that allows the students to keep track of main ideas in one column while anticipating the results, or effects, in another column (Fagan, 2003). Cause and effect charts are one of the most common types of graphic organizers used for students of all grade levels because of their versatility. These types of organizers can allow students to organize their ideas for an essay, outline the events leading up to a specific outcome or illustrate the actions of a character in a story (Baxendell, 2003). Cause and effect charts allow students to illustrate what they have learned while analyzing, and anticipating, what events will come next (Gower & Saphier, 1997).

Compare and contrast diagrams are usually presented as two or three circles that have overlapping portions, allowing students to see the both the commonalities and differences between a subject(s) (Naylor, 2006). The Venn diagram is a commonly used compare and contrast graphic organizer that allows a student to compare and contrast two or three different subjects simultaneously (Gallavan & Kottler, 2007). Burke (2002) stated that the structure of a Venn diagram allows students to list the distinguishing characteristics of the subjects, as well as explain their similarities. This type of diagram is helpful in terms of organization and essay writing as the information can easily be translated into thoughtful paragraphs (Burke). Baxendell (2003) stated that compare and contrast diagrams are commonly used in classroom instruction at all educational levels.

A concept map is used to represent the relationship between primary and secondary concepts (Tseng, 2007). A concept map is meant to provide a student with an overview of important vocabulary and key concepts (Edmonds & Vaughn, 2006). This type of organizers allows a student to specifically understand how smaller ideas and concepts are associated with a main idea (Gallavan & Kottler, 2007). A concept map often looks like a web or a chart as it allows students to brainstorm ideas to stimulate their learning and determine key concepts (Edmonds & Vaughn, 2006). Edmonds and Vaughn (2006) noted that a concept map can be modified to be used in any content area at any grade level.

A Know-Want-Learn, or K-W-L, chart is set-up in columns to allow students to brainstorm, determine the purpose for their reading or research, and monitor comprehension (Szabo, 2006). K-W-L chart is used to specifically help students to organize their research as it generates focus. Fisher, Frey, and Williams (2002) stated that the format of this diagram is very specific, asking students first what the already know about the information. Next, students are asked to examine what they want to know before they have completed their reading and/or research. Lastly, after the reading or research is completed, students answer the third, and final question, what did you learn about the topic. A K-W-L chart serves as a way for students to both assume and anticipate information while allowing the student to engage in the process of inquiry

(Gallavan & Kottler, 2007). Allen (2004) amended this chart to incorporate other facets of knowledge forstudent learning. Allen developed a Background-Know-Want-Learn-Question chart to allow students to capture additional knowledge. In the background portion students list all background knowledge of the anticipated topic. The question portion encourages students to expand upon what they have learned through their research by asking additional, related questions (Szabo, 2004). For example, a cause and effect chart would commonly be used in literature to analyze plot the happenings of a character while a K-W-L chart may be used to organize a wide variety of information for research purposes (Gallavan & Kottler, 2007).

Sequence charts are usually visually constructed to allow students to follow the sequences of an event or the flow of ideas through the use of arrows, numbers, or lines (Baxendell, 2003). Sequence charts are frequently used to help students identify a chain of events in a variety of content areas, including math, science and social studies (Baxendell). Often sequence charts can be used while students are planning their writing as they allow students to visually represent the flow of their ideas (Andrade & Saddler, 2004). In order for a sequence chart to be effective it should be clearly labeled, detailed and organized.

History of Graphic Organizers

Johannes Amos Comenius, a 17th century educator, first wrote about effective ways to teach others information through the use of pictures (Egan, 1999). Ausubel (1960) expanded on the idea of using visual aids to assist in the learning process through his theory of meaningful learning. Ausubel maintained that learners must be able to anchor newly acquired information to pre-existing concepts or ideas. By linking information the learner can engage in the process of meaningful learning by using visual maps of important concepts to focus their learning (Kelly & Odom, 1998).

Sousa (1995) stated students first used visual maps, or graphic organizers, in science and math to assist them in understanding difficult concepts. The most common compare-and-contrast diagram is the Venn diagram which was first used by John Venn in the late 1800’s. Its purpose was to show similar relationships between mathematical concepts; however, these diagrams later became applied to more content areas, such as social studies or reading (Sousa, 2004). Their use is now widespread and effectiveness of graphic organizers in learning can be seen at any grade level within all content areas (Egan, 1999).

Today graphic organizers are effective strategies in both teaching and learning. Educators can use a graphic organizer during several different areas of a lesson, such as during a class discussion, as a forum for taking notes or after reading selected information to assess learning (Gallavan & Kottler, 2007).

Research on Graphic Organizers in Science Education

            Science education research has attempted to operationalize meaningful reception theory by promoting graphic structures. Ausubel (1963, 1968) drew on the principles of this theory when initially developing advance organizers. Ausubel (1963, 1968) contended that information introduced to the student, in advance, provided the framework to connect ideas presented at a later time.

Interest in advance organizers expanded in the 1970’s although there was inconclusive evidence as to their effectiveness (Barnes & Clawson, 1975). Of the 32 studies Barnes and Clawson reviewed, they found only 12 that reported advance organizers facilitated learning. When separately considering the variables: length of study, ability level of subjects, grade level of subject, type of organizer, and cognitive level of learning task, the authors concluded no clear patterns emerged concerning these variables and their effect in facilitating learning. Thus, Barnes and Clawson concluded that advance organizers generally do not facilitate learning.

            Subsequently, other researchers proposed that an organizer should be a visual-spatial representation rather than strictly prose. These teacher-constructed diagrams that visually depicted relationships were called structural overviews ( Barron, 1969; Estes, Mills, & Barron, 1969).Three related studies examined whether or not the two types of organizers were of practical use in the classroom. The studies assessed the effects of the devises relative to three learner variables: (a) grade level ( Barron, 1972), (b) general reading level as measured by a standardized test (Estes, 1972), and (c) passage-specific reading ability as measured by a close readability test on the learning passage (Barron & Cooper, 1973). None of the studies reported significant differences between students who received advance organizers or structured overviews and those who received no introductory material.

            A fourth study ( Barron & Stone, 1974) attempted to explain the uncertain outcomes of research surrounding advance organizers by positing a rival hypothesis to  Ausubel’s ( 1963,1968). The researchers suggested that in studies where organizers enhance learning, the positive effect was due to the organizer signaling learners to create a meaningful learning set.”…. presentation of the trigger a conscious effort to incorporate new information on a non-rote basic” ( Barron & Stone, p.173). This hypothesis was an alternative to Ausubel’s (1963, 1968) view that advance organizers facilitate learning in two ways: (a) they provide a conceptual framework to which the more specific information can be related, and (b) they assist learners in discriminating between new ideas and similar or conflicting ideas in their existing cognitive structure.

            Barron and Stone (1974) developed a technique called a graphic postorganizer. One hundred and forty-one twelfth-grade students were assigned to three treatment conditions, graphic postorganizer group, graphic advance organizer, and control group. In the graphic postoraganizer condition, the student read a passage on the first day. On the second day they were shown how to construct a graphic organizer. Then they were placed in a group of three to five students and provided with another passage and a packet of index cards. Finally, they were asked to construct a group placed on index cards. On the first day the advance organizer group received their experimenter-constructed graphic organizer before reading the passage. On the second day they reread the passage. The control group read the passage on both days.

            The results indicated no significant difference between the graphic advance organizer and the control group. However was a significant difference in favor of the student-constructed postorganizer condition over the advance organizer group? The authors interpreted this outcome to support their hypothesis that the effects of advance organizers may be due to their influence on a learner’s reading/learning process rather than to an influence on cognitive structure. In other words, these effects can be distinguished in two different ways: graphic advance organizer effects on prior knowledge and graphic postorganizer effects on learning strategies, such as creating an effort at comprehension, processing information at different levels, and rehearsing information ( Moore &  Readence, 1984).

            Today structural overviews are generally referred to as graphic organizer, but unlike structural overviews, there are numerous variables, that characterize graphic organizers in the literature. They may be teacher-constructed, student-constructed, or teacher/student-constructed ( Moore & Readence, 1980). They may be presented in a prereading and/or a postreading position ( Simmons, Griffin< & Kameenui, 1988). The information may be depicted in several different format such as hierarchical/central (Horton, Lovitt, & Bergerud, 1990), comparative (Bos, Anders, Filip, & Jaffe, 1989), directional (Crank, 1991) and representation forms (Bergerud, Lovitt, & Horton, 1988). The amount of training for students using graphic organizers is another variable ( Alvermann & Boothby, 1986).

            In a meta-analysis of research findings from 16 grahpic organizers studies, Moore and Readence (1980) involved 2, 098 normally achieving students and used 92 measures of effectiveness of graphic organizers. It was found that graphic organizers had a small overall effect. The greater effect occurred when the graphic organizer was student-constructed after reading the text (in the post position). A follow-up study ( Moore & Readence, 1984), using 23 studies, produced similar results. There was a medium average effect size (M =. 57; SE =. 17) with the student-constructed graphic postorganizer treatment. The author concluded,”…graphic postorganizers seem to produce greater effects than graphic advance organizers. The key to this benefit seems to be student cautiously” (p. 15).

            The author’s cautions interpretation is appropriate since other studies have shown mixed results for comparing graphic organizers in the pre-and post-position. Simmons et al. (1988) sought to evaluate the effect of the position of the graphic organizers on sixth-grade normally achieving student’s comprehension and retention of science text when used as a teacher-constructed reading aid. The overall results suggested that teacher-constructed graphic organizers, whether presented in the pre or post position appeared no more effective than traditional instruction. One of the author’s plausible explanations dealt with the nature of teacher-constructed graphic organizers.

            “…Teacher-constructed graphic organizer treatment for all practical purposes ‘spoon-fed’ sixth-graders the critical information that experimenters gleaned from the text and housed in a mnemonic” (p. 20). Since the students in this study were not required to act on the textual information, the researchers felt they may not have deeply processed the information. The researchers   acknowledged these factors as a limitation of teacher-constructed comprehension aids. However, they further acknowledged the greater amounts of instructional time associated with depth of processing activities (Bean, Singer, Sorter, & Frazee, 1986).

            Bean et al. (1986) examined tenth-grade students who were divided into three different treatments groups: (a) graphic organizers used by a group who had been previously trained in summarizing and generating question about the text, (b) outlining by a group who had not been trained in these study strategies, and (c) student-constructed graphic organizers by a group who had not been previously trained in the strategies. The students in the graphic organizer group who were previously trained achieved significantly higher written recall scores than the other two groups. The researcher suggested that systematic instruction in teaching students to construct graphic postorganizers was most beneficial for students who had already received training in study strategies. They also recommended a 14 week training period for teaching average and above average students the complex process of reconstructing concepts for text using graphic organizers.

            The results of another study that focused on the training variable had similar findings, although the time allotted to their training period was far less. Alvermann and Boothby (1986) reported that the fourth-grade students, all average and above average in reading ability, who received 14 days of graphic organizer instruction recalled significantly more information than those who received no instruction in the use of graphic organizers. No significant difference was found between those who received seven days of training and those who received none. Alvermann and Boothby suggested that the results substantiated the hypothesis that the length of treatment is a critical variable in classroom training studies on graphic organizers.

            In contrast, the finding of another study reported mixed results when focusing on training. Balajthy and Weisberg (1990) trained at-risk college students in both constructing graphic organizers and summary writing for a total of 10 hours over a 2-week period. Each of the four sessions lasted 40 minutes. The results indicated that intensive training did not significant improve passage comprehension or summarization skills. However, when the authors further investigated the variable they postulated that training in the use of graphic organizers may have assisted the learners with general comprehension. This finding may be very useful to poor readers whose academic performance is often hampered by low general comprehension skills.

            Although Barron and Stone’s (1974) study did not specifically focus on training, it further illustrates the diversity in training procedures. The participants in their study (141 tenth-and eleventh-grade students) were divide into three groups, a graphic advance organizer group, a graphic postorganizer group and a control group. The graphics postorganizer group received a short demonstration about how to construct their own graphic postorganizers. Results indicated that the graphic postorgainzer group scored significantly better on the dependent variable than the graphic advance organizer group, a group that did not receive training in graphic organizers.

            A graphic organize in science classes can allow a student to visualize elements of the new learning giving them the ability to more readily retain the new knowledge and develop new schemas (Sousa, 1995). Students will begin to anticipate the type of material found in an

expository text, thus allowing them to build a new schema and later retrieve the information with ease (Almasi). According to Dearner and Reardon (2004), meaningful learning occurs when new knowledge is connected to pre-existing schemas, since schemas have already been established and new knowledge is being absorbed. When used effectively during classroom instruction, visual materials could serve as the bridge between existing and new knowledge for a student (Brunn, 2002).

Graphic organizers allow students to convert the information in ways that will actually enhance their comprehension abilities (Brunn, 2002). Willis (2007) stated that graphic organizers allow students to think at high cognitive levels as they assist in the organization of information and the creation of new knowledge through the chunking of information. Graphic organizers can guide students in making better choices as writers, thus allowing students to be in control of their writing (Dean, 2005).

Organization is an extremely important element for students to learn when writing (Andrade & Saddler, 2004). Graham and Perin (2005) stated that struggling writers often do not attempt to create a plan for their writing or organize their information. Students are better able to grasp key concepts, and organize their information from a text, when main ideas are illustrated in a visual manner (Brunn, 2002). Therefore, by utilizing graphic organizers students will be better able to decipher between main ideas and supporting details (DiCecco & Gleason, 2002). In addition, students will learn how to visually organize key elements often found in expository text through the use of graphic organizers (DiCecco & Gleason).

The Importance of Organizing Cognitive Structure Relationships among Science Concepts

Research in cognitive science stresses the importance of organizing information, particularly in a hierarchical manner (Ausubel, 1963, 1968). “The importance of organization is due in large to the need to access stored information. It is difficult to conceive how human memory, or any large information storage and retrieval system could function without organization” (Goetz, 1984, p. 55). Research on the differences between the way novices and experts organize information suggest that hierarchical organization may contribute to better use of knowledge (Eylon & Linn, 1988). In a study by Eylon and Reif (1984) physics material was organized hierarchically by level of detail. This organization was taught to students in an individualized learning sequence that trained them in the assimilation and use of the hierarchical organization. Tests comparing performance of this group of students and those who studied the same topics in a linear manner indicated more gains for the group using a hierarchical organizational system.

            Other issues of organization in science education focus on reconceptualizing science from the perspective of related themes rather than disconnected units of instruction (Rutherford & Ahlgren, 1990). Finding from studies on science education emphasize the need for this more in-depth thematic coverage (Eylon & Linn, 1988). Ausubel described a process that enable the learner to cover a topic more in-dept. This principle is called “integrative reconciliation” (Ausubel & Robinson, 1969, p. 169). Integrative reconciliation refers to an attempt by the learner to view the relationship between concepts and not compartmentalize them (Starr & Krajcik, 1990). Ausubel pointed out that this principle is violated by many author of textbooks who compartmentalize and segregate topics or concepts within their respective chapters or subchapters, so that each topic is presented in only one of the several possible relevant places. “Hence little serious effort is made explicitly to explore relationships between these ideas, to point out significant similarities and difference, and to reconcile Nussbaum and Novak (1976) interpreted their findings concerning second-grade children’s notions of the Earth to be an example of Ausubel’s principle of intergrative reconciliation. That is, concepts of the Earth that were dissimilar, such as space and gravity, were seen as relevant to a larger conceptual framework of the Earth.

            Early studies by Ausubel (1963, 1968) have influenced the direction of science reform today. Several instructional principles are being advocated that are a direct result of studies in cognitive science, in particular, the use of graphic organizers in science education. This instructional strategy is based on the instructional principles characteristic of meaningful reception learning theory.

Implementing a Graphic Organizer in Science Classes

Baxendell (2003) believed that in order for a graphic organizer to be effective it must be both coherent and consistent. A graphic organizer should be easy to understand, or expressly coherent, so as to allow a student to immediately anticipate the main ideas of the new learning that will take place (Jeffries & Merkley, 2001). Concepts should be clearly labeled with their relationships visible to the student (Baxendell, 2003). Cooter and Flynt (2005) stated that when graphic organizers are direct, or easily understood, students are able to utilize them more effectively in their writing. The primary purpose of the organizers, in all content areas, is to make the association between concepts clear and well-defined (Edmonds & Vaughn, 2006).

Teachers should develop graphic organizers that are relevant to both the text and the unit that they are teaching (Edmonds & Vaughn, 2006). Dye (2000) emphasized that teachers must determine what the learning is and create an organized, yet detailed, graphic representation of the information that assists students in labeling key concepts and main ideas. Students will then be able to manipulate different types of organizers to fit their learning needs once they are consistently exposed to both their use and structure (Baxendell, 2003).

Conclusion

Students are often unable to clearly decipher the information in science concepts that they are presented with because they are unable to connect them with any real world meaning or application (Calkins, 1997). Graphic organizers allow students to connect ideas; therefore students can focus their ideas to create a working outline for science concepts, making the task seem less daunting (Burke, 2004). Students have a better sense of these concepts, as well as what is important, when using graphical representations to help them in the organization of their knowledge. This allows them to exude an amount of confidence that comes from an acute understanding (Burke, 2004). As students build their confidence, and become more comfortable with factual material through the use of graphic organizers, they will be more organized and therefore their writing understanding of science concepts will improve.

Educational Evaluation

Educational organizations decision-makers and stakeholders want to ensure that educational programs are accomplishing their intended purpose. They are interested in assessing the effects of educational programs by asking questions like “What changes occurred?” or “Are we satisfied with the results?” (French, Bell, & Zawacki, 2000). Therefore, educational evaluation is utilized by educational organizations to periodically assess their processes, procedures, and outcomes.

According to Wholey, Hatry, and Newcomer (2007), the field of educational evaluation provides processes and tools that workforce educators and developers can apply to obtain valid, reliable, and credible data to address  diverse questions about the performance of a teaching strategy. Educational evaluation is often defined by Scriven (1991) as “judging the worth or merit of an educational product or process” (p. 139). Guskey (2000) updated this definition stating that “educational evaluation is a systematic process used to determine the merit or worth of a educational program, curriculum, or strategy in a specific context.”

Despite its essential function, teaching strategy evaluation may well be the most widely misunderstood, avoided, and feared activity by practitioners (Shrock, & Geis, 1999).

The scope and purpose of this literature review is to provide an overview of the historical evolution of educational evaluation by describing seven significant time periods. This overview was intended to give students, educators, and practitioners a succinct synopsis of the field of program evaluation and its advancement from the late 1700’s through the 21st Century. The growth and evolution of this field establishes the need for such a study. Further, five program evaluation approaches that are currently used by practitioners were identified. It is the hope of the researcher that a better understanding of evaluation will reduce the fear and misunderstanding identified by Shrock & Geis (1999).

The researcher used systematic search methods to collect a broad swath of relevant literature. The review synthesized the literature on program evaluation as it relates to seven significant time periods in the evolution of program evaluation identified by Madaus, Stufflebeam, and Kellaghan (2000). Primary resources for this study were collected from refereed print-based journals, ERIC documents, and books with an academic focus. A variety of search terms were used, including educational evaluation, teacher strategy evaluation and history of evaluation.

Historical Evaluation of Teaching Strategy Evaluation

The historical development of educational evaluation is difficult, if not impossible, to describe due to its informal utilization by humans for thousands of years. Scriven (1996) noted that “evaluation is a very young discipline – although it is a very old practice” (p. 395). In the past 20 years, the field of educational evaluation has matured. According to Conner, Altman, and Jackson (1984), evaluation is an established field and is now in its late adolescent years and is currently making the transition to adulthood. Madaus et al., (2000), described seven development periods of educational program evaluation. First, the period prior to 1900, which the authors call Age of Reform; second, from 1900 until 1930, they call the Age of Efficiency; third, from 1930 to 1945, called the Tylerian Age; fourth, from 1946 to about 1957, called the Age of Innocence; fifth, from 1958 to 1972, the Age of Development; sixth, from 1973 to 1983, the Age of Professionalization; and seventh, from 1983 to 2000 the Age of Expansion and Integration.

Time Period 1: The Age of Reform (1792-1900’s)

The first documented formal use of evaluation took place in 1792 when William

Farish utilized the quantitative mark to assess students’ performance (Hoskins, 1968).

The quantitative mark permitted objective ranking of examinees and the averaging and aggregating of scores. Furthermore, the quantitative mark was historically important to the fruition of program evaluation as a discipline for two reasons: (a) it was the initial step in the development in psychometrics; and (b) its questions were designed to measure factual technical competence in subject areas that gradually replaced questions aimed at assessing rhetorical style (Madaus & O’Dyer, 1999). During this period in Great Britain, education was reformed through evaluation. For example, the Powis Commission recommended that students’ performance on reading, spelling, writing, and arithmetic would determine teachers’ salaries. It was not uncommon to have annual evaluations on pupil attainments (Madaus & Kellaghan, 1982).

The earliest method of formal evaluation in the United States occurred in 1815 when the Army developed a system of policies for “uniformity of manufacturers’ ordinance” (Smith, 1987, p.42). These policies set standardized production processes that fostered conformity of materials, production techniques, inspection, and product specification for all suppliers of arms to the military. The first formal education evaluation in the United States took place in Boston, Massachusetts in 1845. Printed tests of various subjects were used to assess student achievement in the Boston education system. Horace Mann, Secretary of the State Board of Education, wanted a comprehensive assessment of student achievement to assess the quality of a large school system. According to Stufflebeam, Madaus, & Kellaghan (2000), this event served to be an important moment in evaluation history because it began a long tradition of using pupil test scores as a principal source to evaluate school or instructional program effectiveness.

From 1887 to 1898, an educational reformer named Joseph Rice conducted a similar assessment by carrying out a comparative study on spelling instruction across a number of school districts. He was concerned about methods of teaching spelling, because U.S. students were not learning to spell. Rice was able to determine that there was no relationship between time devoted to spelling and competence. He reported his findings in The Forum in 1897, in an article entitled “The Futility of the Spelling Grind” (Colwell, 1998). Rice’s evaluation has been recognized as the first formal educational program evaluation in America (Stufflbeam et al., 2000).

Time Period 2: The Age of Efficiency and Testing (1900-1930)

Fredrick W. Taylor’s work on scientific management became influential to administrators in education (Biddle & Ellena, 1964). Taylor’s scientific management was based on observation, measurement, analysis, and most importantly, efficiency (Russell & Taylor, 1998). Objective-based tests were critical in determining quality of instruction. Tests were developed by departments set up to improve the efficiency of the educational district. According to Ballou (1916), tests developed for the Boston public schools were described as being objective referenced. The tests were used to make inferences about the effectiveness of the district. During this period, educators regarded measurement and evaluation as synonyms, with the latter thought of as summarizing student test performance and assigning grades (Worthen, Sanders, & Fitzpatrick, 1997).

Time Period 3: The Tylerian Age (1930-1945)

Ralph Tyler, considered the father of educational evaluation, made considerable contributions to evaluation. Tyler directed an Eight-Year Study (1932-1940) which assessed the outcomes of programs in 15 progressive high schools and 15 traditional high schools. Tyler found that educational objectives could be clarified by stating them in behavioral terms, and those objectives could serve as the basis for evaluating the effectiveness of a teaching strategy (Tyler, 1975). Tyler further wrote, “each teaching objective must be defined in terms which clarify the kind of behavior which the course should help to develop” (cited in Walbesser & Eisenberg, 1972). Stufflebeam et al. (2000) concluded that Tylerian evaluation involves internal comparisons of outcomes with objectives; it need not provide for costly and disruptive comparisons between experimental and control groups, as were utilized by comparative studies used by Rice. According to Worthen et al. (1997), Tyler’s work formed the basis of criterion-referenced testing.

Time Period 4: The Age of Innocence (1946-1957)

Starting in the mid 1940’s, American’s moved mentally beyond the war (World War II) and great depression. According to Madaus & Stufflebeam (1984), society experienced a period of great growth; there was an upgrading and expansion of educational offerings, personnel, and facilities. Because of this national optimism, little interest was given to accountability of national funds spent on education; hence the label of this evaluation time period, The Age of Innocence.

In the early 1950’s during The Age of Innocence, Tyler’s view of evaluation was rapidly adopted. Bloom, Engelhart, Furst, Hill, and Krathwohl (1956) gave objective based testing advancement when they published the Taxonomy of Educational Objectives. Bloom, Engelhart, Furst, Hill, and Krathwohl (1956) indicated that in the cognitive domain there were various types of learning outcomes. These objectives could be classified according to the type of learner behavior, and that there are hierarchical relationship among the various types of outcomes. Moreover, Bloom, Engelhart, Furst, Hill, and Krathwohl (1956) indicated that tests should be designed to measure each type of outcome (Reiser, 2001).

Time Period 5: The Age of Development (1958-1972)

In 1957, the Russian’s successful launch of Sputnik I sparked a national crisis. As a result, legislation was passed to improve instruction in areas that were considered crucial to the national defense and security. In 1958, Congress enacted the National Defense Education Act (NDEA) which poured millions of dollars into new curriculum development projects and provided for new educational programs in mathematics, sciences, and foreign languages (Stufflebeam, Madaus, & Kellaghan, 2000). Evaluations were funded to measure the success of the new curricula.

In 1960’s, another important issue in the development of educational evaluation was the emergence of criterion-based testing. Until that time, most tests, called “norm Online referenced tests”, were designed to discern between the performances of students. In contrast, a criterion-based test was intended to measure individual performance in terms of established criteria. It discerns how well an individual can perform a particular behavior irrespective of how well others perform (Reiser, 2001).

The passage of the Elementary and Secondary Education Act (ESEA) of 1965 was recognized as the birth of the contemporary educational program evaluation and included requirements for evaluation. According to Ferguson (2004), the ESEA was “ intended to supplement academic resources for low-income children who needed extra support in the early grades.”  Educators were required to evaluate their efforts. Senator Robert Kennedy sponsored the Act because he wanted to authenticate that federal money was not going to support schools’ exhausted practices, but rather would help disadvantaged students in new ways (Weiss, 1998).

Time Period 6: The Age of Professionalization (1973-1983)

During the 1970’s, evaluation emerged as a profession. A number of journals including Educational Evaluation and Policy Analysis, Studies in Educational Evaluation, CEDR Quarterly, Evaluation Review, New Directions for Program Evaluation, Evaluation and Program Planning, and Evaluation News were published (Stufflebeam et al., 2000). Further, universities began to recognize the importance of evaluation by offering courses in evaluation methodology. Among them were the University of Illinois, Stanford University, Boston College, UCLA, University of Minnesota, and Western Michigan University (Stufflebeam et al., 2000).

Time Period 7: The Age of Expansion and Integration (1983-Present)

In the early 1980’s, evaluation struggled under the Reagan administration. Cutbacks in funding for evaluation took place and emphasis on cost cutting arose. According to Weiss (1998), funding for new social initiatives were drastically cut. By the early 1990’s, evaluation had rebounded with the economy. The field expanded and became more integrated. Professional associations were developed along with evaluation standards. In addition, the Joint Committee on Standards for Educational Evaluation developed criteria for personnel evaluation.

Evaluation approaches for the 21st Century

Many evaluation approaches have emerged since the 1930’s and range from checklists of suggestions to comprehensive prescriptions. Worthen et al., (1997) classified the different evaluation approaches into the following five categories: (a) objectives-oriented, (b) management-oriented, (c) consumer-oriented, (d) expertise oriented, (e) adversary-oriented, and (f) participant-oriented evaluation approaches. In addition to these categories, specific evaluation approaches have emerged due to the attention given by researchers and practitioners. These specific evaluation approaches include: (a) CIPP (discussed in management-oriented), (b) CIRO, (c) Kirkpatrick’s

Evaluation Approach, and (d) Phillip’s Evaluation Approach.

Objectives-Oriented Approach

The objectives-oriented evaluation approach focuses on specifying the goals and objectives of a given program and determines the extent to which they have been attained. Ralph Tyler, who conceptualized the objectives-oriented approach to evaluation in 1932, is recognized as being the pioneer of this approach (Stufflebeam & Shinklefield, 1985). Expertise-Oriented Approach The expertise-oriented evaluation approach is the oldest and most widely used Educational evaluation approach to judge a program, activity, or institution (Worthen, Sanders, & Fitzpatrick, 1997). Evaluators utilizing this approach draw on a panel of experts to judge An educational evaluation  program and make recommendations based on their perceptions. The review process can be formal or informal. Worthen et al. (1997) defined a formal review system as, “one having (a) structure or organization established to conduct periodic reviews; (b) published standards; (c) a prespecified review schedule; (d) a combination of several experts to judge overall value; and (e) an impact depending on the outcome of the evaluation” (p. 121). Any other evaluation lacking one of the five components is considered to be an informal review system. In the eyes of critics, the overall limitation to the expertise-oriented evaluation approach is the central role of the expert judge. Critics suggest that the use of expert judges permits evaluators to make judgments that are personally biased, inherently conservative, potentially incestuous, and are not based upon program objectives (Worthen et al., 1997).

Adversary-Oriented Approach

The adversary-oriented evaluation approach utilizes a judicial process in examining a program. Worthen et al., (1997) identified the central focus of adversary-oriented evaluation is to obtain results through the examination of opposing views. The pros and cons of an issue are examined by two separate teams who then publicly debate to defend their positions and mutually agree on a common position. The evaluation process involves a hearing, prosecution, defense, jury, charges and rebuttals. According to Levine (1982), the adversarial approach operates with the assumption that the truth emerges from a hard, but fair, fight in which opposing sides present supporting evidence. One advantage to this approach is that it illuminates both positive and negative view points. Additionally, the approach is open to participation by stakeholders and decisions place greater assurance in the conclusion of the trial. This evaluation approach is not commonly adopted because of it’s determination of guilt. Worthen et al (1997) stated, “Evaluation should aspire to improve programs, not determine their guilt or innocence.” (p. 149)

Participant-Oriented Approach

The participant-oriented evaluation approach stresses firsthand experiences with program activities and emphasizes the importance of the participants in the process. As defined by Royse, Thyer, Padgett, and Logan (2006), participative evaluation “centers on enlisting the cooperation of the least powerful stakeholders in the evaluation from start to finish” (p. 93). Stakeholders define the evaluation approach and determine the evaluation parameters. The participant-oriented approach allows for the evaluator to engage with the stakeholder as a partner in solving the problems. Empowerment evaluation has been considered a sub classification within participative-oriented evaluation (Secret, Jordan, & Ford, 1999). Strober (2005) described empowerment evaluation as a type of formative evaluation in which participants in a project generate goals for a desired change, develop strategies to achieve them, and monitor their progress. Fetterman (2001) identified three steps as a part of empowerment evaluation: (a) developing a unifying purpose; (b) determining where the program stands, including strengths and weaknesses; and (c) planning for the future by establishing goals.

The participant-oriented evaluation (including empowerment) approach is not without disadvantages. According to Worthen et al., (1997), because of the reliance on human observation and individual perspective there is a tendency to minimize the importance of instrumentation and group data. Additionally, advocates have been criticized because of the subjectivity of the evaluation process and possibility of conflicts to arise among participants. Finally, participants could manipulate the situation or withdraw at crucial times causing the evaluation to be negated.

In light of the excitement over the past decade with educational evaluation approach, advantages and disadvantages with this ROI methodology have surfaced. Apparent advantages of this evaluation approach are twofold: (a) gain a better understanding of factors influencing training effectiveness, and (b) determine the monetary value of specific training initiatives. Despite the obvious advantages, the ROI methodology can become overly complex in determining a bottom line organizational value on training, as it is not an inexact science. Specifically, it can be difficult to isolate the effects of training. According to Shelton and Alliger (1993), one way to measure the effectiveness of training is to compare the results of a control group with the results of the experimental group or trainee group which can be burdensome for practitioners.

Current and Future Status of Program Evaluation

Worthen, Sanders, and Fitzpatrick (2004) identified twelve emerging trends that have and will have the greatest influence in shaping the current and future status of evaluation. Following are the twelve trends:

1. Increased priority and legitimacy of internal evaluation.
2. Expanded use of qualitative methods.
3. A strong shift toward combining quantitative and qualitative methods in each program evaluation rather than depending exclusively on either method.
4. Increased acceptance of and preference for multiple-method evaluations.
5. Introduction and development of theory-based educational evaluation.
6. Increased concern over ethical issues in conducting program evaluations.
7. Increased use of program evaluation within business, industry, foundations,
8. and other agencies in the private and nonprofit sector.
9. Increased use of evaluation to empower a program’s stakeholders.
10. Increased options that program evaluators should assume the role of advocates for the programs they evaluate.
11. Advances in technology available to evaluators, and communication and ethical issues such advances will raise.
12. Educators’ increased use of alternative assessment methods (as opposed to traditional testing) to assess students’ performance, and increased pressure on educational evaluators to use such methods in evaluating school programs.

Modifications in evaluation strategies to accommodate increasing trends of government decentralization and delegation of responsibilities to state/provinces and localities. (pp. 49-50)

Conclusion

Most experts would agree that educational program evaluation has an exciting and dynamic history. Due to its development over the past 200 years, educational program evaluation has matured significantly into an established field of study. The overarching trend of this field of

study has been the transition from more traditional summative evaluation approaches focusing on outcomes toward formative evaluation (Marshall, Crowe, Oades, Deane, & Kavanaugh, 2007). Through this trend, universities have accordingly developed courses and organizations utilize its approaches to understand their processes, procedures, and outcomes.

            Worthen and Sanders (1993) conducted a comparison of contemporary evaluation models.  Some of the models included in the sample were Stake (1967), Scriven (1967), Provus (1969), Alkin (1969), and Tyler (1942).  Among the evaluation models compared, the CIPP model was the only model Worthen and Sanders rated as being completely thorough. The elements of the CIPP model as depicted by Stuffelbeam (1971) are presented in Table 1.

Stufflebeam’s CIPP Model

A second framework used for this study was the Stufflebeam Context, Input, Process, and Product program evaluation model (CIPP) (Stufflebeam & Shinkfield, 2007).

Stufflebeam and Shinkfield (2007) defined the CIPP Model as “a comprehensive framework for conducting formative and summative evaluations of programs, projects, personnel, products, organizations, and evaluation systems” (p. 325). Product evaluations “identify and assess outcomes – intended and unintended, short term and long term” (p. 326) to determine “impact, effectiveness, sustainability, and transportability” (p.327).

As the definition stated, the CIPP Model uses formative and summative evaluations. The most appropriate type of product evaluation for this study was a formative one; to measure the implementations success or failure of graphic organizers in science education.  Based on an assessment of the desired outcomes within each group, the results of a formative product evaluation may offer guidance to the research strategy on the need for improvement and/or modifications (Stufflebeam & Shinkfield, 2007).

Teaching strategy evaluations can include various methods of study, both qualitative and quantitative (Stufflebeam & Shinkfield, 2007). It was recommended that a combination of techniques be used, specifically to be thorough in the evaluation process as well as allowing for cross-checking of the various findings (Stufflebeam & Shinkfield). This study used experimental and control groups as the means for evaluation.  This process is explained more fully in the following section.

Structure

            The CIPP evaluation model is an educational lens that aids in monitoring and adjusting the programs and objectives of the use of graphic organizers in science classes. CIPP is an acronym for four components: context, input, process, and product.  Daniel Stufflebeam (1969) described the major objectives of the four components as follows:

1. Context evaluation is to define the environment`s unmet needs and problems underlying those needs.
2. Input evaluation is to determine how to utilize resources to meet program goals and objectives.
3. Process evaluation is to detect or predict, during implementation stages, defects in the procedural design or its implementation.
4. Product evaluation is to relate outcomes to objectives and to context, input, and process, i.e., to measure and interpret outcomes.

Part 3 Proposed Methodology

This chapter details the methods used in this study. Each section specifically describes and addresses the participants, procedures, instrumentations, and statistical methods used to address each research question.

Description of the Study

The implementation of a new strategy in two groups of 8th-grade students was the foundation of this study. The sample size is 25 8th grade advanced science students who received the treatment and 25 8th grade science students who are not receiving the treatment.   The implementation will run for 6 weeks and all the students will be given a pre and post practice Science FCAT exam to gauge learning improvements.

The treatment is that the first 10 minutes of every class, the group receiving the treatment will be a given vocabulary word map and fill it out. The researcher will then go over the word with the test group as the students put the FCAT science vocabulary word in a sentence, draw a picture to help them visualize it and list synonyms or other related words to that specific FCAT science vocabulary word.

The non treated group will not receive any graphic organizers at all. Both groups will be given the practice science FCAT to show learning gains.

Participants

The participants in this study were 25 8th grade advanced science students who received the treatment and 25 8th grade science students who are not receiving the treatment.    At the time of the study, the demographic makeup of this 8th-grade students are 48.3% White, 39.8% African American, 5.3% Hispanic, 6.3% Asian, 0.3% Native American, and 0.1% Pacific

Islander. The demographic makeup of the school system of which this school is a part was 59.6% White, 32.0% African American, 3.6% Hispanic, 4.3% Asian, 0.4% Native American, and 0.1% Pacific Islander. Of the total school population, approximately 21.3% were Economically Disadvantaged, while 27.9% of the school district’s students were Economically Disadvantaged. Approximately 60% of the students in this school lived in homes with both parents. Less than 8% of the students in this school were special needs students, which included gifted students and students who receive speech services.

Procedures

A detailed proposal of this study will be submitted to the Internal Review Board of _____ University for approval. The school board and the school site will received letters that described this study in its entirety and obtained permission to administer the  Daniel Stuffelbeams CIPP Evaluation Model to measure the implementations success or failurein this particular school (see Appendix C).

All the 50 8th grade students will have both pretest and posttest scores. Then, the pretest scores will be compared to their posttest scores. All data from the 50 students who will be administered the Stuffelbeams CIPP Evaluation Model will be used to measure the implementations success or failure of graphic organizers in science classes.  To determine the below proficient and proficient or above proficient groupings of the 8th graders to be involved in this study, the researcher had access to the 2005-20006 FCAT science scores of the 8th  grade participants. The number of students in the below proficient category and above proficient students will be equaled in both the control and the experimental groups.

The experimental group and control group should be represented with  students’ proficiency. Once students will be categorized as members of the experimental group or the control group, their scores on the pretests and posttests of the FCAT will be compared, respectively.

In summary, the 8th grade science teacher will use graphic organizer (MAGO) in the areas of vocabulary development and comprehension for an eight-week period of time from January, 2009, to March, 2009.

Instrumentation

The Daniel Stuffelbeams CIPP Evaluation Model instrument will be adequately address the research questions presented in this study. This instrument was chosen because of its’ ability to measure the outcomes of the implementation success or failure of the MAGO System. This evaluation method will allow for a comprehensive evaluation of the implemented system.

1. Will the implementation of graphic organizers [MAGO]) (see Appendix C), in the areas of Science vocabulary development and comprehension, influence the gain scores of the 8th -grade students in the experimental group over the control group.
2. Will the implementation graphic organizers [MAGO]) (see Appendix C), in the areas of science vocabulary development and comprehension, influence the gain scores of the experimental 8th – grade students with below proficient scores in science and  compare with average or above average proficiency in science?

Evidence of construct validity will be provided in an examination of test performance at each the 8th grade level.

Data Analysis

The statistical methods to be used to answer each research question will be selected based on the number of independent and dependent variables.

1. Will the implementation of   graphic organizers [MAGO]) (see Appendix C), in the areas of Science vocabulary development and comprehension, increase the science scores of the 8th -grade students in the experimental group as compared with students in the control group.

The independent variables for this research question are the increased scores of the 8th-grade students in science. The graphic organizers make up the dependent variable for this study because they are being used in tandem during instruction on science vocabulary development and comprehension lessons.

A one-way analysis of variance will be used to compare pretests and posttests of the FCAT. However, using a two-way ANOVA (analysis of variance) will be appropriate in finding the variance between these samples of students’ FCAT Science scores to see whether there is any correlation between the two factors.

1. Will the implementation of graphic organizers [MAGO]) (see Appendix C), in the areas of Science vocabulary development and comprehension, influence the gain scores of the 8th-grade students with below proficient skills in science and their counterparts with average or above average proficiency in science?

The independent variables for this research question are the increase scores of the 8th-grade students in science. The graphic organizers make up the dependent variable for this study because they are being used together in the areas of vocabulary development and comprehension. A one-way analysis of variance was appropriate to use for the comparison of the randomly selected pretest and posttest scores of the 8th-grade students’ FCAT test. The pretest and posttest scores from 50 students will be randomly selected to use in the data analysis.

Once again, a two-way ANOVA was appropriate in finding the variance between these samples of students and their longevity at this school.

Limitations

This study was implemented in the 8th-grade where FCAT science are available

The MAGO System might have been successful due to the implementation of the strategy in a stage where the majority of the students could use reading as a means to gather science knowledge and their texts began to present new words and ideas beyond their own language and their knowledge of the world. Therefore, the use of the MAGO System might not be the most effective instructional strategy with students who are not at the developmental stage of reading for information gathering and understanding.

EPart 4 Expected Outcome

            Chapter 4 begins with a brief summary of the expected findings. Next, conclusions are presented that are supported by a theoretical frame of reference. The theoretical frame of reference focuses on two components: (a) the important of passive vs. active learning, and (b) the importance of organizing concepts. The next section addresses methodological issues and describes the generalizability of the study. The concluding sections offer recommendations for future research and implications of graphic organizers for science education.

Summary of Findings

            This study explored the following research question: Using graphic instruction with science students, is there a difference in the mean scores on a science FCAT test among: (a) students using graphic organizers  (GO), (b) those students not using graphic organizers (NGO).  In order to answer this question, the science FCAT mastery posttest scores will be adjusted; these groups mean scores will be compared; and an analysis of covariance will be conducted. The research results is hoped to indicate that there will be a significant difference in science achievement among these groups when differing graphic organizers were used.

            Results of a Scheffa’ post hoc procedure indicated that the GO group performed significantly better on a science FCAT mastery posttest than the NGO group. When comparing the GO group and the NGO group, there will be a significant difference in the groups’ achievement.

            The present study is consistent with findings of the limited research in the literature that compares student-use of graphic organizers with nonuse (Moore & Readence, 1980, 1984). However Moore and Readence’s (1980, 1984) findings that student use of graphic organizers are more effective than nonuse of graphic organizers must be viewed with caution because of the variation of treatment outlined in the literature. These variations limit the generalizability of the results. In Moore and Readence’s (1984) meta-analysis comparing student of constructed graphic organizers, they found a range of treatments: (a) learners grouped words written on index cards; (b) learners filled words into prepared tree diagrams that included superordinate terms; (c) learners read passage three times and sought different types of information with each reading ; and (d) learners viewed teacher-constructed graphic organizers after reading and were encourage to suggest changes (p. 15) . The present study used a treatment consistent with the studies Barron and Schwartz (1984) and Barron and Stone (1974). The treatment required students themselves to group words on index cards into diagrams. Both of Barron’s studies also found graphic oganizers to be a more beneficial method of instruction graphic organizers.

Theoretical Frame of Reference

                        Based on the studies in the literature, a theoretical frame of reference was presented. The results of this present study can be integrated into this frame of reference.

Active vs. Passive Learning

                        Student –constructed graphic postorganizer, coupled with traditional instruction, actively involved the students in their own learning. This method of instruction proved  more effective in assisting the students in learning science information than the two more passive methods of instruction, teacher-constructed graphic postorganizers coupled with traditional instruction, and traditional instruction only. In this study, teacher-constructed graphic instruction relied on the teacher to organize the information both passive learning methods. Several researchers who have studied the use of student-construction graphic organizers have attributed higher achievement scores to the active involvement of the participants. They concluded that active involvement of the assisted the students in processing the information at deeper, more meaningful level (Barron & Stone, 1974; Berkowizt, 1986; Holley & Dansereau, 1984).

                        Barron and Stone (1974) hypothesized that graphic organizers triggered a conscious effort by the learner to incorporate the new information in a meaningful way. They focused on the conscious or active effort of the learner. Other researchers also discussed the importance of the learner’s active involvement. The advantages of engaging student participation in presenting information from the text were stressed by Alvermann and Boothby (1986). Additionally. Holley and Dansereu (1984) postulated that creating visuals, a form of elaboration in which the learner is engaged in deeper processing of the text, leads to better memory for the information. In the present study, students created a visual arrangement of the science information. This may have accounted for the SC group’s performance.

                        In contrast to active learning, researchers have the nonuse of graphic organizers as a more passive and, therefore, less effective method of instruction . Simon, Griffin, and Kameenui (1988) found that graphics organizers were no more effective than traditional instructional in facilitating recall in science material. The authors hypothesized that graphic organizers (a) may have mistakenly assumed background knowledge of facts with an “advanced and premature attempt to establish relational knowledge” (p.20). The researchers also concluded that because the students were not required to act on the textual information, they may not have deeply processed the information. These authors acknowledge this factor as a limitation to teacher-constructed graphic organizers. Furthermore,Dunston (1992) pointed out that: “teacher reseacher-constructed orgnizers . are designed to match the schema of the teacher/reasearcher and not the student” (p. 63) The results of the present study seemed to substantiate the importance of students’ active participation .

            Other studies, however have demonstrated that teacher-demonstrated that graphic organizers are also effective learning tool. Two studies by Bergrud, Lovvit , and Horton (1988,1990) favored both the use of student-directed and teacher-directed and graphics organizers for student with learning disabilities. Similarly,a study by Darch and carnine (1986) reported favorable results in science and social studies achievement when using teacher-constructed visual displays for students with learning disabilities as opposed to the use of text only. The authors concluded that teacher-constructed visual displays allow teachers to focus on the most critical concepts of the unit, eliminating irrelivant details. Although the participants were directly taught information graphicaly organized by teacher,higher performance seemed to occur because of the guidance the participants received in focusing on important information. Active participation of the students in arranging the information themselves (students-constructed graphis organizers) did no seem to be a key factor in acquisition of knowledge.

            In summary, the selected studies presented conflicting results as to effectiveness of graphic organizers and teacher-constructed graphic organizers. In the present study, graphic postorganizers will be found to be of value and may have provided a means for the students to be actively involved in learning and doing science.

The Importance of Organizing Concepts

            The organizational structure of information could explain the higher achievement of the GO group. Ausubel (1963, 1968) stressed the importance of organizing information, particularly in a hierarchical manner. Other researchers have discussed organization as necessary aspect of the retrival system (Goetz, 1984) and as a more efficient use of knowledge ( Eylon & Linn, 1988). When comparing hierarchical  and linear organization, hierarchical structures that organize the information and levels of details seemed to be more effective in learning (Eylon &  Reif, 1984). Holley and Dansereau (1984) explained that information acquisition does not necessarily occur in a linear manner; therefore, hierarchical in structuring maybe better much for the encoding process.

            In this study students in the GO group arrange the concept in a hierarchical, nonlinear manner. In order to accomplish this task, the students needed to know how to select the superordinate ideas and show their relationship to subordinate ideas. Creating hierarchical graphic organizers provided visual cue when the information was encoded. Tulving and Thomson (1973) theorized that retrieval process is dependent on how information is stored. This principle is referred to as the “ encoding specificity principle”. Dunston (1992) alluded to this principle when explaining how the cues used in initial learning situation may assist with retrieval and memory. The “cues that were presented in the text and/or learning situation during encoding process can function as cues for retrieval of information later” (p. 62). The graphic organizer illustrated the relationship among the concepts, and this may have assisted in retrieval of knowledge.

            This explanation, however, does not fully account for the performance of the GO group. It is possible the same encoding specificity principle occurred when teacher-constructed graphic organizer were presented to the students. This teacher-constructed graphic organizers were hierarchically arranged and could have acted as cues in the encoding process.

            In this present study, the benefits of arranging the concepts in a hierarchical manner could have assisted students in both the GO group and the NGO group. However, the result indicated that the GO group may have gained more from hierarchically arranging in the concepts. These results must be viewed with caution because of the stated limitation of the study.

Methodological Issues

            The methodological issues of the study focus on (a) sampling procedures, (b) the reliability of the FCAT pre and post test, and (c) the teaching procedures. As stated, there is statistically significant difference in the demographic characteristics of the two groups when considering the racial composition and socio economic level of the groups. This is a concern because the difference in performance among the two groups could have been due, in part, to the difference demographics rather than the treatments. If this study will be replicated, it would be necessary to assure group equivalency. Randomly selecting groups that are not significantly different in demographic make-up would assist with establishing group equivalency.

            The procedure for selecting the sample contributed to the differences in a group demographics. The researcher selected the schools based on the highest  available number of 8th-grade science students within the school. If this study were to be replicated, random procedures for sample selection should be used as much as possible (Kerlinger, 1986).

            Another method of demonstrating group equivalency would be to use a valid and reliable instrument to measure the achievement level of the groups before treatment. Although all of the participants in this study completed a science mastery pretest, the reliability of the pretest was in the moderate positive range (coefficient =.54). Therefore, this measure of equivalency was tenuous. The internal consistency of the science master post test was also low; the coefficient was .36, a score that fell in the low positive correlation range.

            This low reliability of both the FCAT pretest and posttest may not consistently reflect the participants’ science achievement. In order to determine how well a student’s performance on one measure, it is necessary to examine how consistent the performance is across each item on the test. Low internal consistency for a test means that there is a little congruency in performance across the individual items and, therefore, a threat to reliability. Ideally, test items should indicate performance that could be duplicated if another similar test were conducted. Therefore, it is possible that the scores on the pretest and posttest may not offer a reliable view of the participant’s achievement. For example, each groups’ performance on the posttest may be quite different if a second and similar test was given.

            If this study were to be replicated, a pilot test needs to be conducted with the science FCAT mastery pre and posttests. An item analysis would determine which questions assist in discriminating between those students who know the information and those who do not. Both tests need to be revised based on the conclusions of the item analyses. Also, additional test items could be added to added to both tests. An increased number of items should improve the internal consistency and, therefore, the reliability of the tests (Pedhazur & Schmelkin, (1991).

            There was also a concern about the length of the training for students who learned to construct their own graphic organizers (GO group). Since two of the students who were initially part of the training did not meet mastery on the performance-based assessment, they were dropped from the study. The students who successfully constructed their own graphic organizers needed to be able to categories various concepts. They needed to perceive relationships between superordinate and subordinate concepts, a skill that is difficult for many students with learning disabilities (Deshler,Ellis &Lenz,1996).With more training the two students who were dropped from the study may have learned this higher order thinking skill and reached the criterion to participate in the remainder of the study.

            Another issue associated with training involved the training of the instructor in this study. The researcher and the instructors each taught at different schools during the same time frame . If this study were to be replicated, the experimenter effect would be diminished if the researcher could observe the instructors teach a lesson to issue adherence to the specified teaching instructions.

Genelizalibility of the Study

           The small sample size of 25 per group limits the external validity of the study. For example, if only 11 students, as in the GO group, perform at a certain level under certain conditions, generalizability of the results is limited. However many of the studies in literature involving science students have small sample sizes. In five of the studies, the average sample size was 32.6(Bergerud,Lovitt, & Horton, Lovitt, & Bergerud, 1990)).Furthermore, these small numbers are typical of mainstream classes where students receive their science education (Horton et al.)Additionally, the study was limited to only 8th-grade science students. Therefore, the results can be generalized to this narrow population.

             Finally, the study was conducted during a six-week time period. This short time span was used to insure control in the study. If there is more control over outside influences, changes in the posttest scores are more likely to due to the varying methods of instructions. A longer study would, however, add additional data and enhance the generalizability of the study.  For example, the researcher could have repeated the method of instruction with several chapters from the science textbook and examined the pattern of results.  The above limitation should be considered when generalizing the results.

Recommendations for Future Research

            The following issues should be addressed in future research on graphic postorganizers: performance of specific subgroups of students and training procedures.  If this study were to be replicated, there are additional concerns that should be addressed.

References

1. Adams, M.J. & Collins, A. (1995).  A schema-theoretic view of science and reading.  Newark, DE: International Reading Association.
2. Almasi, J. (2003). Teaching strategic processes in science. New York, NY: The
3. Guilford Press.
4. Anderson, R.C. (1994).  Some reflections on the acquisition of knowledge.  Educational
5.            Researcher, 13 (8), 5-10.
6. Ausubel,D.P. (1993).  The use of advance organizers in the learning and retention of meaningful
7.            verbal material.  Journal of Educational Psychology, 51, 267-272.
8. Ausubel, D.P. (1998).  Educational Psychology.  New York:  Holt, Rinehart, and Winston.
9. Bartlett, R.F. (1992).  The effect of student constructed graphic post organizers upon learning
10.            vocabulary relationships.  Clemson, SC:  The National Reading Conference.
11. Barron, R.F. (1996).  A systematic research procedure, organizers, and overviews: An historical
12.   perspective. Paper presented at the Annual Meeting of the National Reading Conference, San Diego, CA (ERIC Document Reproduction Serviced No. ED 198 508). Barron & Stone, 1994
13. Barron, R.F. & and Schwartz, R.M. (1994).  Graphic organizers: A spatial strategy.  In C.D.
14.   Holley and D.F. Dansereau (Eds.), Spatial learning strategies.  Orlando, F.L: Academic Press, pp.275-289.
15. Bernard, R.M. (2000).  Effects of processing instructions on the usefulness of graphic organizer
16.            and structural cueing in text.  Instructional Science, 19.  207-217.
17. Blough, G.O. & Schwartz, J. (1990).  Secondary school science and how to teach it (8th ed.).
18.            Fort Worth: Holt, Rinehart & Winston.
19. Bower, J. (2002). Tools for Teaching Scientific Literacy. Portland, ME: Stenhouse
20. Publishers.
21. Bransford, J.D.,  & Vye, N.J. (2006).  A perspective on cognitive research and its implications
22. for instruction.  In L.B. Resnick & L.E. Klofer (Eds.), Toward the thinking curriculum: Current cognitive research.  Reston, VA: Association for Supervision and Curriculum Development.
23. Cawley, J.F. (1994). Effects of a teacher mediated spatial strategy instructional routine on
24. learning science concepts by adolescents.  Unpublished manuscript.
25. Crank, J.N., & and Bulgren, J.A. (2003).  Visual depictions as information organizers for
26. enhancing achievement.  Learning Research and Practice, 8 (3), 140-147.
27. Driver, R., & Oldham, V. (2006).  A constructivist approach to curriculum development in
28. science. Studies in Science Education, 13,105-122.
29. Griffin, C.C.  & Tulbert, B.L. (1995).  The effect of graphic organizers on students`
30. comprehension and recall of expository text: A review of the research and implications
31. for practice. Reading and Writing Quarterly, 11, 73-89.
32. Hawks, P.P. (1986).  Using graphic organizers to increase achievement in middle school life
33. science.  Science Education, 70, 81-87.
34. Holley, C.D.,  & Dansereau, D.F. (1994).  The development of spatial learning strategies.
35. Spatial learning strategies: Techniques applications, and related issues.  Orlando: Academic Press.
36. Horton,S.V., & Lovitt, T.C., & Bergerud, D. (2000).  The effectiveness of graphic organizers for
37. three classification of secondary students in content area classes.  Journal of Science
38. Education, 23 (1), 12-22.
39. Hudson, P. Lignugaris-Kraft, B., & Miller, T. (1993).  Using content enhancements to improve
40. the performance of adolescents in science classes.  Learning Science Research &
41. Practice, 8(2), 106-126.
42. Jones, B.F., Pierce, J., & Hunter, B.  (1999). Teaching students to construct graphic
43. representation.  Educational Leadership, 20-25.
44. Klopfer, L (1991).  Evaluation of learning in science.  Handbook of formative and summative
45. evaluation.  New York: McGraw-Hill.
46. Lovitt, T.C.,  & Horton, S.V.  (1994).  Strategies for adapting science textbooks for youth with
47. learning disabilities.  Remedial and Spatial Education, 15(2), 105-116.
48. Moore, D.W.,  & Readence, J.E.  (1994).  A quantitative and qualitative review of graphic
49. organizer research.  Journal of Educational Research, 78, 11-17.
50. Rumelhart, D.E. (2000).  Schemata: The building blocks of cognition.  Theoretical issues in
51. reading comprehension, Hillsdale, NJ: Erlbaum.
52. Rumelhart,D.E.,  & Ortony, A. (1997).  The representation of knowledge in memory.  Schooling
53. and the acquisition of knowledge.  Hillside, NJ: Erlbaum.
54. Rutherford, F.J., & Ahlgren, A. (1992).  Science for all Americans.  New York: Oxford
55. University Press.
56. Scruggs, T.E., Mastropieri, M.A., Bakken, J.P., & Brigham, F.J. (2003).  Science versus doing:
57. The relative effects of textbook-based and inquiry-oriented approaches to science learning in science education classrooms.  The Journal of Science Education, 27, 1-15.
58. Shymansky, J.A. , Kyle, W.C., & Alport , J.M. (1993). The effects of new science curricula on
59. students performance. Journal of Research in Science Teaching, 20, 387-404.
60. Shymansky , J.A., Woodworth , G.,Norman, O , Dunkhase, J., Mattthews, C., & Liu, C. (1989).
61. A study of changes in middle school teachers’ understanding of selected ideas in science as a function of an in-service program focusing on student preconceptions. Journal of Research in Science Teaching, 30, 737-755.
62. Starr, M.L., & Krajcik, J.S. (2000).  Concept maps as a heuristic for science curriculum
63.            development: Toward improvement in process and product.  Journal of Research in
64.            Science Teaching, 27, 987-1000.
65. Tyson, & Woodward. (1989).  Why students aren`t learning very much from textbooks.
66.            Educational Leadership, 14-17.
67. U.S. Department of Education. (1991). America 2000: An education strategy.  Washington, DC: Author.
68. Wood, T.L., & Wood, W.L. (1998).  Assessing potential difficulties in comprehending secondary science textbooks.  Science Education, 72, 561-574.