Using a Mixed Methods Study to Improve K-12 Alternative Education Instructional Design

Gabrielle L. McBath
Ph.D. in Educational Leadership, Northcentral University, USA.

Abstract

The purpose of this Mixed Methods study is to conduct a full quantitative and qualitative research method on a hypothetical K-12 Alternative Education program, wherein ten students have had poor grades and are at-risk of withdrawing. Within a six-part action plan on the quantitative side, students were tested twice, before, and after a two paired sample t-test in their 9th grade year. On the qualitative side, students were evaluated on their Electronic Portfolios (e-Portfolios). Results were triangulated by Concurrent Design. Despite the results of the quantitative research, the collaborative efforts of the students' e-Portfolios showed that students worked-well collaboratively and the Action Research Framework encompassed well both the t-test and the e-Portfolios. Future research is still needed on the same group of students in other subject areas of the course work.

Keywords :

Introduction

In high schools across the nation, Alternative Education students should not be defined hastily as “disaffected” due to their underachieving status when numerous factors represent their social and academic alienation (Haughey, 2009). Alternative Education may be defined as a vast category of students who range from being homeschooled, disruptive, advanced-placement, or a part of a Charter School. However, for the intent of this discussion, Alternative Education contained students who were at-risk of dropping-out, who were behaviorally disruptive, and did not have the same educational goals as those in a traditional classroom setting (Foley and Pang, 2006; Van Acker, 2007).

A very high drop-out rate occurs within the Alternative Education program. This is based on unrealistic district assessments and goals, lack of teacher-training within any special programs, inadequacy of students' basic learning skills, and inaccurate student-transitioning expectations from Middle School to High School. Truancy and poor grades in the elementary and middle school years predicted often a future high school student's drop-out status. Unable to fit into the status quo of employment, these students may no longer earn a living wage in the United States without a high school diploma (Haughey, 2009; Siegrist et al., 2010). Therefore, a hypothetical Mixed Method study was comprised to assess the instructional design in an Alternative Education program. A Mixed Methods design combines a full quantitative and qualitative design. An action research project (Appendix I) with a small sample group of ten Alternative Education students was constructed. Data collection and analysis are aligned specifically to the Mixed Methods' concurrent triangulation design (Appendix II).

On the quantitative side of this study, students were tested twice (pre and post-test of a two-paired sample t-test) in their 9th grade year. This Action Research Project was conducted on a single-school level. On the qualitative side, students maintained, presented, and were evaluated th on their 9 grade Electronic Portfolios (e-Portfolios), inclusive of six projects, and three peer-evaluations. After these two coexistent research segments were completed, results were triangulated (against each other) for three purposes: (a) to validate the research's integrity by Concurrent Triangulation design, (b) to take action implementing sound educational outcomes, and (c) to delineate future focus areas of research, i.e., to implement a better writing program at this school.

1. Six-part Action Plan

1.1 Focus

Establishing a focus for this research began with substantiating goals on professional, personal, and political levels. On a professional level, the author wanted students' scores to improve, while understanding the reasons behind their low-achievement status. The Alternative Education program was evaluated additionally. On the personal level, a better relationship with staff members, teachers, and students did not occur if she had not understood the causation of the students' underachievement. Communication was also important in diagnosing the variables used in this pending research. Understanding another's projected-identity, i.e., how they acted within a set point-of-view, provided an ample barometer for understanding the study's focus (Searle, 2007). On a political level, this Alternative Education program was important enough to justify its funding to various community, state, and federal sources (Gall et al., 2007).

The blanket focus for this research determined if improving educational outcomes for these ten students paralleled curricular reform. The literature highlighted that curricula within the Alternative Education program varied as well as the institution's responsibility toward it, as the program deviated from the traditional setting with no clear standards for alternative assessment. In order for students to be successful, there must be a tight alignment between the district's expectations, teacher preparation, and curricular subject matter, while shadowing the state standards of a traditional program (Suell and Piotrowski, 2007).

1.2 Quantitative Data Collection and Analysis

Under the broad focus, the quantitative hypothesis stated that students' scores remained below passing (i.e., 75%) on both exams because Alternative Education curricula were disjoined to the standardized test objectives.

Participants: ten (n=10) Alternative Education students were sampled throughout their 9th grade year. The gender of the participants was not applicable to this study. All students had successfully completed the Middle School Alterative Education program in the district, and were either 14 or 15 years of age.

1.2.1 Procedure

In this longitudinal study, students' scores were measured twice, once at the beginning of the year (labeled pre-test), and once at the end (labeled post-test). The first measurement was taken in October, in order to establish the inception variable. The second measurement, taken in May, aided the research in order to examine the change score linked to their curricula, and potentially evaluate the subject areas of their purported underachievement.

1.2.2 Measurement

A paired-sample t-test was selected for purposes of this study. This model matched the same sample group in two different scenarios, making it easier to detect the true differences between exam scores. The variable of “exam score” remained consistent throughout the sample. The percentage of the difference between exams was an integral criterion of analysis; however, the two means found in the exams will also be pertinent (and not their averaged difference). The confidence interval was set to 95% (8.94 in the pre-test and 8.9 in the post-test), as it was the most commonly reported. The Degree of Freedom (ta) was 9 (n-1); the hypothesis was Ha : Do (i.e., the difference is not zero) (Norusis, 2008; Trochim and Donnelly, 2008).

1.2.3 Results

Appendices III and IV were created signifying the histogram for this study to delineate the set of test scores for all ten students and the combined descriptive statistics. The mean was 71.7 (3.3% below the passing rate of 75%) (i.e., pre-test: 71.5 and post-test: 71.9). The median was 69.5; mode signified 62 with a range of papers at 41%. The variance was 146.64 and standard deviation was 12.10. The Pearson correlation was 0.91; the two-tailed P-value for this t-test is 0.82 (t= -0.24).

1.2.4 Discussion

A preceding hypothesis signified that students did not achieve good test results within the school year based on incongruous curricular topics. Overall, at the end of the 9th grade year, students' mean scores were raised slightly by 0.4% (71.5 to 71.9). However, this fell below the passing rate, while affecting potential funding for the Alternative Education program. The repetitive mode was 62% or 13 points below passing. Also, the range of highest to lowest test scores averaged 41 points. This discrepancy was very wide not accounting for various deterrents such as teaching styles or students' socio-economic status. The Pearson coefficient showed that a 0.91 connection between tests was a very strong positive correlation. Because it exceeded 0.05, the null hypothesis is not rejected.

1.3 The Importance of Building an e-Portfolio

E-Portfolios, or e-folios, were a vital contribution to K-12 academics. They were diverse, malleable, and tailored to the needs of the students and district's goals. With overwhelming options for their contents, promoting behavioral or constructive learning styles, as well as diverse options for its evaluation, e-Portfolios provided a diverse element of assessing these ten Alternative Education students (Buzzetto-More, 2010). E-Portfolios were available in different platforms dependent upon the school's budget. Software ranged from the user-friendly Microsoft PowerPoint found within the Microsoft Office suite to additionallypurchased contracts with companies like Blackboard.com, LiveText.com, or PebblePad.co.uk (Buzzetto-More, 2010; Moores and Parks, 2010; Ntuli et al., 2009).

E-Portfolios were a pertinent beginning step for students to showcase their 12th grade capstone project, gain university admission, or begin research projects in their potential area of future employment. For purposes of this study, instructors briefed students on using the e-Portfolio software, as it transitioned easily with them from the Middle School program.

1.4 Qualitative Data Collection and Analysis

During the students' 9th grade year, they were responsible for the creation and posting of different projects and three peer-evaluations of classmates with applicable rubrics. Appendices V and VI were constructed in order to define the e-Portfolio and rubric criteria. In addition to their other class projects, students completed these submissions to the e-Portfolio. In the Business class, students made a contribution to the departmental web page, uploading three pictures, and interesting facts about a profession of their choice. This assessment was worth 10% of the entire project grade. In the area of Math and Science, students wrote two extended labs, while making one presentation at either the Science Fair or the Mathlete Contest (15%). Pictures or written samples were uploaded to the server. In the Industrial Arts category, students worked with two peers, while completing their semester welding assignments (topics included arc welding, brazing, or mig) (15%).

The English requirement specified comprising four writing samples. They were one meter-defined poem, one freeverse poem, a story with an elaborate character development - protagonist and antagonist, and one employer interview with help from either the Guidance Department or their job coaches (15%). For Social Studies, students designed a video story (lasting five minutes) about an historical figure, including acting and dialog (15%). The next assignment was one submission from an elective area of the student's choice (Art, Theater, Music, or Foreign Language). Finally, students completed one peerevaluation rubric during the Social Issues-Class Debate, and explained their findings (15%).

Data analysis of this e-Portfolio project included an encoding process. However, the first step of the analysis was to break-down students' grades per subject area within the e-Portfolio. Other notes and applicable data were a part of this organizational process. After reading through all the data, recurring ideas and tones were separated from the students' project themes before encoding. Encoding contained larger topical ideas to labeling simple and complex patterns. A simple coding table (Appendix VII), delineated each student's favorite subject areas of the e- Portfolio project. Then patterns must be narrowed-down, reassessed, and re-coded, if needed. Finally, the themes were interlinked and the meanings were interpreted (Creswell, 2009; Patton, 2002; Trochim and Donnelly, 2008).

1.5 Comparing Results

After the quantitative and qualitative projects were completed, results were compared and contrasted to each other. The quantitative results showed only a slight increase between tests, but were still below passing. The qualitative results exemplified students excelling in the subject areas of business, welding, and social studies. The new hypothesis posited that the mean difference of students' quantitative test scores reflected the subject matter that was on their e-Portfolio tasks. A qualitative measure of a coding scale (Appendix VIII) verify these results. The pre and post-test scores were reviewed denoting the erroneous student answers. From there, this researcher assessed if the e-Portfolio subject area matched the topics of the correct answers on the tests (Appendix IX).

1.6 Implement Actions

The basis of Action Research was implementation before reflection and modification (then beginning a new focus/ hypothesis). After establishing the subject areas that needed improvement, there was a major decline in the writing portions of the e-Portfolio and quantitative testing. More-concise academic writing was imperative to teach at the high school level within four supporting benchmarks, (a) it aided employment, (b) it contributed well to social success, (c) it accommodated the vast subjects within the curricular topics, and (d) the social community was dependent upon the students' ability to write logically and coherently (Kiuhara et al., 2009). The following strategies were implemented for mandated writing workshops for students. These writing labs were taught by teachers who constructed both a raw/ naturalistic writing sense (i.e., stream-of-consciousness writing) and a structured writing course (Shields, 2007; Tatum and Gue, 2010).

In the former example, students were offered a variety of open-ended questions encompassing two inquiries, (a) the type of writing students conducted, and (b) their thoughtprocess before, during, and post-revision. Some literature suggested that using raw/ naturalistic formatting aided “traditional” writing structure. Students however did not see this immediate correlation, while nullifying that projects of web-design, science/ math lab reports, texting, or e-mail constituted as writing (Raley, 2010).

The latter paradigm stressed formal writing structure, while providing students with an opportunity to aspire to requirements of local-school and state standards. “Poor writing, especially among low-performing students, [was] to give an open-ended assignment with little structure” (Shields, 2007, p. 56). Perhaps instructors assessed the teaching of writing on the complexity of the task, as opposed to single-approach assignments. Examples included: complex sentence structure (avoiding fragments), pronoun usage (agreeing in number), word lists on Tier II, e.g., general words used by a more mature audience, but not such an advanced level that these words were confined to specific domains (e.g., dénouement in English literature), and understanding sentence structure (subject, predicate, direct/ indirect objects, etc.) (Wolsey, 2010).

In the Alternative Education environment, revision and significant changes within students' writings showed little preparation and organization of content. Students worked harder revising their writing pieces only if a reward (e.g., a better grade) was promised. Next, students worked on peer-collaboration. The comfort of working with likeminded authors and friends helped ease the angst of the writing production, while enabling understanding various points-of-view. Finally, conferencing about their completed drafts with peers and instructors allowed for a multi-viewed evaluation, as well as permitting the writer to clarify any structure or content (Raley, 2010).

1.7 Reflection and Modification

Much like the t-test and the e-Portfolio studies, the writing workshop resulted in new areas of writing errors also in need of evaluation. Specifically, even when students did well on the revision process, they were sub-par on the preliminary, organizational process of writing. From there, a future focus (hypothesis) was conducted isolating their organizational problems, offering remedies and pedagogical strategies, and implementing those strategies in context for a betterstructured writing workshop.

1.8 Validity (Legitimation) in this Mixed Methods Study

After the six-phased research action plan concluded, establishing a study's internal and external validity, by emphasizing triangulation was compulsory, especially since there were close to 80 combined “threats.” These were described in the Quantitative Legitimation Model and Qualitative Legitimation Model (Onwuegbuzie and Johnson, 2008). However, within a Mixed Methods study, errors of representation, integration, and legitimation (validity) surpassed triangulation. Errors of representation stemmed from inaccurate portrayal of lived experiences found within words and numbers. Problems of integration derived from conducting concurrent strengths from juxtaposed quantitative and qualitative research designs, and the subsequent problems of their overlapping legitimation. This included repetitive errors that each side “brought” to the results, while showing undependable deductions from said research (Gay et al., 2008; Onwuegbuzie and Johnson, 2008). Despite nine types of legitimation in Mixed Methods research, three were mostapplicable to this study of the Alternative Education students. The first was sample integration, i.e., evaluating the relationship between the quantitative and qualitative samples if they gained high-quality meta-inferences. Meta-inferences were named as adaptive deductions within both sides of the research. The second was weakness minimization, i.e., one research design over-compensated for the “weakness” of the other. The third was multiple validities, asking if each side of the study used these metainferences by various deductions, and diverse forms of evaluations in assessment (Onwuegbuzie and Johnson, 2008).

Sample integration legitimation was analyzed by having a smaller sample size (n=10). Additionally, the same sample was available to complete the pre and post-tests. Results were labeled non-dominance (QUAN-qual or QUAL-quan, respectively) as a concurrent (not sequential) design was run. Additionally, the range of conditions for the sample selection, and why the research designs were used, provided high meta-inference quality (Onwuegbuzie and Johnson, 2008).

Errors of weakness minimization legitimation were avoided by triangulating the results of the quantitative and qualitative designs. The last procedure of using a coding scale to match parallel themes proved that the unilateral writing errors disseminated across curricula. The quantitative and qualitative designs needed to complement each other in order to implement curricular change within this Alternative Education program (Onwuegbuzie and Johnson, 2008).

Finally, errors in multiple validities legitimation emphasized the component found within each research design. The researcher's questions were based upon how the holistic meta-inferences were greater than the aggregates of each side's results. These various writing workshops would be implemented within the succeeding years, while emphasizing reform of recurring writing errors. This must connect the students' needs to the those of the Alternative Education program (Onwuegbuzie and Johnson, 2008).

Conclusion

Researching a Mixed Method study for this Alternative Education program was complex and time consuming, ensuring that a full quantitative design, then a qualitative design, preceded the results' triangulation. Placed within the larger framework of an Action Research design, this study benefitted a group of formerly underachieving students. The research design, cyclical in nature, stressed stating a focus, implemented two diverse sides of data, assessed the results, triangulated each side of datum, implemented writing reform, reassessed that implementation, before beginning a subsequent focus (hypothesis).

However, there were many benefits to a Mixed Method study. Tailored-methods fit the focus or hypothesis. There were diverse assessments to validate and triangulate the findings, and conjoined effectively two separate studies under one study's unified function. Mixed Methods permitted scaffolding results in either a quantitative or qualitative fashion. By far, there were more possibilities for data collection and analysis than a one-sided study (Gall et al., 2007; Onwuegbuzie and Johnson, 2008). Despite the high cost and need of many resources, Mixed Methods research was ideal providing districts accommodated the research on a continual and longitudinal basis (Patton, 2002).

Defining and correcting legitimation within this study commissioned distinguishable criteria for research framework. The t-test exemplified the pre and post-scores of one sample having two distributions of mean and variation. The t-test was one quantitative measure that assessed statistical differences of mean within the spread of standard deviation (Trochim and Donnelly, 2008). EPortfolios substantiated students' growth throughout the year, as well as promoted collaborative and individual efforts. Ideally, the benefits of e-Portfolios minimized overwhelming, traditional paperwork (Ntuli et al., 2009). The striving for such contemporary, educational advancement, and student improvement within an Alternative Education framework was dependent upon unified Mixed Methods research. Since future research was needed on these students' performances in different curricular areas throughout their high school duration, contrarily, no sole assessment ever worked-well in isolation.

Appendices

Appendix I

Figure A1. Design Research Action Plan (Gall et al., 2007)

Appendix II

Figure A2. Create Mixed Methods Concurrent Triangulation (Creswell et al., 2008)

Appendix III

Figure A3. Histogram Delineating Alternative Education Students' Scores

Appendix IV

Table A1. Quantitative Results

Appendix V

Table A2. 9th Grade e-Portfolio Criteria

Appendix VI

Table A3. Rubric Showing Social Issues, Class Debate

Appendix VII

Table A4. Qualitative Data Encoding into Themes (Trochim and Donnelly, 2008)

Appendix VIII

Table A5. Quantitative Data Encoding of Qualitative Themes (Trochim and Donnelly, 2008)

Appendix IX

Table A6. Analyzing Quantitative and Qualitative Results (Trochim and Donnelly, 2008)

References

[1]. Buzzetto-More, N. (2010). Assessing the efficacy and effectiveness of an E-Portfolio used for summative assessment (M. Stansfield, Ed.). Interdisciplinary Journal of E-Learning and Learning Objects, 6, 61-85.
[2]. Creswell, J. W. (2009). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (3rd Ed.). Thousand Oaks, CA: SAGE Publications.
[3]. Creswell, J. W., Plano Clark, V. L., Gutmann, M. L., & Hanson, W. E. (2008). Advanced mixed methods research designs. In Clark, V. L. P., & Creswell, J. W. (Eds.), The Mixed Methods Reader (pp. 161-196). Thousand Oaks, CA: SAGE Publications, Inc.
[4]. Foley, R. M., & Pang, L. S. (2006). Alternative education programs: Program and student characteristics. High School Journal, 89(3), 10-21.
[5]. Gall, M. D., Gall, J. P., & Borg, W. R. (2007). Educational  Research: An Introduction (8th Ed.). Boston, MA: Pearson Education.
[6]. Gay, L. R., Mille, G. E., & Airasian, P. (2008). Educational Research: Competencies for Analysis and Applications (9th Ed.). Upper Saddle River, NJ: Pearson Prentice-Hall.
[7]. Haughey, A. (2009). Pupils disengaged from school: Evaluation of an alternative vocational education programme [sic]. Educational & Child Psychology, 26(1), 52-59.
[8]. Kiuhara, S. A., Graham, S., & Hawken, L. S. (2009). Teaching writing to high school students: A national survey. Journal of Educational Psychology, 101(1), 136-160.
[9]. Moores, A., & Parks, M. (2010). Twelve tips for introducing E-Portfolios with undergraduates students. Medical Teacher, 32(1), 46-49.
[10]. Norusis, M. J. (2008). SPSS Statistics 17.0 Guide to Data Analysis. Upper Saddle River, NJ: Prentice Hall.
[11]. Ntuli, E., Keengwe, J., & Kyei-Blankson, L. (2009). Electronic portfolios in teacher education: A case study of early childhood teacher candidates. Early Childhood Education Journal, 37, 121-126.
[12]. Onwuegbuzie, A. J., & Johnson, R. B. (2008). The validity issue in mixed research. In Clark, V. L. P., & Creswell, J. W. (Eds.), The Mixed Methods Reader (pp. 272-298). Thousand Oaks, CA: Sage Publications, Inc.
[13]. Patton, M. Q. (2002). Qualitative Research & Evaluation Methods (3rd Ed.). Thousand Oaks, California: Sage Publications Inc.
[14]. Raley, J. (2010). Not that kind of writing: A conversation with one student about writing in high school and beyond. Catalyst for Change, 36(2), 9-17.
[15]. Searle, J. (2007). Young people re-engaging with learning. International Journal of Learning, 13(10), 175- 182.
[16]. Shields, J. S. (2007). The art of imitation. English Journal, 96(6), 56-93.
[17]. Siegrist, J., Drawdy, L., Leech, D., Gibson, N., Stelzer, J., & Pate, J. (2010). Alternative education: New responses to an old problem. Journal of Philosophy and History of Education, 60, 133-140.
[18]. Suell, J. L., & Piotrowski, C. (2007). Alternative teacher education programs: A review of the literature and outcomes studies. Journal of Instructional Psychology, 34(1), 54-58.
[19]. Tatum, A. W., & Gue, V. (2010). Adolescents and texts. English Journal, 99(4), 90-93.
[20]. Trochim, W. M. K., & Donnelly, J. P. (2008). The Research Methods Knowledge Base. Australia: Atomic Dog/ Cengage Learning.
[21]. Van Acker, R. (2007). Antisocial, aggressive, and violent behavior in children and adolescents within alternative education settings: Prevention and intervention. Preventing School Failure, 51(2), 5-12.
[22]. Wolsey, T. D. (2010). Complexity in student writing: The relationship between the task and vocabulary uptake. Literacy Research and Instruction, 49, 194-208.