Improving Math and Science Scores in Middle School

Program Evaluation Improving Math and Science scores in Middle School TABLE OF CONTENTS Page Needs Assessment 3 Program Theory 10 Logic Model 20 Conceptualization & Operationalization of Program Outcomes 24 Assessment of Program Impact 25 References 28 NEEDS ASSESSMENT Math and Science are two subjects which most students at any level approach with trepidation and intense dislike, however, both subjects are integral to cognitive thinking. Not only will these subjects provide skills that will help students think more clearly, but students will be academically successful throughout their school career, enjoy wider career choices and earn more money after graduation. Therefore, establishing a strong foundation in these subjects is integral to future academic and career success. However, studying these subjects in middle school is even more difficult. Studies have shown that the transition for a student from elementary school to middle school is academically and psychologically difficult. According to Maurice Elias in an article entitled, â€Å"Middle School Transition: It’s Harder Than You Think†, many former elementary school students are not well prepared for the demands of middle school. They need explicit instruction, coaching, and support with regard to organizing time and resources for homework; responding to work that is more challenging and requires more effort; understanding and addressing the varying expectations of teachers in different subject areas; and accomplishing such basic tasks as taking notes and taking tests (Elias, 2001). Unfortunately, this same sentiment resonates today with the New York City Public School system, specifically, middle schools located in low income areas. The New York City Public School System is struggling with mathematics achievement in the grades beyond elementary school. Over 30% of the city’s elementary and middle school students score at the lowest level of the state mathematics test and only 34% of all students pass that test. The mathematics â€Å"problem† seems connected to the third major trend in the data, the low performance of middle and junior high schools in the city. In both Mathematics and English Language Arts, the city’s middle and junior high schools seem to be the weakest link in the system (Domanico, 2002). Recently, the math state scores were released further underscoring the middle school â€Å"math problem† that exists. Results showed that while 75. 3% of students at the elementary level passed successfully only 38. 9% of grade eight students passed (Andreatta, 2006, 11). As such, the intent of this study, based on the aforementioned information, is to evaluate and make recommendations with regard to middle school students in a particular school who have been struggling with both subjects. This study will focus on a middle school, IS 166- George Gershwin School—located in East New York. The decision to choose IS 166 was dependent on a few factors among which included the fact that the district within which it is located is considered a â€Å"virtual educational dead zone† by a Civic Report drafted by the Manhattan Institute for Policy Research (Domanico, 2002). Additionally, after reviewing the New York City Department of Education’s website—which provides an overwhelming amount of information on every public school in the city as well as their progress over recent years in the core subjects—it was found that of the schools within the 19th School District—primarily East New York, IS 166 is one of the worst performing schools. The school’s poor academic performance is further exacerbated by the outstanding grades displayed by another school in the 19th School District such as IS 409—East New York Family Academy and outside of the district another school MS 114, located in District 2 (Manhattan), whose grades superseded the city’s level as well as the state’s level. The graphs below illustrate how IS 166 performed poorly in the last 2 years on the state Math and Science exams comparatively to other schools, specifically IS 409 in the same district as well other schools in other districts. The last two graphs will show the difference with a higher performing school such as IS 409 and therefore will confirm why this study is going to be conducted. IS 166- George Gershwin School Math and Science Grades Source: New York Department of Education (Division of Assessment and Accountability—School Report Cards 2005). Definition of the Levels on which the scores for both subjects are based: Level 4—These students exceed the standards and are moving toward high performance on the Regents examination. Level 3—These students meet the standards and, with continued steady growth, should pass the Regents examination. Level 2—These students need extra help to meet the standards and pass the Regents examination. Level 1—These students have serious academic deficiencies. Source: New York Department of Education (Division of Assessment and Accountability—School Report Cards 2005). The aforementioned graphs showed how poorly IS 166 has performed in the last two years in both Math and Science. In Math, the number of students performing at Levels 3 and 4 has decreased from 22. % in 2004 and 17. 5% in 2005. The number of students tested for Level 3 was only 60 and for Level 4 only 3 of 361 total students. The remaining students, as displayed on the right hand side of the graph, are still at a Level 1 which as noted by the above definition means that they are in grave need of assistance. Therefore, for the purposes of the study, the target population will be defined as â€Å"in need† students. Although there has been a slight increase in Science, the results are still less than desirable when compared with other schools in the district and the City. As seen in the graph, only 14% of the students passed at Levels 3 and 4 in 2004 and by 2005, only 18% were able to pass at the same levels. Therefore, if IS 166 continues on this trajectory, it will continue to be labeled an underperforming school that graduates below average students incapable of performing the basic tasks in both subjects. The goal of the evaluation study is to thoroughly review the problems that exist and hopefully get the school to achieve grades similar to IS 409-East New York Family Academy sometime in the near future as is reflected in the following graphs. IS 409- East New York Family Academy Math and Science Grades Source: New York Department of Education (Division of Assessment and Accountability—School Report Cards 2005). As noted in the above graphs, IS 409 is performing extremely well at Levels 3 and 4 and has outperformed schools in both math and science in the district (which is truly exceptional given the neighborhood and its history) as well as other City schools. Very few students if any are far below the standard in both subjects. Moreover, as noted before, other schools such as MS 114 and IS 289 located in District 2 have maintained exceptional scores over the two year period. For 2004 and 2005, MS 114 scored 88% and 81% consecutively in Math and 97% and 91% in Science. IS 289 also scored high grades-for both years in Math, the school displayed 83% and 73% when compared to other schools in the district and city and in Science, they scored 87% and 82%. Other schools in other districts from Queens and Staten Island have also demonstrated solid scores. This makes designing a program even more of a priority in light of the above referenced comparisons. The study will not focus on the students at all levels in the middle schools but specifically, the eighth grade students destined for high school who have yet to grasp the necessary skills needed to succeed and have been the center of test score analysis over the years. These eighth grade students will be approximately 14 years old but depending on factors such as repeating a grade or special needs, the age may vary from 14-16 years old. As noted before, they will be identified as â€Å"in need† students and the study will attempt to identify the worst performing students by looking not only at grades but possibly contributing factors such as income, special needs, and possible crime involvement. The improvement of Math and Science scores is a gargantuan task which requires a major overall of the school at all levels, however, to begin the following services are needed and they are but not limited to: ?Offering training sessions for the math and science teachers. The difference between not only IS 409 and other schools in District 2 is that the teachers have more experience, education, and are less likely to be absent more than average. The training sessions will be implemented on weekends or after-school whichever is more convenient for the teachers and will be done prior to establishing an after-school program for the students. The training sessions will allow teachers from higher performing schools an opportunity to impart their techniques for achieving higher grades. Offering a separate informative session for the Principal, Maria Ortega, so that she is more knowledgeable on what is needed to succeed in both areas. In most cases, the principal of a school has a general idea of what is needed in most subject areas, however, if the principal is more involved, informed, and fully comprehends the nuances of the subject matter, then she will be able to make better choices in hiring and understanding the teaching of the curriculum. This is an i dea which originated out of reading the case of MS 114 in District 2 which showcases a principal that has not only taught but has written Math books for children. Also, in IS 289, the principal knows each student individually and is fully acquainted with their needs. ?Offering additional services for children that may range from an after-school program to extending class hours to offering classes on the weekend. One of the schools in District 2 actually has classes that last at least 50 minutes giving students a better opportunity to absorb the material thereby performing better in exams. ?Offering programs that will incorporate the parents as well. Perhaps this will be in conjunction with the after-school program. As noted, most of the students in this district are from low income families and perhaps some of the parents are in low paying jobs or living on welfare. The parents can take advantage of the program by refreshing themselves with the basic concepts of each subject so that they may assist their children and perhaps help themselves. PROGRAM THEORY In order to address the dire academic situation at IS 166-George Gershwin School, and before implementing an after-school program, it is important to address the issue at the higher levels which means analyzing teaching techniques and more importantly, principal participation. At the Center for Civic Innovation Luncheon featuring Chancellor Joel Klein held on Thursday, October 5th at the Harvard Club, Chancellor Klein began his speech with an analogy of the leaky roof and the squeaky floor. He stated that there was a school located in uptown Harlem that had a leaky roof and a squeaky floor. One day a repair man came to repair the floor and the custodian stated that the floor cannot be fixed prior to the roof being fixed to which the repairman replied â€Å"That’s not my concern, I am just here for the floor†. The Chancellor began his speech with that story to underscore the problems with the NYC Education system. He believes that everyone wants to fix the underlying problems without addressing the issues at the surface. The Chancellor’s story may be applied to the case of IS 166 and any other school in need of improvement. Many observers and parents are often led to believe that their children are primarily the problem in achieving higher scores and possibly that their children lack the intellect to truly analyze or process the information given to them. However, it is just as important for the heads of the respective schools to be cognizant of what is needed to improve these scores and the principal is just the person to ensure this. Therefore, before implementing a program, we have recommended that Principal Maria Ortega participate in a briefing session lasting approximately one month in the summer—right after the end of the school year and before the hiring season begins—for at least 4 hours a day, three days a week. According to reports of comprehensive school reforms in Chicago and Louisiana, the schools’ academic success was primarily attributed to the principals in charge and the contributions they made throughout the reforms. In one report, it stated that â€Å"highly effective schools communicated expectations for teachers. The principal was active in working to improve teacher skills; ineffective teachers were let go. † Moreover, the principals played an important role in four areas a) selection and replacement of teachers; b) classroom monitoring and feedback; c) support for improvement of individual teachers; and d) allocating and protecting academic time (Good et al, 2005, 2207). Therefore, implementing a program or briefing session solely for Principal Ortega would help her improve in all these areas. Principals, under Chancellor Klein’s tenure, have been given more empowerment opportunities and have more responsibilities to ensure the success of their schools. IS 166 has been categorized as a Title I School In Need of Improvement (SINI) under the No Child Left Behind Act (NCLB) and as such, Principal Ortega has to work harder than ever to improve the english, math, and science scores—subjects that are integral to a student’s academic success. The program we have suggested will illustrate to Principal Ortega that math, in particular, cannot be taught in the traditional manner, that is, using rote. In fact, the National Council of Teachers of Mathematics (NCTM) advocates the development of an inquiry-based mathematics tradition. Students taught using this tradition are encouraged to explore, develop conjectures, prove, and problem solve (Manswell Butty, 2001, 20). Students are best able to absorb the material in not only math and science but other subjects if the teachers are able to present it in an interesting manner that entails connections to the outside world. Principal Ortega should also be familiarized with the requirements for the exams and then know exactly how the staff should approach student preparation. She should also ensure that with respect to math, she adheres to the recommendations Lyle V. Jones reiterated in his article entitled â€Å"Achievement Trends in Math and Science† in which it was stated: ? Only teachers who like mathematics should teach mathematics ? The chief objective of school mathematics should be to instill confidence ? Mathematics teaching must be based on both contemporary mathematics and modern pedagogy (Jones, 1988, 333). After completion of this program, and hopefully with a better understanding of what is needed to improve the scores at IS 166, the next step would be to address the teaching staff. As noted, the methodology used is integral to ensuring that the students comprehend, absorb, and analyze the information being disseminated. If they fail to process the information then they will ultimately perform poorly in the state exams and possibly continue to do so throughout high school. We recommend prior to the beginning of the academic year, and the implementation of the after-school program, that teachers, specifically, the math teachers enroll in a summer institute similar to one reported in an article entitled, â€Å"Toward a Constructivist Perspective: The Impact of a Math Teacher InService Program on Students†. The reason being is that the teaching of math more so than science requires certain techniques that are far from the traditional methods that most teachers employ. The summer institute in the report offered participating teachers intensive two-week summer institutes and weekly classroom follow-up during one academic year. Moreover, they received an opportunity to reexamine their ideas about the teaching and learning of mathematics. During the summer institutes, these teachers experienced mathematics classes in which they were encouraged to construct solutions and ideas and to communicate them to a group. They analyzed student understandings as revealed in interviews and they planned lessons which reflected their evolving ideas about mathematics learning and teaching (Simon and Schifter, 1993, 331). Teachers need to plan their lessons in such a manner as to engage the students so that they may effectively communicate their thoughts or problems with a particular issue. In fact, after completion of the summer institute, and after the teachers began using their newfound techniques, the results were noteworthy and ranged from students stating that â€Å"it’s fun to work math problems† to â€Å"I’d rather do math than any other kind of homework† to â€Å"I like to explain how I solved a problem†(Simon and Schifter, 1993, 333). Therefore, using the above referenced example, the summer institute that we propose for the math teachers will last approximately three weeks in the summer and it would begin approximately mid-August prior to the beginning of the academic year. This program would be mandated by the principal and would include veteran staff members as well new ones brought on board. Another factor that teachers have to take into consideration is the population they cater to during the academic year. IS 166 consists of predominantly black and Hispanic students residing in East New York and its surrounding environs, thereby, making them not only an â€Å"in need† group in terms of grades but an â€Å"at risk† group in terms of their backgrounds and predisposition to engage in illicit activities. Many believe teaching techniques are generic and if they are employed in one school then they may be applicable in another. However, studies have shown that minority children in low income neighborhoods require a different set of techniques employed. According to Manswell Butty, African-American children have further been identified as favoring four learning styles a) person-centered, b) affective, c) expressive, and movement oriented (Butty, 2001, 23). Therefore, teachers need to use laboratory or group exercises, discussion sessions, or instructional uses of music and the visual and dramatic arts, especially when those pedagogical techniques promote Black students’ greater academic involvement, interest, and performances (Butty, 2001, 23). However, this is not a generalization implying that all minority children respond to this technique but most will probably respond positively. Therefore, teachers must be made aware of the group of children that they are dealing with and ensure that they employ the above referenced techniques to garner success. In fact, there are Learning through Teaching in an After-School Pedagogical Laboratories (L-TAPL) in California and New Jersey, which not only offer a program for elementary students but also serves as a practice-rich professional development for urban teachers. The program aims to improve the achievement of urban students and the competence of their teachers (Foster et al, 2005, 28). According to the Foster article, numerous studies, policies, and programs have addressed the persistent problem of underachievement among poor urban students and its array of possible causes. The NCLB links teacher quality to improved student achievement, especially among low-income urban children of color. Consequently, improving teacher quality has become one of the hallmarks of current reform efforts (Foster et al, 2005, 28). These laboratories groom future urban teachers to deal with students similar to the target population at IS 166. And as such, as an alternative to our summer institute, the teachers are free to enroll in the program offered by this lab in New Jersey. Therefore, taking into account the above referenced studies, improving teacher quality is of utmost importance when taking into consideration the improvement of math and science scores. All of the above has brought us to the most important element of the study establishing an after-school program. Establishing an After-school Program-Resources Funding Under the NCLB Act, Title I schools, such as IS 166 that are listed as Schools In Need of Improvement, have failed to reach student achievement targets that have been set for every school. This means the school has failed to meet state proficiency level for all students in English Language Arts, Mathematics, Science and/or high schools graduation rate. Schools falling in the above referenced category may be eligible for Supplemental Educational Services (SES). SES include free after-school/weekend remedial help or tutoring services. The SES provision offers providers an opportunity to offer low-income children, who may be struggling in school, extra academic help and individual instruction. Through SES, innovative leaders and educators can start a new tutoring program or expand an existing one to serve more students (New York City Department of Education). However, instead of using an SES provider—which in some cases the DOE will offer contracts of over a million dollars to provide services to various schools—we will request additional funding that would have been used to acquire an SES provider to establish the after-school program by ourselves with the assistance of The After-school Corporation (TASC). TASC is renowned for establishing successful after-school programs and have no contract with the DOE and thus, are not labeled SES Providers. In addition to wanting to establish a program using solely school staff, it is important to note, that there have been several complaints about SES providers and most are being investigated either by the Special Commissioner of Investigation for the New York City School District or the Office of Special Investigation and in the best interest of the target population, we have decided to forego those providers. Therefore, the funding used from SES will be used to offer per session rates for the teachers participating in the program as well as pay for the consultant from TASC. The funding will also be used to acquire additional supplies such as the KidzMath program which is highly popular and is used around the country to get students interested in math and to improve scores. Funding will also be used to secure additional bus transportation from the Office of Pupil Transportation as well as food and refreshments for the children. Staffing and Facilities The program will be housed in the school recreation room and so there will be no need to rent a facility to do so. The program will be supervised by the TASC consultant who will preferably be someone from the community who is familiar with the target population and can easily relate to their situation. The principal and assistant principal will take turns observing the classes and ensuring that the teachers and participants are abiding by the rules. The teachers will be eighth grade math and science teachers who deal with the target population on a daily basis and who are familiar with the problems they are experiencing. Additionally, the teachers will be assisted by high school students who are well versed in the subject areas, who have been recruited from neighboring high schools and would like to add an after-school tutoring activity to their resume. Therefore, these students will not be paid but will use the after-school program as a learning experience. Participants The students participating in the program will be chosen based on their past academic performance in grades six and seven and failure to show any signs of improvement. To reiterate, this program is geared specifically for eighth grade students, ages 14-16 years old, and will begin a month into the beginning of the academic year towards the end of September early October after the students and teachers have settled in the new semester. Letters will be sent to the parents at the beginning of the academic year notifying them of their child’s progress and advising them that the program is mandatory if they are to improve and move on to high school (the letters will be followed up by phone calls). While the school has no recourse if a student fails to attend even though it has been marked mandatory, offering a voluntary program usually encourages those that are really not in need of it to participate and those that do need it usually don’t. The parents will be informed of the structure of the program and the fact that transportation will be provided so that their children will be taken home safely after the program. In fact, parents who may not be working full time or at all will be encouraged to observe or participate in another session that will help them to understand what their child needs to improve. The session, which will last as long as the tutoring session, will more than likely be conducted by the assistant principal or a math/science staff member and will give the parent an opportunity to be truly acquainted with the activities being conducted. This program may also be helpful to them as well as some of these parents lack the basic educational skills that are necessary to obtain a job. Activities and Schedule Based on successful programs in Arkansas, the after-school program we will establish will mirror these successful programs and therefore, the program will entail classes of one and a half hours each day, Monday through Thursday between the hours of 3pm and 4:30pm. Mondays and Wednesdays will be dedicated to math and Tuesdays and Thursdays will be dedicated to science. The sessions will be divided into 40 minute periods during which the first period will be dedicated to the teacher illustrating the subject material and the second period will be dedicated to the students participating in groups and working together to complete the work presented in the first period. The students will get a ten-minute break during which they will receive refreshments. In the Camden School District in Arkansas, school officials credited the success of the after-school programs to the schools being released from the â€Å"Adequate Yearly Progress† (AYP) status under the NCLB Act (Arkansas Advocates for Children & Families, 2006). Throughout the course of the program, teachers will be encouraged not to utilize the same material or techniques used on a daily basis. The teachers will be reminded that the program is geared towards individuals who have a negative attitude toward the subjects which may be as a result of not only failure to comprehend the material but also the teacher’s emphasis on traditional methods. Therefore, the program will forego any emphasis on memorization, computation, and equation and will focus on modeling and real world problem solving. Engaging in group work, especially in math, has proven to be successful and will be the focus of the program. According to Jones, group work differs from cooperative learning in its lesser emphasis on the teacher as instructor and its greater dependence on students teaching other students. Moreover, cooperative learning procedures as dependent first on instruction by the teacher, then on practice engaged in actively by members of an established student team (often of four team members), has evidence that supports the efficacy of the approach to elevate not only achievement but also self-esteem, interpersonal effectiveness and interracial harmony (Jones, 1988, 328). Therefore, the students will work together in groups over the period of the academic year and will be exposed to hands-on experiences, games, and projects. KidzMath should really be a good stimulant and with the assistance of the teachers, the students should be motivated. Teachers will also be encouraged to maintain a weekly progress report which will ultimately be used to assess the program’s progress. Another aspect of the program would entail having the Principal establishing stronger ties with the community and getting more community leaders involved by dropping by the after-school program to give advice and encouragement to the students. Students are not only stimulated by various activities that are outside of the norm of the regular classroom but are also stimulated by role models or individuals they deem to be successful from their part of the neighborhood. According to a report done on the Chicago School Reform, the schools that experienced major changes and improvements were led by principals who were strong veteran leaders with good relationships with their local school councils and the community (Hess, Jr. , 1999, 79). Additionally, incentives can also be offered for the students in the program which will encourage their continued participation and potential success and can range from visits to museums or amusement parks if they have showed slight improvements. While these children who performed poorly are from low income families, and a reduction in poverty rates might have a salutary effect on measured school achievement, according to Lyle V. Jones, the influence of poverty on educational achievement may be ameliorated by introducing school-parent programs to improve academic conditions in the home. After reviewing nearly 3,000 investigations of productive factors in learning concludes that such programs have an outstanding record of success in promoting achievement (Jones, 1988, 327). Explanation of Logic Model Inputs: consist of the fundamental resources—human and capital—that the program needs in order for it to achieve its goals. These resources consist of funding for per session rates for the teachers, payment for the TASC Consultant, supplies such as KidzMath, transportation, and refreshments. The most important resources needed are the children to whom the program is directed. Activities: Once the fundamental resources are in place, the schedule has been established and the techniques for teaching have been agreed upon, then the after-school program will proceed as planned throughout the academic year. The sessions will be conducted four days a week, Mondays and Wednesdays, for math and Tuesdays and Thursdays for science lasting 1. 5 hrs each period. The sessions will entail a great deal of group work and collaboration along with potential visits from community leaders and role models. Outputs: Upon implementation of the program, it is important to ascertain if the program is reaching its target population, if the services provided are being done in the manner discussed and if the population are benefiting or if they have any concerns those will be noted throughout the assessment. This will be done by conducting site visits, performing observations and conducting surveys. Outcomes: If the program is successful in achieving its goals, then the immediate goals will see the students passing their in-class tests and ultimately the state exams—which has been the focal issue with the school and the reason for the Title I status under the NCLB Act. The long-term goals include the participants of the program actually going on to high school and possibly even college. From that point onward, if students succeed in college, they may even pursue challenging careers thereby improving their socio-economic status. The reason the logic diagram is done in a cyclical manner is to demonstrate that if the program is successful and the students do improve significantly, then the school may be eligible for the same amount or a higher amount of funding which they can use to increase their resources for the input phase for the upcoming academic year. PROGRAM PROCESS Once the program has been implemented, it is important to ascertain if the services are in fact being delivered as planned and if the participants are learning with the teachers employing the new techniques as discussed. In order to do this, we will conduct an observational study as fashioned from the TASC’s site visit procedures in addition to teacher and parental surveys to see if they have noted any differences in the children participating in the program. This assessment will be done halfway throughout the semester at approximately the end of January which will also coincide with the first set of state exams (students also take these exams towards the end of the academic year-approximately June). The assessment will begin with a two-person team (my colleague and I) visiting the after-school’s program for two days a week, for a total of two weeks—one day for math and the other for science. The visit will include an interview with the principal and assistant principal (who, as noted before, would have taken turns monitoring the program). There will be 90 minute observations including the 10 minute break to see how the children are behaving and the teachers’ interaction with them accordingly. The assessment will look at three of the five primary factors as fashioned from TASC’s rating on project activities: Staff-directing relationship-building ?Staff use positive behavior management techniques ?Staff show positive affect toward youth ?Staff attentively listen to and/or observe youth ?Staff encourage youth to share their ideas, opinions, and concerns Staff strategies for skill-building and mastery ?Staff verbally recognize youth’s efforts and accomplishments ? Staff assist youth without taking control ?Staff ask youth to expand upon their answers and ideas ?Staff challenge youth to move beyond their current level of competency ? Staff plan for/ask youth to work together ?Staff employ two or more teaching strategies Activity content and structure ?The activity is well organized ?The activity challenges students intellectually, creatively, and/or physically ? The activity requires analytic thinking The observers will rate each indicator on a scale from 1 to 5 where 1 meant that the indicator was not evident during the observation period and a 5 meant that the indicator was highly evident and consistent. These ratings will provide a systematic method for the observation team to quantify its observations of the factors that contribute to the possible success of the program (TASC Catalog of Publication and Reports, 2005, 3). The assessment will also ask teachers to document any changes they have observed in their students’ behavior throughout the program. This will be extracted from a weekly progress report that they were encouraged to write at the commencement of the program. This will give us an idea if the students have made any progress in the eyes of the educators. The last assessment will be done with the parents who will be asked their views of the program. The questions will include but not be limited to: ?Is the program meeting your expectations? ?Do you see any noticeable changes in your child’s progress? Does your child show any more interest in math or science? ?Do you feel you have benefited from observing or partaking in the informative sessions conducted by the principal or staff? ?Are you satisfied with the transportation provided? These questions will receive ratings from 1-5 as noted above and will give us an overall idea of the process of the structure. We can use the res ults of the assessment to facilitate mid-term improvements before the conclusion of the program. The results can also be used for future improvements should the after-school program enter its second academic year. CONCEPTUALIZATION AND OPERATIONALIZATION OF PROGRAM OUTCOMES The goal of this study is to determine the impact of an after-school program on improving the scores of low performing eighth grade students in IS 166. Therefore, the hypothesis is eighth grade Math and Science students who have performed below average in state exams are more than likely to improve their grades in both subjects after enrolling and completing the year long after-school program. In this case, the independent variable would be the after-school program and the dependent variable would be the overall improvement in grades. Independent Variable: After-school Program The after-school program (in this project) may be conceptualized as any academic activity that takes place outside of the mandated school hours that is geared towards the improvement of a child’s academic achievement in a specific subject area. It may be operationalized by examining the responses from the observations conducted in the assessment phase which were based on five primary factors ranging from staff-directing relationship building to staff strategies for skill-building and mastery to activity content and structure. Under each category there are various indicators which will be rated on a scale from 1 to 5 where 1 is meant that the indicator was not evident during the observation period and a 5 meant that the indicator was highly evident and consistent. Dependent Variable: Overall Improvement in Grades Overall improvement in grades may be conceptualized as a notable or significant increase which may be anywhere from 15-20% in the in-class and state scores. The increase in scores would hopefully translate into passing grades. Improvement in grades can be operationalized by examining both the in-class and state test scores and comparing both to the previous year’s scores and as such, we can begin to measure some sort of success based on the increase in the scores. It should be noted that while the overall improvement in grades is the primary dependent variable on which the focus is placed, there are other variables that should be taken into account, however, due to the constraints of this paper, they will be mentioned briefly. They are but not limited to: improvement in student attitudes—that is the effect the after-school program has had on their approach to the subjects. Do the students now have a positive attitude towards the subject after improving their ability to process and analyze the new information provided? Also, there is the parental support aspect which must be taken into consideration. Did the after-school program increase parental awareness, that is, making parents aware of what students need to excel in both subjects? Do parents now know how to assist or provide support for their children in these subject areas? Assessing Program Impact—Strategy In order to determine if the after-school program had an effect on overall Math and Science scores, a randomized control-group pretest and posttest design will be conducted. (Please note that steps 1-3 would have been done prior to the implementation of the after-school program). The following steps will be followed in order to execute this test: 1) Students will be selected from the eighth grade roster by random methods, specifically, randomly choosing social security numbers from the database. ) The students with social security numbers ending in even numbers will be assigned to the treatment group (X)—the after-school program, while the students with social security numbers ending with odd numbers will be placed in the nontreatment group (Y). 3) An in-class test similar to that given at the state level will be administered to both groups to ascertain their scores—the dependent variable. The scores will be added for both the ex perimental and control group. 4) After totaling the scores, the experimental phase will begin. Both groups will be exposed to the same conditions with the exception of the experimental group (X) who will have the experimental treatment—the after-school program for the academic year. 5) After the experimental group has completed the after-school program, both groups will be evaluated again using an in-class test similar to the one given in the pre-testing period. Once again, the scores will be added for both the experimental and control group. 6) The scores between the pre-testing period and the post-testing period will be calculated to establish the difference. ) The difference in the scores will be compared to determine if the after-school program (the treatment) was associated with a change favoring the experimental group over the control group—who did not participate in the after-school program. 8) A statistical test will be used to determine whether the difference in the scores is truly significant—that is, if the difference is large enough to reject t he null hypothesis that the difference is simply a chance occurrence. According to Stephen Isaac in his book, â€Å"Handbook in Research and Evaluation† nternal validity gains strength with the randomized design because extraneous variables are controlled since they affect both groups equally (Isaac, 1971, 39). To elaborate, extraneous variables such as differential selection is controlled by random selection methods. Maturation and pre-testing effects occur equally for all groups, differential mortality can be assessed for nonrandom patterns, and statistical regression is controlled when extreme scorers from the same population are randomly assigned to groups (statistical regression will occur but it will occur equally with all groups) (Isaac, 1971, 39). The disadvantages to this design are to be found in the within-session variations during which time the experimental and control groups are tested and treated separately. There may be differences in room conditions, personalities of teachers, or wording of instructions. According to Isaac, the students should be tested individually or in small groups, randomly assigning subjects, times, and places to experimental and control conditions. The effects of any unwanted situational factors are thus randomly distributed among the subgroups, allowing them to be ignored (Isaac, 1971, 39). Isaac further states that to control for within-session instrument differences, it is necessary also to assign mechanical instruments, teachers, observers and raters to sessions—or preferably to a single session. Ideally, if observers or judges are involved, they should remain unaware of which groups are being used for control or experimental purposes, since they may have subtle biases that could influence their observations. REFERENCES Andreatta, Dave. â€Å"Math Concerns Are Adding Up† New York Post, October 12, 2006: 11 Arkansas Advocates for Children & Families (2006). After-school programs in Arkansas: A solution whose time has come. Little Rock, AR author Accessed on 10/29/2006 http://www. arkleg. state. ar. us/data/education/ Birmingham, Jennifer, Pechman, Ellen M. , Russell, Christina A. , and Monica Mielke. â€Å"Shared Features of High-Performing After-School Programs: A follow-up to the TASC Evaluation† TASC Catalog of Publications and Reports, November 2005. Accessed on 11/2/2006 Domanico, Raymond. State of the NYC Public Schools 2002† Civic Report-Manhattan Institute for Policy Research. March 2002, # 26. Accessed on 10/16/2006 Elias, Maurice. â€Å"Middle School Transition: It’s Harder Than You Think-Making The Transition to Middle School Successful† Middle Matters, Winter 2001: 1-2 Accessed on 10/19/2006 Foster, Michele, Lewis, Jeffrey and Laura Onafowora. â€Å"Grooming Great Urban Teachers† Educational Leadership, March 2005, (62) 6 : 28-32. Good, Thom as, L. , Legg Burross, Heidi, and Mary M. McCaslin. Comprehensive School Reform: A Longitudinal Study of School Improvement in One State† Teachers College Record, October 2005, (107) 10: 2205-2226. Hess, Jr. , G. Alfred. â€Å"Understanding Achievement (and other) changes under Chicago School Reform† Educational Evaluation and Policy Analysis, Spring 1999, (21) 1: 67-83. Isaac, Stephen (1971). Handbook in Research and Evaluation. San Diego: EDITS Publishers Jones, Lyle V. â€Å"Schooling in Mathematics and Science and What Can Be Done to Improve Them† Review of Research in Education, 1988-1989, (15): 307-341. Manswell Butty, Jo-Anne L. â€Å"Teacher Instruction, Student Attitudes and Mathematics Performance among 10th and 12th grade Black and Hispanic Students† The Journal of Negro Education, Winter-Spring 2001, (70) ? : 19-37. New York City Department of Education 2004-2005 Annual School Reports (Provided by the Division of Assessment and Accountability) Accessed on 10/14/2006 Simon, Martin A. , and Deborah Schifter. â€Å"Toward a Constructivist Perspective: The Impact of a Mathematics Teacher InService Program on Students† Educational Studies in Mathematics, December 1993, (25) 4: 331-340.