NSF TBR Teacher Preparation Program

Partnership Evaluation Report

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Prepared by:

 

The Institute for Assessment and Evaluation

College of Education, Health and Human Sciences

The University of Tennessee

March 2, 2009


 

Contributors to This Study

 

Data collection, analysis, and report preparation have been conducted by the following team members:

 

Dr. Gary Skolits, Evaluation Principal Investigator

Dr. Judy Boser, Senior Research Associate

Ms. Erin Mehalic Burr, Graduate Research Assistant

 

 

All of the above referenced evaluation team members are associated with the: 

 

Institute for Assessment and Evaluation

College of Education, Health and Human Sciences

University of Tennessee

1126 Volunteer Blvd, Bailey Complex A503

Knoxville, TN 37996-3456

(865) 974-2777

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

We also thank our support staff, Ms. Shauna Cook for the many hours of work she contributed to the study.

 

 

 

 

 


 

Preface

The Tennessee Board of Regents (TBR) – Teacher Preparation Partnership (TPP) project received funding from the National Science Foundation for a three-year period:  Year 1, 2005-2006; Year 2, 2006-2007; and Year 3, 2007-2008.  The original project grant application was designed to include all Tennessee two-year and four-year higher education institutions which are governed by the Tennessee Board of Regents.  However, the eventual funding of $1.25 million awarded to the project was substantially less than the funding requested, and the reduced level of resources required that the project be scaled back.  In response, the project was only able to fund and include the Tennessee Board of Regents institutions in the eastern region and a portion of the middle of the state.  Ultimately, the following eight institutions participated in the grant:

·       East Tennessee State University (ETSU)

·       Tennessee Technological University (TTU)

·       Chattanooga State Technical Community College (CSTCC)

·       Motlow State Community College (MSCC)

·       Northeast State Technical Community College (NSTCC)

·       Pellissippi State Technical Community College (PSTCC)

·       Roane State Community College (RSCC)

·       Walters State Community College (WSCC)

 

This TPP grant project built upon several previous articulation projects, especially a project that enhanced science and mathematics teacher education content courses articulated between Pellissippi State and Tennessee Technological University (TTU).  Pellissippi State and TTU staff who had been involved in previous efforts was instrumental in the development of the current TPP grant addressed in this evaluation. For the current TPP grant, Pellissippi assumed the role of lead institution (grant coordinator) and fiscal agent for the project.  The senior project principal investigator, Dr. Jim Kelley, is a sciences dean at Pellissippi State.  Three co-PIs participated in the administration and coordination of the grant project (Dr. Judy Hector at Walters State; Dr. Maggie Phelps at Tennessee Tech, and Dr. Anant Godbole at East Tennessee State University). 

The original project evaluator who began work at the inception of the project left about mid-way through the three-year life of the grant.  As a result of his departure, the senior project principal investigator requested assistance from the Institute for Assessment and Evaluation (IAE) for evaluation services throughout the remainder of the grant.   Given that the life of the grant is essentially over (year three ends in August of 2008), the following is the first of two evaluation reports to be prepared by the IAE evaluation team.


 

Table of Contents

 

 

Contents                                                                                                                Page

 

 

Preface .....................................................................................................................  i
 
Executive Summary .................................................................................................  3

 

Introduction .......................................................................................  7

 

Data Collection Methods ...................................................................  7

 

Findings ............................................................................................ 11

 

§      Evaluation Question One ............................................................. 11

§      Evaluation Question Two ............................................................. 16

§      Evaluation Question Three ........................................................... 26

 

            Summary of Conclusions and Recommendations .......................... 27

 

 

 


 

Executive Summary (cONCLUSIONS and recommendations)

            In the following report, findings, conclusions, and considerations for the future are presented in response to the evaluation study questions.   As an executive summary, the evaluation team consolidated their conclusions and recommendations as follows.

Conclusions

Based upon the data collected and analyzed for the evaluation of the third and final year of the project, the evaluation team offers the following conclusions:

Conclusions - Evaluation Question 1 (Achievement of Project Priorities)

  1. TPP project staff, campus level project coordinators, and faculty addressed the project priorities.  There is substantial evidence that meaningful progress was achieved in establishing systemic change in community college and university collaboration in the preparation of K-6 teacher education students. 
  2. This project successfully established and implemented a unique standards-based content faculty collaboration model that included the active engagement of two-year and four-year teacher education and content faculty.
  3. While two-year to four-year program articulation has a long history in Tennessee, the process for community college - university articulation has been substantially strengthened for K-6 teacher education.
  4. This project resulted in a fully articulated standards-based science, math, and technology curriculum for elementary education majors at the eight participating campuses. 

 

Conclusions - Evaluation Question 2 (Project Outcomes)

  1. Due to the Partnership nature of this project, the number of faculty members who participated in the project are substantial. Moreover, review of institutional project reports and interviews with participants are laudatory of the collaboration fostered by the project. 
  2. All participating institutions have made changes in their courses and curricula, although a few changes, new courses in science, have not yet been implemented.
  3. It is still too early to determine the impact of this project on the following elements of participating elementary education programs: 

-    enrollment

-    majors

-    graduates

-    diversity of the teaching pool

However, the necessary data will be available in the future, after the grant has been completed.

 

 

  1. The project is well received by faculty and students, and it is impacting courses and faculty instruction beyond the scope of the project (elementary education coursework in mathematics, science, and education). 

Conclusions – Evaluation Question 3 (Sustainability)

1.      Several aspects of this project have a strong potential to be sustained (course content, faculty collaboration, pedagogical enhancements, etc.), including the curriculum change model introduced with this project.

 

Recommendations

The evaluation team offers two types of recommendations: overarching recommendations and recommendations resulting from the analysis of data related to each specific evaluation question.

Overarching Recommendations

Based upon the data collected and analyzed for all aspects of the evaluation for this project, the evaluation team offers recommendations for promoting the sustainability of this project as well as the potential enhancement of future projects involving project staff or NSF (funding agency). 

Overall, three themes emerged from this evaluation that suggest a potential framework for enhancing project sustainability and the long-term monitoring of project impacts. These include: 1) the ongoing collection of student achievement data for continuous program improvement; 2) the collection of specific student data (enrollment, retention, graduation), including demographics on diversity, for monitoring program productivity and its effects on the diversity of the teacher pool; and 3) the sustaining of ongoing opportunities for teacher education and content faculty collaboration and interaction across institutions on both informal and formal bases.  The evaluation team addresses each of these themes below.

 

1.      Collection of Teacher Education Student Achievement Data

 

Consistent with the grant proposal, the focus of this project was on teacher education content course curriculum design, development, and implementation; a substantial level of project activity and inter-institutional effort and time was necessarily focused on the Partnership development of new/revised content and pedagogical strategies. Accordingly, several courses developed or re-designed through this project were implemented later in the project or are scheduled for initial implementation after the scheduled three year life of the grant project has ended.  Given the time required for Partnership curriculum design and implementation in a three-year grant schedule, the opportunity to collect more than initial outputs and outcomes data is limited. More definitive data will not be available until all courses affected by the project have been implemented and a critical mass of students have enrolled and completed the whole program sequence, attained their teaching license, and entered the teaching workforce.

 

However, participating institutions have a unique opportunity to ensure that critical student achievement outcome data related to this project are collected and integrated into future ongoing institutional effectiveness assessments. These data would assist in the monitoring of elementary education courses and the overall effectiveness of elementary education programs of these institutions.  The collection and analysis of these data would be consistent with Tennessee Board of Regents and accreditation agencies expectations for ongoing program evaluation and the subsequent use of evaluation results for continuous improvements.  As such, consideration should be given to the following assessment elements related to the elementary education students and programs addressed by this grant project:

 

·         Offering of end of course assessments to ensure desired teacher education standard “competencies” have been met 

·         Development of common end-of-course assessment instruments for these teacher preparation courses within institutions (This presented an interesting situation because the State of Tennessee revised the state K-12 mathematics and science standards this year to align them with NAEP. This new information is scheduled to be addressed in the August project workshop which is specifically aimed at the standards as well as proposed workshops occurring throughout the no cost extension of the grant).

·         Sharing of teacher education student PRAXIS I and II assessment results with each student’s originating community college (it should be noted that some community colleges were not aware of how to retrieve their PRAXIS I data).

·         Sharing of assessment results among participating community colleges and universities

      A plan for collecting, summarizing, and disseminating these results among participating two-year and four-year institutions would require substantial thought and discussion among faculty and academic administrators. There will be some technical and practical problems in collecting these data that would need to be addressed. To assist this process, project institutions may find assistance from their campus institutional research staff to be of significant benefit.

2.  Program Outcomes Data (including student counts/demographics related to course and program enrollment, retention, graduation, licensure, and employment) at the elementary education program level.

The elementary education programs of the two-year and four-year institutions participating in this project have the potential to serve a vital need for more, better-trained (including training in STEM content areas), and highly qualified teachers across east and middle Tennessee as well as enhance the diversity of the teacher labor pool as envisioned in the grant project. However, it is still too early to determine the effects of the project on addressing the teacher shortage and the diversity of the teacher pool in eastern and central Tennessee. Over time, as all project courses are implemented and more students enter the program, successfully complete it, and become licensed teachers, it will be possible to determine the long-term project impact. However, this outcomes assessment requires institutional assessment data that are disaggregated to the level of the elementary education programs that were the focus of this grant. Some of the initial campus project data reporting has been at an aggregate level (i.e., all education majors), and this aggregated data does not inform impact assessment of this project. 

3. Sustain Formal/Informal Opportunities for Content Faculty Collaboration. 

The following evaluation report will offer evidence that one major benefit or effect of this project was the opportunity it provided faculty to interact with their colleagues from other institutions, especially colleagues who work at a different institutional level (e.g., two-year or four-year).  Beyond the collaborations sponsored by this project, campus content faculty reported having limited opportunities to meet with these colleagues at other state/regional meetings. Usually typical (non-grant) collaboration opportunities were not structured to provide focused opportunities for inter-institutional faculty engagement. One of the strengths of the faculty collaboration provided by this grant was that faculty had the opportunity to work together for a common purpose (i.e., collaboration of content faculty within/between institutions focused on standards-based teacher education). It should also be noted, as reflected in the interim report, that there has been considerable visitation among project faculty within the past year. These collaboration opportunities, and the many benefits they offer faculty to work together to improve course and program articulation, need to be sustained if at all possible. 

Recommendations Per Evaluation Question

Throughout the following report, the evaluation team has developed specific recommendations resulting from the analysis of data collected for each evaluation question.  The following is a summary listing of all of recommendations offered for each evaluation question:

Recommendations - Evaluation Question 1 (Achievement of Project Objectives)

1.      Project staff should promote elementary education K-6 data collection related to this project after the grant has ended.  The data necessary to provide a critical mass of definitive information regarding the long-term effectiveness of this project will not be available until more students have completed the program. 

2.      Institutions need to address the challenges of providing project data on the diversity of the teaching pool.  Diversity of the “teacher pool” in Tennessee was a key aspect of the grant, and analysis of the effects of the grant is incomplete without diversity data related to elementary education teacher enrollments, graduates, transfers, etc., at the project institutions.

Recommendation - Evaluation Question 2 (Project Outcomes)

  1. Project institutions should develop and implement a methodology that accurately reflects the number of faculty members who have been affected by the project. The numbers of faculty participants provided in the institutional reports appears to undercount faculty involvement in the project.

Recommendation - Evaluation Question 3 (Sustainability)

  1. Project staff should encourage institutions to promote the sustainability of project elements, and they should share the project model, especially the Partnership nature of the project, with other Tennessee Board of Regents institutions.

 

 

 

 


 

I.   Introduction

            This evaluation report addresses the progress achieved on the TBR-NSF Teacher Preparation Partnership (TPP) project near the end of the third and scheduled final project year (as of Summer 2008).  This evaluation report is organized around three evaluation questions developed as the foundation of the evaluation design. 

The three questions guiding the evaluation included the following: 

1.                  To what extent have the project’s stated priorities and goals been accomplished?

2.                  What project outcomes of interest to NSF (the funding agency) (e.g., course content and development, course alignment, curriculum changes, enrollments/graduates, etc.) have occurred across participating institutions?

3.                  To what extent are key project components likely to be sustained among project institutions after the life of the grant?

As indicated within the following sections of this report, the evaluation team utilized multiple sources of data to address these evaluation study questions.  Based on these multiple data sources, the evaluation team addressed each evaluation question to the extent that data are available at this stage of the project.  However, given the future implementation schedule of some of the newly created, revised, and articulated courses, it is not possible to address the ultimate impact of the project at this time since very few students have completed project courses and progressed to the four-year institutions to complete their teacher education program.   

II.   Data Collection Sources and Methods

            This report is based upon multiple sources of data that were used by the evaluation team to generate answers to the evaluation study questions.  These data sources included:

1.      Interviews with the project director and co-PIs

2.      Review of project-related products (e.g., workshop materials and documents from participation institutions)

3.      Observations of project meetings and workshops

4.      Campus site visit to each participating institution that included interviews and focus groups

The evaluation team will introduce data from each of these sources where appropriate for each evaluation question in the body of this report.  The following presents a brief introductory preview of data collection sources and related issues.

  1. Interviews with the Project Director and Co-Principal Investigators

The evaluation team members interviewed the project director and co-principal investigators on multiple occasions. These include three semi-structured interviews with the project director and one interview with each of the three co-principal investigators.

 

 

  1. Review of Project-Related Products

Campus project artifacts were made available to the evaluation team.  These included:  campus status reports, campus curriculum materials and documents, project annual reports, and course syllabi.

  1. Observations of Project Meetings and Workshop

While the evaluation team was added to the project after the first implementation year of the project (to replace a previous evaluator who left the project), evaluation team members were able to attend and observe two project workshops that were offered to faculty and staff from all participating institutions. 

  1. Campus Site Visits that included Interviews and Focus Groups

The evaluation team visited each of the two universities and six community colleges participating in the project.  These site visits occurred during December and January of the 2007-2008 academic year.  Each site visit provided the evaluation team an opportunity to: 1) observe project facilities and artifacts; and 2) interview project staff, project faculty, and senior academic administrators.  The evaluation team was also able to talk with a few students at several institutions and observe a project designed class.

III. Findings

TBR Teacher Preparation Partnership (TPP) Program Description

To establish a project context supporting the analysis of findings under each evaluation question, the evaluation team begins this section of the report with a brief review of the project purpose, priorities, key project elements, project years, participating institutions, and the overall evaluation approach.  These contextual elements provide relevant background information supporting the subsequent evaluation team discussion offered under each evaluation question.

Project Purpose:  Among the related purposes of the grant reflected in the stated grant proposal priorities, project staff indicated that the project ultimately seeks to: improve K-6 teacher education by redesigning mathematics and science classes to align with state K-6 content standards, establish instructional alignment and articulation across all participating eight universities and community colleges, and promote meaningful student learning through activity and experience- based instruction that integrates hands-on manipulative and materials

Project Priorities:  The project entailed five priorities.  These are: 1) Improve the mathematics, science, and technology preparation of future teachers through statewide collaboration and systemic change; 2) Improve articulation and advising in the field of teacher education among all institutions involved in this project by offering and requiring the same mathematics, science, and educational technology courses at each institution; 3) Develop student support systems; 4) Improve the capacity and training of a diverse teaching pool in Tennessee; and 5) Provide opportunities for both in-service and pre-service teachers to gain content credit hours to meet NCLB requirements.

Key Project Elements:  In meeting these project goals, several project elements were addressed:

·   Curriculum development (K-6 Teacher Preparation)

·   New and revised courses (mathematics, science, and education)

·   Pedagogical improvement

·   Curriculum alignment to state content standards

·   Curriculum articulation (community college to university)

·   Faculty collaboration

·   Faculty professional development

·   Student development resources including expanded or new campus student teachers education associations

·   Instructional materials and resources

·   Instructional settings (dedicated reserved classrooms and secure storage for instructional materials and equipment)

Project Years:  The TBR-TPP project is a grant initiative funded by the National Science Foundation.  The life of the project covered a three-year period as follows:

            Year 1              2005-2006 (September 1, 2005)

            Year 2              2006-2007

            Year 3              2007-2008 (ends August 31, 2008)

Participating Institutions: Eight Tennessee Board of Regents institutions participated in the project.

Universities

East Tennessee State University

Tennessee Technological University

Community Colleges

Chattanooga State Technical Community College

Motlow State Community College

Northeast State Technical Community College

Pellissippi State Technical Community College

Roane State Community College

Walters State Community College

The original project proposal was designed as a statewide project, but the lower level of federal funding eventually awarded required that project staff scale the project down in scope to two universities and six community colleges within the eastern and a portion of the middle regions of the state.  One of the participating institutions, Pellissippi State Technical Community College, served as the fiscal agent. 

Evaluation Approach:  After the original project evaluator left the project, an evaluation team from the Institute for Assessment and Evaluation (University of Tennessee) was engaged as the project evaluator. The Institute evaluation design was jointly developed by the project principal investigators and the evaluation team members.  In addition to data collection efforts that included interviews, meetings with staff, review of project materials, records, etc., the evaluation team also conducted site visits to each of the two universities and six community colleges engaged in the project.  The purposes of the evaluation team on these site visits were to:

·         Receive a project update from campus coordinator, faculty, and academic administrators regarding the accomplishments of the project since its inception.

·         Interview campus coordinator, faculty (group interview), campus academic administrators, and students (on some campuses students were available to talk with the valuation team).

·         Observe project facilities (e.g., reserved classrooms) and project-purchased instructional resources and materials. 

Two evaluation team members conducted campus visits November, 2007 through January of 2008.   During and after the evaluation team visits, campus project participants shared PowerPoint slides, reports of progress, course syllabi and a variety of other materials in support of their project efforts.  These materials were shared with project staff who added them to artifacts of the permanent project record. 

Brief Grant History: This TPP grant project expanded on the efforts of two institutions, Pellissippi State Technical Community College (PSTCC) and Tennessee Technological University (TTU) to enhance the articulation of two-year to four year institutions and promote the STEM preparation of elementary education graduates. Also occurring early the life of the grant, TTU received approval to offer a 2 + 2 degree option on community college campuses, and the common course structure developed through this 2 + 2 promoted the efforts of the current TPP project.  Moreover, the Tennessee Board of Regents approved the Associate of Science in Teaching degree for the community colleges in the project, required PRAXIS I, a minimum GPA of 2.75 and assessment of student dispositions.  While these developments did not result from this grant project, they provide important contextual elements for this project.

 


 

Evaluation Question One:  To what extent have the project’s stated priorities been accomplished?

Status of Progress on Project Priorities

            The NSF TBR Teacher Preparation Partnership (TBR TPP) established five project priorities. These include:

1.      Improve the mathematics, science, and technology preparation of future teachers through statewide collaboration and systemic change.

2.      Improve articulation and advising in the field of teacher education among all institutions involved in the project by offering and requiring the same mathematics, science and educational technology courses at each institution.

3.      Develop student support systems. 

4.      Improve the capacity and training of a diverse teaching pool in Tennessee.

5.      Provide opportunities for both in-service and pre-service teachers to gain content credit hours to meet NCLB requirements. 

            There is a substantial body of evidence addressed in this report indicating that project staff and participating institutions have made substantial progress toward the achievement of these project priorities.  Progress on each priority at the time this evaluation report was being prepared (June 2008) is as follows.

Project Priority #1:  Improve the mathematics, science, and technology preparation of future teachers through statewide collaboration and systemic change.

This project has focused exclusively on the improvement of the mathematics, science, and technology preparation of future teachers.  Several key elements of the project influenced the achievement of this priority: 

·         STEM content course creation and re-design of existing courses

·         Enhancements in math and science content faculty pedagogy

·         Infusion of new technology and instructional materials into the teacher education program that promote experiential learning.

·         Linking of mathematics and science content and instruction to state and national K-6 science, math, and technology content standards, as well as state performance indicators

·         Statewide faculty collaborations structured to promote interactions and sharing

·         Faculty professional development

 

Figure 1 presents a more complete list of the key activities of the project.

 

 

 

 

Figure 1.  Key Project Elements

Collaboration

·         Project-wide campus content faculty and project staff collaboration

·         Conferences and presentations promoting faculty development

·         2  year – 4 year teacher education program articulation

Curriculum Re-design

·         Standards based content

·         Common math/science content for K-6 majors through new or re-designed:

o   math classes (6 to 9 hours)

o   12 hours of science under two options

-    (3)  4 hour courses option

-    (4)  3 hour courses option

·         An education course incorporating technology content

·         Enactment of the Associate in Science in Teaching degree by the Tennessee Board of Regents

Pedagogy

·         Infusion of active/experiential learning promoting student engagement in teacher education

·         Purchase/Implementation of hands-on instructional materials and manipulatives

Faculty Support

·         Purchase of materials for classrooms ($10,000 per year for each institution)

·         Support for faculty to attend professional development ($5,000 per institution)

·         Faculty release time or stipends/support $10,500 - 12,000 per school per year

·         Two workshops for all faculty at PSTCC

·         Collaboration with colleagues (within/between institutions)

Student Support

·         Student Teacher Education Association

·         Student support, especially for testing preparation related to PRAXIS

·         Designated education centers/classroom and storage space

 

 

 

 

 

The campus project status reports produced by each of the participating institutions provide substantial detail regarding the specific project elements implemented on campus to enhance teacher preparation.  Given that these descriptive reports are part of the project record, it is not necessary for the evaluation team to restate them again for this evaluation. However, this evaluation report will draw upon these institutional reports (and the other data sources previously identified under Section II) to highlight grant accomplishments and the status of progress on each priority. 

Improved Mathematics, Science, and Technology Preparation for Future Teachers:  Since some of the project courses are still being implemented for the first time this year, it is not possible to determine the overall effectiveness of the project based on a substantial number of students who have completed the full complement of these newly enhanced two-year and four-year elementary education programs. However, the project record clearly demonstrates that all participating institutions have enhanced their elementary education programs through new or revised content and teacher education courses consistent with K-6 content standards, and the institutional project reports and interviews demonstrated that participating faculty have introduced new instructional materials and pedagogical practices designed to engage teacher education students in more experiential learning activities.  Additionally, there is substantial evidence that the Partnership element of this project has helped insure and broaden the role and involvement of community college and university content faculty in teacher preparation. It is also important to note that project elements are consistent with the educational best practices that are currently receiving attention in the teacher education literature.  Course content has been aligned to state standards, and university content faculty members have expanded their pedagogical strategies to model experiential learning.

Statewide Collaboration:  One of the most important aspects of the TPP project according to campus participants was the two-year and four-year faculty collaboration model introduced by the project. Community college and university faculty reported that the structured, yet flexible, collaboration opportunities provided twice each year for participating mathematics and science content faculty was the most productive, rewarding, and effective aspect of the grant.  Through the Partnership settings offered by the project, content faculty were provided with the opportunity to work on course content and pedagogical innovation with both university and community college peers, all of whom had a common interest in  promoting standards-based teacher education (K-6).  These collaborations continued informally throughout the year, periodically reinforced and supported by the formal faculty meetings provided by the project two times during the year.  Faculty reported substantial benefits from Partnership experiences with their content peers, especially regarding the sharing of pedagogical strategies that had proven effective at other campuses.  While many curriculum projects are reported to be Partnership, this project was truly designed and implemented through a Partnership model empowering content faculty. 

Systemic Change:  The design of this project, and the Partnership model it is built upon, was meant to establish a fundamental and sustainable model for systemic change in teacher preparation.  This project has demonstrated, and is expected to continue to demonstrate, that teacher preparation is most effective when content area and teacher education faculty members jointly take the lead in developing course content and incorporating research-based experiential, hands-on pedagogical strategies that link directly to state content standards.  It is important to note that this approach to teacher education represents a significant departure from previous practice.  Content faculty collaboration, especially between university and community college faculty, has typically not occurred among these institutions in the past with the intensity and frequency generated by this project.  Moreover, the project focus on empowering content faculty to use state content standards for the design of course content was also essentially introduced by this project.  For example, many content faculty members reported that were either not aware or only somewhat vaguely familiar with the details of the state K-6 mathematics and science content standards prior to the project. 

 

Project Priority #2:  Improve articulation and advising in the field of teacher education among all institutions involved in this project by offering and requiring the same mathematics, science, and educational technology courses at each institution.

Formal articulation between two-year and four-year higher education programs is not something new in Tennessee.  Project coordinators and campus project staff reported that all participating higher education institutions had previously established multiple program articulation agreements for a range of programs between community colleges and universities for students seeking a four-year degree upon completion of their associate degree.  While many of these community college - university articulated transfer programs included state institutions within the two Tennessee higher education systems (the Tennessee Board of Regents and the University of Tennessee), these program articulation agreements also included private four-year universities.  However, this project appears to have established a new model for articulation, especially from the perspective of the active, Partnership role of content faculty in the design and teaching of mathematics and science content courses for teacher education students in line with state education content standards. 

Moreover, campus project coordinators and faculty reported that the articulation and advisory function has been greatly enhanced through this project from the perspective of both community colleges and universities.  From a formal perspective, the common, shared development of courses among the institutions, resulting in similar standards-based content has greatly promoted articulation.  Not only are these articulation agreements in place, but there is shared agreement on the specific content to be covered (e.g., linked to the Tennessee elementary education content standards). University campus coordinators report that the teacher education students who complete the project curriculum (as embedded in the new AST degree) are much easier to advise and integrate into the university elementary education program.  University teacher education programs enrolling community college elementary education transfer students now know the specific science and mathematics skills and competencies of these entering students. In these instances, universities reported that advising for these elementary education transfer students is straightforward. 

 

Project Priority #3:  Develop student support systems

The primary mechanism project staff used to promote and enhance teacher education student support systems was through the student teacher education association (STEA) chapters. Most of the participating institutions already had existing STEAs, but some were more active than others. Under the auspices of the STEAs, institutions have expanded and strengthened the academic and professional support to K-6 teacher education students.  Examples of some of these student support activities include: 

·         Implementation of student services supporting education majors

·         PRAXIS preparation

·         Involvement of teacher education students in public school classrooms

·      Education career days for teacher education students

·      Joint STEA membership (university transfers retain membership in their previous  community college STEA organization)

·         Sponsorship of future teacher conferences

·         Development of a website of interest to future teachers 

·         STEA sponsored community service project opportunities

·         STEA sponsored school programs ranging from book drives to grade-specific initiatives

·         Sponsoring of the Leadership Conferences for student teachers

 

Project Priority #4:  Improve the capacity and training of a diverse teaching pool in Tennessee.

The project record offers substantial evidence of favorable changes in the design of STEM course content and pedagogy of the elementary education teacher education programs of participating institutions, changes which are expected to have an impact on future elementary school teachers.  However, campus coordinators and project staff need to collect and analyze additional data on project course enrollments, majors, graduates, licensures, teaching employment, etc. (along with associated student demographics) to be able to determine the project’s influence on the size, diversity, and effectiveness of the teaching pool in eastern and central Tennessee.  Expanding student diversity will be a challenge for many of these programs, as the majority of these institutions are located in rural areas where little ethnic diversity exists.  However, Chattanooga State is located near a larger, diverse urban population, and Pellissippi State established its teacher preparation courses at a satellite campus in an inner city neighborhood to promote access to the program for minority community residents. This project priority needs further consideration, and the effort required may be challenging.

 

Project Priority #5:  Provide opportunities for both in-service and pre-service teachers to gain content credit hours to meet NCLB requirements.

The primary focus and effort of this grant was on pre-service teachers, especially from the perspective of content credit hours/instruction that meet NCLB requirements. Campus project administrators and faculty reported some limited in-service teacher involvement with the grant, but this priority essentially focused on pre-service teachers’ content knowledge in light of NCLB requirements.

Conclusions and Recommendations

Based on the data presented in response to this evaluation question, the evaluation team offers the following conclusions and considerations for the future:

Conclusions

  1. TPP project staff, campus level project coordinators, and faculty addressed the project priorities.  There is substantial evidence that meaningful progress was achieved in establishing systemic change in community college and university collaboration in the preparation of K-6 teacher education students. 
  2. This project successfully established and implemented a unique standards-based content faculty collaboration model that included the active engagement of two-year and four-year teacher education and content faculty.
  3. While two-year to four-year program articulation has a long history in Tennessee – the process for community college - university articulation has been substantially strengthened for K-6 teacher education.
  4. This project resulted in a fully articulated standards-based science, math, and technology curriculum for elementary education majors at the eight participating campuses. 

Recommendations

1.      Project staff should promote elementary education k-6 data collection related to this project after the grant has ended.  The data necessary to provide a critical mass of definitive data regarding the long-term effectiveness of this project will not be available until more students have completed the program. 

2.      Institutions need to address the challenges of providing project data on the diversity of the teaching pool.  Diversity of the “teacher pool” in Tennessee was a key aspect of the grant, and analysis of the effects of the grant is incomplete without diversity data related to elementary education teacher enrollments, graduates, transfers, etc.,  at the project institutions

Evaluation Question Two:  What key project outcomes of interest to NSF (funding agency) (e.g., course content and development, course alignment, curriculum changes, enrollments/graduates curriculum assessment etc.) have resulted within and across participating institutions?

  

In response to this evaluation question, the evaluation team addressed five aspects of this project. These include 1) the Partnership element of the curriculum development process, 2) the curriculum design/changes, 3) curriculum implementation, 4) curriculum assessment, and 5) faculty perceptions regarding the project.

 

Partnership Curriculum Development Process

            The project’s process for content and teacher education faculty to partnership in developing a fully articulated curriculum deserves significant consideration as a best practice.  It is rare to encounter scientists who are university or community college faculty members with an in-depth knowledge of teacher education state standards and pedagogical strategies that are reflective of the current research-based literature.  Each campus reported broad involvement of campus faculty.  While institutional reports identify the faculty involved with the project, the evaluation team discussions with campus faculty suggest that campus figures are undercounting. For example, careful reading of campus reports suggested that new faculty have been added to the project across the years. Faculty reported, during campus interviews with evaluation team members, that major elements of the project have been shared with university and community college colleagues who were not directly involved with the project. One of the activities that project institutions should undertake before the grant ends is to establish a methodology for accurately reflecting the number of faculty that have been impacted by the project, and that number may be surprisingly larger than expected.

 

Curriculum Changes

Table 2 identifies the mathematics, science, and teacher education content courses that were established or enhanced as part of this project. The curriculum changes for each course have been identified within the separate individual institutional project reports. Project staff, campus coordinators, and project faculty confirmed the nature and depth of the curriculum changes that are reflected with newly created or revised courses. Elements of the curriculum included: new content, new pedagogical practices, the introduction of hands-on and experiential learning activities, new texts, new course syllabi, and new course sequencing options in the sciences. For the sciences, some institutions used 3 four-credit hour courses or 4 three-credit hour courses to achieve the science content need to address the state standards. Institutions also indicated the helpful role of the Associate Science Teaching degree that was not a part of the grant but provided a beneficial structure in which to embed the curriculum in an articulated program structure.  The student reactions to the new or revised courses have been very positive, and some campuses reported that students who are not teacher education majors are attracted to and enrolling in them.  It should also be recognized that in addition to the creating and revising of courses shown in Table 2, at the beginning of this project the aforementioned courses were offered at only nine sites across the project. As of Spring Semester 2008, these courses had been offered at 23 sites associated with the eight participating institutions.

In addition to changes in courses for the teacher preparation program, content faculty also frequently reported making changes in other courses as a result of the successes they observed in the teacher education courses. 

 


 

Table 2. Institutional Requirements TPP (K-6)

 

 

Math

Science

Education

ETSU

Math 1410

Math 1420

Math 1530

Physics

Chemistry

Biology

Earth Science

 

SCED 4321

TN Tech

Math 1410

Math 1420

New K-6 Math course introduced this year

Biology 1310

Chemistry 1310

Physics 1310

Geology 1310

FOED 2011

CSTCC

Math 1410

Math 1420

 

Physics 1310

Chemistry 1310

Psci.  1310

Biology 1310

ED 201

ED Psy 207

MSCC

Math 1410

Math 1420

 

Biology 1030

Geol 1030

PSCI 1030

EDU 1120

NSTCC

Math 1410

Math 1420

Math 1530

Biology 1030

PSCI 1010

PSCI 1020

EDUC 2300

PSTCC

Math 1410

Math 1420

 Math 1010 or 1530

Biology 1310

Chemistry 1310

Physics 1310

Geology 1310

EDUC 2010

RSCC

Math 1410

Math 1420

Math 1530

BIOL 1110

GEOL 1410

MSC 1012 (Physics/Chemistry)

EDUC 101

WSCC

Math 1410

Math 1420

Biol 1030/31

Chem 1020/21

PSCI 1030/31

EDUC 2300

           

 


 

Revised Sequencing

The mathematics and education courses included in this project were in existence prior to the grant. While some existing science courses were revised through this project, several new science courses (and course sequences) were also introduced as a result of this grant.  Institutions used two strategies to address the science content courses:  three (four hour) courses or four (three hour) courses.  Regardless of the number of courses, the underlying design criteria were to ensure that students received the science content knowledge required of them as elementary education teachers in Tennessee.  Table 3 reflects the courses reported to be covered under this project and their implementation schedule (where appropriate).

Table 3.  Course Creation Sequencing

 

Institution

Course

Revised

Created

First Offered

ETSU 

Math 1410

Math 1420

Math 1530

Biology

Physics

Chemistry

Earth Science

SCED 4321

 

X

X

X

X

X

X

X

X

-

-

-

-

-

-

-

-

NA

NA

NA

NA

NA

NA

NA

NA

TTU 

Math 1410

Math 1420

Bio 1310

Chem 1310

Geol 1310

Phys 1310

 

X

X

X

X

X

X

-

-

-

-

-

-

NA

NA

NA

NA

NA

NA

CSTCC

Math 1410

Math 1420

ED 201

ED PY 207

Phy 1310

Chem 1310

PSCI 1310

Biology 1310

X

X

X

-

-

-

-

-

 

-

-

-

X

X

X

X

X

NA

NA

NA

TBD

TBD

TBD

TBD

TBD

 

Note.  Course revisions in existing courses, and even in newly created courses, occur over time.  It is not always possible to pinpoint a specific semester when courses were revised due to the ongoing nature of the project and evaluative reflection on the changes as they are introduced.
 

Institution

Course

Revised

Created

First Offered

MSCC

Math 1410

Math 1420

PSCI 1030

Biology 1030

Geol 1030

Educ 1120

X

X

X

X

X

X

 

-

-

-

-

-

-

 

NA

NA

NA

NA

NA

NA

 

NSTCC 

Math 1410

Math 1420

Math 1530

Biology 1030

PSCI 1010

PSCI 1020

EDUC 2300

X

X

X

-

-

-

X

-

-

-

X

X

X
-

NA

NA

NA

Fall 2008

Fall 2008

Fall 2008

NA

PSTCC 

Math 1410

Math 1420

Biology 1310

Chem 1310

Geol 1310

Phys 131-

EDUC 2010

X

X

X

X

X

X

X

-

-

-

-

-

-

-

NA

NA

NA

NA

NA

NA

NA

-RSCC 

Math 1410

Math 1420

Math 1530

Biology 1110

Geol 1410

MSC 1012

EDUC 101

X

X

X

 

X

X

X

-

-

-

X

-

-

-

NA

NA

NA

Spring 2008

NA

NA

NA

WSCC 

Math 1410

Math 1420

Biology 1030/31

Chem 1020/21

PSCI 1030/31

EDUC 2300

X

X

-

-

X

-

-

X

X

X

-

NA

NA

Fall 2007

Spring 2008

Fall 2008

NA

 

 


 

Curriculum Participation - Enrollment, Majors, Graduates

Campus project coordinators were asked to provide data on project course enrollments, majors, and graduates. However, the lack of a common template and timeframe for reporting student enrollment, majors and graduates proved to be problematic.  The primary focus of this grant was elementary education; but in some cases enrollment, majors, and/or graduates were reported in a manner that did not provide for the disaggregation of data for elementary education students.  Institutional challenges also presented problems for the reliability of project data.  In some cases, campuses expressed various reservations about the count of elementary education majors, since campus information systems and processes do not enable accurate counts of majors – in such cases counts of majors are simply estimates. 

These data are reported in the campus project reports that have been filed with the project coordinators.  Also, campuses used different time periods for the reporting of course enrollment, majors and graduates.  In some cases just one year (i.e., the current year) of data were reported.  The evaluation team has attempted to re-construct a table of course enrollments, but the reader is required to review campus reports.  A common template and time frame is required.  From the perspective of the evaluation team, the following comments can be offered within the limitation of the different approaches project campuses used in reporting. 

Course Enrollments

            Based on the campus project reports, the evaluation team sought to develop a table that depicted course enrollment in project courses (Table 4).  However, several challenges were encountered:

·         Reported enrollments sometimes included all education majors.

·         Several new/revised courses had not yet been offered.

·         Baseline years were not consistent across institutions.

·         Course enrollments are not always limited to teacher education statewide.

·         This three-year project has focused on curriculum development, and much has occurred in the three years focused on curriculum design and implementation – enrollment changes due to the grant can only be determined upon implementation of all courses. 

·         In many cases, the campuses created new science courses, and these courses were being offered for the first time after the last year of the project.  These courses offer no enrollment history, obviously, so it is too early to determine project impact on enrollment. 

Table 4 represents the current data regarding enrollment in project courses provided to the evaluators through campus reports.  Campus representatives would have been in a better position to provide these data in a common framework had a framework been provided. The senior project principal investigator will be sharing a common framework for the reporting of project data, and these data will be reflected in the final evaluation report.
Table 4. Enrollment in Project Courses Reflected In Institutional Project Reports

 

ETSU 

·    Specific course-by course enrollment data not included in the campus report, but large numbers of graduates suggests high program course enrollment.

TTU

·    Actual headcount enrollment data reported    as follows:

§          2005-2006=1071

§          2006-2007=1011

§          2007-2008=1085

CSTCC (Education and math courses only)

·          Fall 2005 = 154

·          Fall 2006 = 169

·          Fall 2007 = 213

MSCC

·    Reported an increase from 236 education students before the grant to 374 students by Fall 2007.  (The grant began during the 2003-2004 school year).

NSTCC  

·    Enrollment data were available only for the math courses since the science courses would be offered for the first time after the grant is completed.  Enrollment in these math courses reported: 2004-05=97, 2005-06=114, 2006-07=32.

 

PSTCC

·    Fall 2005 = 251; Fall 2006 = 223; Fall 2007 = 218

·    This institution provided course by course detailed enrollment on the main campus and one off-campus center. 

·    Demographics data (including ethnicity/race, gender, age, etc.) were provided, but these data referenced all education majors, which included elementary education majors as well as all other education majors.  This type of data provided is helpful, but of interest for this grant project are the elementary education majors.

RSCC

·    Detailed course enrollment data provided for 1410, 1420, and Educ 101.

WSCC

·    Before the grant, grant course enrollment in courses enhanced by the grant was 167.  By the third year of the grant, course enrollment was 204 in these courses.  Enrollment in education courses by the grant is expected to increase because of changes inspired by the grant.

 

 

 

Majors

The data reported by institutions on majors was inconsistent and problematic. More than one two-year institution reported that counts of majors at their institution were not accurate – the process for recording and identifying and updating students major codes did not reflect the “ true” count of majors in institutional programs. In other cases, headcounts of majors were reported, but they were not disaggregated to include only elementary education, while in other cases there was more than one listing of majors for elementary education. Another challenge is related to the newness of the Associate of Science Teaching program. The issue of the accuracy of major headcounts and their role in the project needs to be reconsidered. If, for whatever reason, the headcount of majors does not accurately reflect the true numbers of majors, reporting and tracking the numbers of majors may not be helpful. However, it may be time that institutions re-examine this issue as some of the participating institutions appear to have no problem reporting what they claim are accurate major counts.

 

 

 

Graduates

Data project institutions provided on graduates were more definitive (Table 5), but it is too early to expect changes in the numbers of graduates due to this project. 

 

Table 5 Institutional Graduates – Elementary Education

 

 

ETSU

 

2004-05       72         (K8)

2005-06       89         (K6)

2006-07       61         (K6)

 

TTU 

 

Spring 2006 =88

Fall 2006 =8

Spring 2007 =102

Fall 2007 =5

Spring 2008 =159 (projected)

 

CSCC 

 

2004-05 =16

2005-06 =29

2006-07 = 22

 

MSCC   (Data provided as ranges)

 

2004-05 =<50

2005-06 =<50

2006-07 =>70

 

 

NSTCC 

 

2004-05 =29

2005-06 =34

2006-07 =15

 

PSTCC  

         

 

RSCC 

2004 =80

2005 =65

2006 =65

2007 =78

 

WSCC 

 

2004-05 =19

2005-06 =19

2006-07 =24

 

 

Student Demographics

Some institutions participating in this grant do not have readily available demographic data on students from the perspective of majors, graduates, and transfers.  Given the central aspect of the grant related to diversity, these data should be collected, especially regarding race and gender.  Project staff should consider computing and reporting demographic data for their final project report.

Curriculum Assessment

Curriculum assessment is an area where this project has not had a major focus.  While common content, linked to state standards has been a focus, much needs to be done in the assessment of student achievement.  In one sense, the assessment process will be greatly promoted by the existence of specific content standards.  Campus coordinators and faculty reported the existence of course student assessments, and these are identified on course syllabi.  However, there was less emphasis on assessment across the curriculum.  Additionally, important outcomes measures will occur in some cases after the elementary education students have transferred to a four-year institution.  A systematic feedback process is essential for reporting PRAXIS scores and other pertinent data (e.g., academic success in junior and senior education courses) back to the students’ original community colleges. 

Student outcomes data for students involved and project courses are being collected as part of the multiple assessments associated with each course.  Review of course syllabi suggests that student assessments are frequently conducted and they cover a wide range of assessment strategies.  However, there are many opportunities to use these assessment results in a more informative manner.  For example, at one institution student grade distributions are compared for project courses across each semester.  There are other comparisons that may also prove to be helpful, such as comparison of student outcomes for teacher education students sections to students in other sections that are not restricted to teacher education students.  Moreover, there is an opportunity to track student achievement of competencies across courses.

 

Faculty and Student Perspectives

The evaluation team conducted site visits to each of the eight participating institutions.  During the site visits, evaluation team members conducted formal individual and group interviews that included the site coordinator, faculty, academic administrators, and in several cases project students. 

Campus Personnel Perceptions

Faculty Collaboration At each campus evaluation team visit, faculty offered comments about the value of the project from the perspective of inter-institutional faculty collaboration.  Faculty members across all project institutions were united in their assessment that one of the most fulfilling aspects of the project was the regularly scheduled meeting (twice each year) of faculty from each participating institution.  These meetings provided the opportunity for faculty from the same content areas to share content materials, pedagogical techniques, and curriculum development best practices.  Many faculty members indicated that these collaborations continued after the meetings through informal mechanisms such as emails and phone calls.  Moreover, several of the project faculty members noted that they had other opportunities to interact with other faculty members at state conferences.  However, the structure of the project meetings, formal group sessions and informal faculty content sessions, along with the purposes they shared through the project, provided an ideal environment for content faculty collaboration across institutions.  Among the many project benefits to this collaboration, faculty members noted how helpful it was for two-year and four-year faculty to gain a better understanding of each other. 

Awareness of State Content Standards Several community college content faculty members indicated that they first became aware of state elementary education content standards as a result of their participation in the project.  They reported that while they have been teaching future educators in their mathematics and science classes, they were not necessarily sensitive to the content that these future teachers would be addressing.  Knowledge of these standards helped them focus their content in a manner that would be supportive of future teachers. 

Sharing Beyond Project Faculty:  Several faculty reported that other content faculty members on their campus had expressed interest in the project and curiosity about the pedagogical aspects being introduced.  Project materials, strategies and pedagogical techniques have been shared with non-project faculty, but the extent of this sharing and the depth of the adoption and use of materials and strategies are beyond the scope of this evaluation. 

Curriculum Materials At almost all of the campus site visits conducted by the evaluation team, campus project coordinators and/or faculty made it a point to display the learning materials and resources that were purchased through the grant and currently used in project courses.

Education Center As a part of the grant, each participating institution was required to establish a room exclusively to support the project as a dedicated classroom. Campus site visits were usually conducted in the designated room for campus education center, but in a few cases these rooms were being used for project classes.

Pedagogical Enhancement Content faculty members at the community colleges also reported that their project experiences introduced them to a broader range of pedagogical strategies, especially strategies related to experiential and inquiry-based learning models.  While these strategies are quite common in the field of education, they are often not included in the educational training of mathematics and science content faculty members.  The impact of exposure to these newly introduced techniques was substantial in the cases of several faculty members.  One faculty reported when discussing how the project influenced his teaching, the inquiry-based and experimental pedagogical techniques he learned from the project have been so effective in his teacher education courses that he has integrated them into all of his science classes. 

Student Perceptions

            Faculty were helpful to project evaluations in reporting an initial sense of how students were responding to the changes in these teacher education courses.  Most faculty noted that students responded well to the hands-on activity-based format of the classes.  The few students who the evaluation team was able to talk with reflected positively on their classroom experiences, but it is not known to what extent these students reflected the perceptions of all students.  Faculty members at one institution suggested that the pedagogical strategies resulting from the grant work well with some students, but not all students are comfortable initially with this style of learning. Some students may not have been exposed to hands-on and group instructional techniques, and for some of these students being engaged during class may move them out of their comfort zone. This was reflected in student evaluations of instruction for project courses.

Conclusions and Recommendations

Based on the data presented in response to this evaluation question, the evaluation team offers the following conclusions and recommendations:

Conclusions

  1. Due to the Partnership nature of this project, the numbers of faculty members who participated in the project are substantial. Moreover, review of institutional project reports and interviews with participants are laudatory of the collaboration fostered by the project. 
  2. All participating institutions have made changes in their courses and curricula, although a few changes, new courses in science, have not yet been implemented.
  3. It is still too early to determine the impact of this project on the following elements of participating elementary education programs: 

-    enrollment

-    majors

-    graduates

-    diversity of the teaching pool

However, the necessary data will be available in the future, after the grant has been completed.

  1. The project is well received by faculty and students, and it is impacting courses and faculty instruction beyond the scope of the project (elementary education coursework in mathematics, science, and education). 

 Recommendations

  1. Project institutions should develop and implement a methodology that accurately reflects the number of faculty who have been materially affected by the project. The numbers of faculty participants provided in the institutional reports appears to undercount faculty involvement in the project.

Question Three:  To what extent will key project components likely be sustained among project institutions after the grant period?

As identified in several sections of this report, several project elements appear to have the potential for long-term sustainability from the perspective of project participants.  These included:

·      Course Requirements:  Course requirements in science, mathematics, and education for elementary education have been revised, are in print, and are jointly recognized and respected by both the community colleges and the universities.  New courses have been approved and are (or soon will be) taught.

·University and community college content faculty collaboration: This is the element of the grant that most grant participants, and especially the faculty, would prefer to see continued. 

·Standards-based STEM content courses for teacher education students: The standards-based content for the elementary education has a high probability to be sustained beyond the life of the grant. 

·Constructivist pedagogy: Participating faculty reported the benefit of their experiences with the project related to new pedagogical strategies. 

The evaluation team also suggests that this program may serve as a model for inter-institutional collaboration that may be unique and worthy of adoption across the state (as originally intended in the grant proposal but scaled back due to financial limitations) and at other institutions across the country. 

Conclusions and Recommendations

Based on the data presented in response to this evaluation question, the evaluation team offers the following conclusions and recommendations:

Conclusions

1.      Several aspects of this project have a strong potential to be sustained (course content, faculty collaboration, pedagogical enhancements, etc.), including the curriculum change model introduced with this project.

 

Recommendations

  1. Project staff should encourage institutions to promote sustainability of project elements, and they should share the project model, especially the Partnership nature of the project, with other Tennessee Board of Regents Institutions.

 

 

IV. Summary of Evaluation Conclusions and Recommendations)

Conclusions - Evaluation Question 1 (Achievement of Project Priorities)

  1. TPP project staff, campus level project coordinators, and faculty addressed the project priorities.  There is substantial evidence that meaningful progress was achieved in establishing systemic change in community college and university collaboration in the preparation of K-6 teacher education students. 
  2. This project successfully established and implemented a unique standards-based content faculty collaboration model that included the active engagement of two-year and four-year teacher education and content faculty.
  3. While two-year to four-year program articulation has a long history in Tennessee – the process for community college - university articulation has been substantially strengthened for K-6 teacher education.
  4. This project resulted in a fully articulated standards-based science, math, and technology curriculum for elementary education majors at the eight participating campuses. 

 

Conclusions - Evaluation Question 2   (Project Outcomes)

  1. Due to the Partnership nature of this project, the numbers of faculty members who participated in the project are substantial. Moreover, review of institutional project reports and interviews with participants are laudatory of the collaboration fostered by the project. 
  2. All participating institutions have made changes in their courses and curricula, although a few changes, new courses in science, have not yet been implemented.
  3. It is still too early to determine the impact of this project on the following elements of participating elementary education programs: 

-    enrollment

-    majors

-    graduates

-    diversity of the teaching pool

However, the necessary data will be available in the future, after the grant has been completed.

  1. The project is well received by faculty and students, and it is impacting courses and faculty instruction beyond the scope of the project (elementary education coursework in mathematics, science, and education). 

 

 

Conclusions – Evaluation Question 3 (Sustainability)

1.      Several aspects of this project have a strong potential to be sustained (course content, faculty collaboration, pedagogical enhancements, etc.), including the curriculum change model introduced with this project.

 

Recommendations

The evaluation team offers two types of recommendations: overarching recommendations and recommendations resulting from the analysis of data related to each specific evaluation question.

Overarching Recommendations

Based upon the data collected and analyzed for all aspects of the evaluation for this project, the evaluation team offers recommendations for promoting the sustainability of this project as well as the potential enhancement of future projects involving project staff or NSF (funding agency). 

Overall, three themes emerged from this evaluation that suggest a potential framework for enhancing project sustainability and the long-term monitoring of project impacts. These include:   1) the ongoing collection of student achievement data for continuous program improvement; 2) the collection of specific student data (enrollment, retention, graduation), including demographics on diversity, for monitoring program productivity and its effects on the diversity of the teacher pool; and 3) the sustaining of ongoing opportunities for teacher education and content faculty collaboration and interaction across institutions on both informal and formal bases.  The evaluation team addresses each of these themes below.

 

2.      Collection of Teacher Education Student Achievement Data

 

Consistent with the grant proposal, the focus of this project was on teacher education content course curriculum design, development, and implementation; a substantial level of project activity and inter-institutional effort and time was necessarily focused on the Partnership development of new/revised content and pedagogical strategies. Accordingly, several courses developed or re-designed through this project were implemented later in the project or are scheduled for initial implementation after the scheduled three year life of the grant project has ended.  Given the time required for Partnership curriculum design and implementation in a three-year grant schedule, the opportunity to collect more than initial outputs and outcomes data is limited. More definitive data will not be available until all courses affected by the project have been implemented and a critical mass of students have enrolled and completed the whole program sequence, attained their teaching license, and entered the teaching workforce.

 

However, participating institutions have a unique opportunity to ensure that critical student achievement outcome data related to this project are collected and integrated into future ongoing institutional effectiveness assessments. These data would assist in the monitoring of elementary education courses and the overall effectiveness of elementary education programs of these institutions.  The collection and analysis of these data would be consistent with Tennessee Board of Regents and accreditation agencies expectations for ongoing program evaluation and the subsequent use of evaluation results for continuous improvements.  As such, consideration should be given to the following assessment elements related to the elementary education students and programs addressed by this grant project:

 

·         Offering of end of courses assessments to ensure desired teacher education standard “competencies” have been met 

·         Development of common end-of-course assessment instruments for these teacher preparation courses within institutions (This presented an interesting situation because the State of Tennessee revised the state K-12 mathematics and science standards this year to align them with NAEP. This new information is scheduled to be addressed in the August project workshop which is specifically aimed at the standards as well as proposed workshops occurring throughout the no cost extension of the grant.

·         Sharing of teacher education student PRAXIS I and II assessment results with each student’s originating community college (it should be noted that some community colleges were not aware of how to retrieve their PRAXIS I data)

·         Sharing of assessment results among participating community colleges and universities

 A plan for collecting, summarizing, and disseminating these results among participating two-year and four-year institutions would require substantial thought and discussion among faculty and academic administrators. There will be some technical and practical problems in collecting these data that would need to be addressed. To assist this process, project institutions may find assistance from their campus institutional research staff to be of significant benefit.

2.  Program Outcomes Data (including student counts/demographics related to course and program enrollment, retention, graduation, licensure, and employment) at the elementary education program level.

The elementary education programs of the two-year and four-year institutions participating in this project have the potential to serve a vital need for more, better-trained (including training in STEM content areas), and highly qualified teachers across east and middle Tennessee as well as enhance the diversity of the teacher labor pool as envisioned in the grant project. However, it is still too early to determine the effects of the project on addressing the teacher shortage and the diversity of the teacher pool in eastern and central Tennessee. Over time, as all project courses are implemented and more students enter the program, successfully complete it, and become licensed teachers, it will be possible to determine the long-term project impact. However, this outcomes assessment requires institutional assessment data that are disaggregated to the level of the elementary education programs that were the focus of this grant. Some of the initial campus project data reporting has been at an aggregate level (i.e., all education majors), and this aggregated data does not inform impact assessment of this project. 

3. Sustain Formal/informal Opportunities for Content Faculty Collaboration. 

The following evaluation report will offer evidence that one major benefit or effect of this project was the opportunity it provided faculty to interact with their colleagues from other institutions, especially colleagues who work at a different institutional level (e.g., two-year or four-year).  Beyond the collaborations sponsored by this project, campus content faculty reported having limited opportunities to meet with these colleagues at other state/regional meetings. Usually typical (non-grant) collaboration opportunities were not structured to provide focused opportunities for inter-institutional faculty engagement. One of the strengths of the faculty collaboration provided by this grant was that faculty had the opportunity to work together for a common purpose (i.e., collaboration of content faculty within/between institutions focused on standards-based teacher education). It should also be noted, as reflected in the interim report, that there has been considerable visitation among project faculty within the past year. These collaboration opportunities, and the many benefits they offer faculty to work together to improve course and program articulation, need to be sustained if at all possible. 

Recommendations Per Evaluation Question

Throughout the following report, the evaluation team has developed specific recommendations resulting from the analysis of data collected for each evaluation question.  The following is a summary listing of all of recommendations offered for each evaluation question:

Recommendations - Evaluation Question 1 (Achievement of Project Priorities)

 

1.      Project staff should promote elementary education K-6 data collection related to this project after the grant has ended.  The data necessary to provide a critical mass of definitive information regarding the long-term effectiveness of this project will not be available until more students have completed the program. 

2.      Institutions need to address the challenges of providing project data on the diversity of the teaching pool.  Diversity of the “teacher pool” in Tennessee was a key aspect of the grant, and analysis of the effects of the grant is incomplete without diversity data related to elementary education teacher enrollments, graduates, transfers, etc.,  at the project institutions.

Recommendation - Evaluation Question 2 (Project Outcomes)

 

  1. Project institutions should develop and implement a methodology that accurately reflects the number of faculty members who have been affected by the project. The numbers of faculty participants provided in the institutional reports appears to undercount faculty involvement in the project.

Recommendation - Evaluation Question 3 (Sustainability)

  1. Project staff should encourage institutions to promote the sustainability of project elements, and they should share the project model, especially the Partnership nature of the project, with other Tennessee Board of Regents institutions.