Improvement Dimension Report
Pellissippi State Technical Community College
Foundations Institutions conduct assessment and maintain associations with other institutions and relevant professional organizations in order to effect improvement. Assessment provides feedback to new students to guide their learning, to faculty to guide their teaching, and to the institution to guide planning, resource allocation, decision making, and improvement of programs and policies. As a way to facilitate improvement, these institutions are knowledgeable about current practices at other institutions as well as relevant research and scholarship.
Committee Leader:
Linda Manning, Associate Professor - Business and Computer Technologies
Committee Members:
Dennis Adams, Dean of Instructional Programs
Becky Blackwell, Assistant Professor - Mathematics
Teresa Fulcher, Associate Professor - Natural and Behavioral Sciences
K. D. Lawson, Assistant Professor - Engineering and Media Technologies
Rick Oster, Associate Professor - Business and Computer Technologies
Rick Patton, Assistant Professor - English
Linda Randolph, Director of Educational Improvement
Jane Schumann, Associate Professor - Natural and Behavioral Sciences
Anne Swartzlander, Professor - Business and Computer Technology
JoAnne Thomasson, Director of Developmental Studies Program
Berta Ward, Dean of Academic Advising, Articulation and Curriculum
Sharon Yarbrough, Dir. of Institutional Research
Current Situation
9.1 Assessment
English Composition I Instructional Strategies (Medium)
English Composition I (ENGL1010) is the only course at Pellissippi State that is required of all degree-seeking students. Systematic assessment is carried out on at least three levels:
New Student Advising (Medium)
Pellissippi State assesses students' perception of advising using various survey instruments. As nearly as we can determine, most of the instruments employed by the college do not differentiate new students' (those who have completed or enrolled in <30 hours) from all students' perceptions regarding the advising process. The current survey instruments used in assessing advising are the Enrolled Student Survey (ESS) replaced by the Community College Survey of Student Engagement (CCSSE); the First Time Freshman Survey; and the Foundations of Excellence 2-Year Student Survey. These instruments differ in the number and type of advising dimensions they measure (e.g. awareness, use, satisfaction). The First Time Freshman Survey (administered in Spring 05), is the only one that measures <30 hour student perceptions.
Academic Skills Assessment/Developmental Studies Program (Very High)
The Developmental Studies Program (DSP) includes extensive systematic assessment. The DSP has well-defined and measurable goals and objectives. Cohort and non-cohort data are both analyzed.
New Student Orientation (NSO) (Medium)
The success of NSO is assessed through a survey that students and parents/guardians complete and submit at the end of orientation sessions.
Scholarships (Very Low/None)
Did not find any assessment procedure for effectiveness of scholarships for new students. Student Survey questions concern financial aid information received prior to enrollment but are not specific to scholarships. How scholarship information is distributed: Admissions counselors take scholarship application forms on recruiting visits and also send out on request. Forms include information about Music Scholarships, Scholarships Today and Tomorrow, Foundation scholarships and Academic Service scholarships (for students with 2.9 minimum GPA). Application Form is also on PSTCC web site. See
http://www.pstcc.edu/departments/financial_aid/aws.htm for links to forms and information. Scholarships are also promoted through financial aid office, at student orientation, and through HS counselors.
A list of the Foundation Scholarships are available on the web:
http://www.pstcc.edu/departments/financial_aid/foundation.htm Requirements for specific scholarships vary greatly. Factors include residency, GPA (in some cases as low as 2.0), major, etc. Some scholarships are specifically for 1st year students, and some scholarship criteria include 1st year students, although those criteria are not included on the above-listed web site.
9.2 Use of Assessment
English Composition I Instructional Strategies (High)
Instructional Strategies and Processes are included in the ENGL1010 master syllabus. They are evaluated during each program review.
New Student Advising (Very Low)
Instruments used for New Student Advising are currently used primarily for Performance Funding purposes (Under Standard 4B).
Academic Skills Assessment/Developmental Studies Program (High)
The Developmental Studies Program (DSP) uses assessment results to improve or confirm progress toward goal achievement, including specific initiatives, identified in PI 9.1 with the exception of the DSP study skills general education goal.
New Student Orientation (NSO) (Medium)
According to Gayle Wood, each year we try to make adjustments to the NSO agenda based on the comments of students and parents.
Scholarships (Low)
Organized assessment procedures do not exist for Scholarships. Any evaluative information is anecdotal. The application form was improved and a brochure is planned.
9.3 Understanding (Low)
Is this information being used to improve institutional understanding of elements of student success?
According the President's Staff Meeting minutes, the Community College Survey of Student Engagement (CCSSE) results were presented on 11/14/05 by Sharon Yarbrough. Alumni Study results were presented on 11/21/05, and on 01/09/06 the Alumni Survey results were presented. After presentations to the President's Staff, results are posted on the Institutional Research Website.
Presentations on all studies/surveys/benchmarks are made to the President's Staff annually as the results are returned to the office of Institutional Research. Presentations were made to Linda Pinkards' Student Services Directors/Staff of the CCSSE results in December, 2005.
At the October 12, 2004 meeting of the Learning Council, the Spring 2004 Enrolled Student Survey results were discussed. Additionally there are plans for the director of Institutional Research to meet with departments throughout the spring 06 and fall 06 to discuss specific survey data.
Departmental Discussions
EMT, Liberal Arts, NBS and BCT
These departments discuss ways to improve student performance at department meetings, but not in context of specific recent data. Student survey data is used during program review.
Mathematics
The Math department uses data from the Learning Center evaluation of tutoring services to recommend improvements in math tutoring. The satisfaction surveys were completed by math students and faculty at Pellissippi and Division Street campuses.
The Math department also discusses factors contributing to student success/failure in math. The number of hours that students are employed and their semester course loads are two factors that we have discussed as being related to student success. It would be great to have hard data demonstrating that there is a high correlation between these two factors.
Attendance is always perceived as being critical to student success. Math faculty members have expressed the opinion that most of their students are capable of mastering the course content, if they would only attend class regularly and would like to have data on student class attendance patterns.
English
The English Department discussed these issues at their department meeting during in-service in January. The department is currently focusing on ways to increase the sense of community at the college as a whole, with the idea of making first year students feel more a part of the school, more connected to faculty and their peers.
Library
Library faculty and staff discussed both their annual report statistics and the student survey results in their most recent staff meeting. That meeting included preliminary discussions for the performance planning and budget planning cycle for next year. The library will be using the annual report stats and survey results frequently over the next weeks as they work on the planning and budget cycle.
The library collects Annual data to determine student use of its resources. The library also completed a survey last spring that is posted on the web where they try to determine student use of services and resources and indirectly we ask them about their time allocation. The most recent student survey is available on the library's website in pdf format for your review:
http://www.pstcc.edu/library/Library_Survey_2005.pdf
9.4 Strategies
Attendance at Higher Education Meetings (High)
Pellissippi State faculty members are engaged in professional activities (e.g. attending conferences, institutes, workshops) in order to assist students in their student success. We assessed the degree to which this occurs by using three methods.
First, in the foundations of Excellence Faculty Assessment Survey, question 41 asks, "To what degree are you engaged in this professional activity focused on student success: Attending conference or meeting sessions?" All faculty were asked this question. The response rate was 84.6% representing 334 faculty members. The responses varied as follows: 17.7% "not at all," 14.7% "slight," 26.3% "moderate," 22.5% "high," and 18.9% "very high." Using a 1 (not at all) to 5 (very high) scale, the mean was 3.10 and the standard deviation was 1.35.
Second, a survey was sent to the six department heads, Responses were gathered from five of the six, with one department head abstaining from responding. The following three items were posed with a summary of responses:
Third, faculty members are required to attend fall and (sometimes) spring in-service programs that often have "student success" as the umbrella theme. The following is a subset of titles included last fall 2005: A Focus on Attention Deficit Disorder: How to Help Our Students; Why Do I Bother? A Classroom Success Story; Thinking Outside the Classroom: Opportunities for Motivating African American Students; Getting Students Excited on the First Day of Class: Sharing Techniques.
Multi-Institutional Initiatives (Low)
Initially, these four "regional projects" were considered: ABE, Tech Prep/Developmental in high schools, dual enrollment, and RODP. Since ABE and Tech Prep do not involve new currently enrolled PSTCC students, they were dropped from the list.
Dual enrollment: In 06S, high school students must meet certain ACT and GPA requirements and have a letter from the principal to be dually enrolled in ENGL. Students receive both an high school and a college grade. Both the high school teacher and the Pellissippi instructor meet the class. In Knox schools, the college instructor is in class at least 1.5 hours weekly; in Blount County, the contact is less than 1.5 hours. In both cases, instructors maintain contact with students via email, follow the ENGL college syllabus, and grade students' papers. Thus, students experience the college classroom to get a perspective on how it differs from the high school environment.
RODP: The Regents On-Line Degree Program is offered to all students, not just new students, as an alternate method to earn credits and a degree through the Web. COL 101 Orientation Course: The College Experience On-Line, developed by another TBR institution, is intended for new students taking on-line courses; it is not a requirement for the completion of a degree.
Institution-Wide Exposure to External Experts (Medium)
A number of external experts have been brought to campus including for workshops, teleconferences, program review-Geier workshops, university parallel program review consultant, and others.
Broad Exposure to Campus-Based Knowledge/Expertise about the New Student Experience (Medium)
Based on the student surveys, students have good knowledge of and satisfaction of institutional and formal aspects of the new student experience such as financial aid and computer facilities and somewhat less when dealing with the more personal aspects of college life such as self confidence, getting a long with people and growing as a person.
Areas of Concern:
9.1 Assessment
English Composition I Instructional Strategies
None
New Student Advising
We question whether any of these instruments are capturing the most important dimensions of student advising. These instruments vary in the number and type of questions asked. Typical questions involve student awareness of advising services, student use of advising resources and general satisfaction with the advising experience. While this information is certainly valuable, it may not provide the information needed to understand the subtle nuances that create an optimal advising experience for both student and advisor.
Currently, there are no plans to administer The First Time Freshman Survey which really is the only survey on new student advising that gives us information regarding 1st year students.
Academic Skills Assessment/Developmental Studies Program
While the study skills course is assessed by a post survey of students, the
assessment is not designed with measurable benchmarks.
New Student Orientation (NSO)
While the survey does help staff to better address what consumers (students, parents/guardians) consider important/interesting, it does not help us to determine whether or not students who attend NSO are more successful than others at adjusting to college life and fulfilling the requirements of their academic programs.
Scholarships
Several concerns about getting scholarship information to students emerged in interviews:
9.2 Use of Assessment
English Composition I Instructional Strategies
None
New Student Advising
There is currently no formal protocol for using results of assessments for
continuous improvement efforts in the area of New Student Advising.
Academic Skills Assessment/Developmental Studies Program
Limited evidence exists that assessment results are used in the study skills.
New Student Orientation (NSO)
Adjustments are made for the NSO based only upon comments of students and parents,
not upon success of participants, which is not tracked.
Scholarships
Organized assessment procedures need to be created for Scholarships. The brochure needs to be higher priority.
9.3 Understanding
Responses from several department heads indicated a lack of awareness of availability of specific survey data. College needs to bridge the gap between data collection and utilization of that information. Dissemination of information needs to be improved.
It is also the perception that we are missing clear data showing a correlation between student success and class attendance, and student success and allocation of time. It is the assumption among all departments that class attendance and allocation of time are critical to student success.
9.4 Strategies
Attendance at Higher Education Meetings
When asked about concerns regarding faculty members' attendance at professional conferences, institutes, and/or workshops, four of the five department heads responding mentioned "funding" for professional activities as a likely contributor to those not attending (one department head abstained from responding to the survey). Three out of
five mentioned "lack of time" as another contributing factor.
Multi-Institutional Initiatives
Dual enrollment: In ENGL, the current situation with college instructors providing face-to-face guidance with students and grading papers at the college level is deemed successful. A proposal was made at the Learning Council that may increase enrollment but also may return control of the classroom to the high school instructor. If the dual enrollment course has the same high school/college classroom teacher, the concern is that the students would miss the college experience; the assigned college liaison may regress to just a paper-grading role.
RODP: While RODP courses are listed in the college Schedule of Classes, no special effort is made to enroll new students (or returning students) in the program. Rather, it is treated as one of many options for attaining a degree.
Institution-Wide Exposure to External Experts
There seems to be an information overload because most events are announced via email. Faculty and students may overlook these events simply because there are so many announcements with in the pstcc email system. Also, many students do not even use their pstcc email but use their own personal email.
Many of these events apply to all students and there is no unified theme for providing expert information about first-year students but rather a shotgun approach. There has not been a thoughtful plan developed and implemented to address specifically consultant assistance in supporting first-year students.
Broad Exposure to Campus-Based Knowledge/Expertise about the New Student Experience
Summary of Evidence:
9.1 Assessment
English Composition I Instructional Strategies
The English department's five-column models, most recent program review
report and ENGL1010 master syllabus were reviewed. In addition, the English department head was interviewed to obtain additional information and feedback on the department's assessment activities. Based on the information gathered, it appears that systematic
assessment is an ongoing part of the delivery and management of the College's ENGL1010 course.
New Student Advising
The First Time Freshman Survey is the only <30 hour student measure currently employed. It appears to measure five dimensions of new student advising. How do we know these are the most important dimensions to measure? What dimensions should we base continuous improvement efforts upon? For example, there are no questions that directly assess "rapport" with the advisor.
Academic Skills Assessment/Developmental Studies Program
The DSP program is made up of four subject areas - mathematics, reading, study skills, and writing. Each of these subject areas is housed in an academic department with complementary disciplines. Therefore, each DSP subject area is regularly evaluated during the academic department program reviews (every five years) as well as during the institutional self-study conducted by the College for reaffirmation of its
accreditation from the Southern Association of Colleges and Schools (SACS) every ten years. These self-evaluations are used to determine whether the goals of the program and needs of the students are being met.
Furthermore, in support of the College's annual planning and evaluation process, academic departments assess the achievement of general education goals. Three of the four DSP subject areas - mathematics, reading, and writing - use a (5-column) model developed for the College to organize the assessments effects. The study skills course is evaluated by a post-attitude survey.
In addition, the DSP Director annually evaluates the overall goal of the DSP program goal: a student who completes the required developmental courses will experience the same or better success in college-level classes and attain his or her college goals as well as a student who does not require developmental courses. This annual report on DSP student success in college-level courses analyzes non-cohort data and compares DSP completers with non-DSP students. An evaluation of the DSP program specific goals such as success in the alternative course delivery programs - learning communities, linked courses, and flexible scheduling (FLAG Center) is also conducted annually.
The DSP program is currently completing a comprehensive self-study and has submitted this study of the four subject areas to the National Association of Developmental Education (NADE) for possible NADE Certification. This review is important in that it does not rely on non-cohort data but also involves extensive analysis of cohort data. Student-centered goals examined in this study are 1) DSP students will complete their DSP course at the same or higher rate as the DSP students in the previous year (measured by DSP course completion rate and DSP course grade distribution with non-cohort data). 2) DSP students will experience the same or better success in their first college-level course as students who do not require the developmental course of study (measured by college-level course completion rate and college-level course grade distribution with cohort data). 3) DSP students will be retained at the College at the same or higher rate as students who do not require developmental studies coursework (measured by retention through 24 hours with cohort data). 4) DSP students will graduate at the same or higher rate as students who do not require developmental coursework (measured by graduation rate and average GPA upon graduation with cohort data).
In a collaboration Fall 2005, the Mathematics and DSP departments/programs selected the following Performance Funding goal: Proportion of students that successfully completed their developmental/remedial course and successfully completed their Math college-level course. Student success indicators will be measured by the National Community College Benchmark Study
During the 2005-2010 five year cycle, the Strategic Planning goal will focus on efforts to increase enrollment in the FLAG program by 15 percent. In addition, success rates will be compared to traditional courses. Benchmark data will reflect 2004-2005 activity and will be used as baseline data in making annual comparisons of student's success in completing FLAG and traditional courses.
NSO
Success of NSO is measured by participants' immediate responses to the sessions but not by tracking of participants and comparison of their success to the success of non-participants.
Scholarships
Currently there are no processes for evaluating the dissemination of scholarship information. There are problems with the dissemination of scholarship information that need to be addressed as itemized in Areas of Concern.
9.2 Use of Assessment
English Composition I Instructional Strategies
Examples that show that assessment results are implemented for English Composition I Instructional Strategies include:
New Student Advising
The ESS and Alumni Surveys have been used as effective indicators as reflected in performance funding reports and in planning (e.g. the creation of the Coordinator of advising position several years ago). Indirect assessment has taken place on advisor training by evaluating errors made primarily on Intent-to-Graduate forms and by recording advising questions received from faculty throughout the semester. This information is used as a basis for developing the agenda for advisor information sessions. Some assessment has been done on new advisor training and feedback from this has been used to modify the program in subsequent years.
Academic Skills Assessment/Developmental Studies Program
A review of 2004-2005 assessment results indicate that benchmarks for improvement were not met for the Development Studies Program. New action plans were written to improve student performance in mathematics (DSPM), reading (DSPR), and writing (DSPW). These action plans include the writing of handouts, adjustment in lecture schedule to reflect additional time on problems not mastered, and notes made available to aid student understanding; adopting a new vocabulary text; and revising course content and grammar instruction.
Analysis of the overall DSP program goal using non-cohort data continues to confirm that this goal is being met. This goal is assessed on a yearly basis.
Evaluations of the specific goals for student success in the alternative course delivery programs - learning communities, linked courses, and flexible scheduling (FLAG Center) is also conducted annually. The 2004-2005 analysis indicated that the goals were being met. Progress toward goal achievement is monitored on a yearly basis.
The DSP program self-study involved assessment of four student-centered goals. A summary of the use of assessment follows:
The Strategic Planning baseline data reflected number of students who enrolled and who completed the FLAG courses. The grades of those students were used to compare with grades of students enrolled in traditional courses. As stated previously, FLAG students perform as well as students who enroll in traditional courses.
New Student Orientation (NSO)
Adjusting the NSO agenda to better address concerns expressed by participants seems very appropriate. However, the agenda is not adjusted based upon success of participants, which is not tracked.
Scholarships
There is no summary evidence for Scholarships since Pellissippi has no formal assessment procedures.
9.3 Understanding
The FOE team examined recent student surveys including the FOE student survey, First Time Student Survey and the CCSSE for questions related to students allocation of time, student/faculty relations, student class attendance patterns and use of student services. The team documented any question related to these dimensions of student success. The team found no questions related to student allocation of time or class attendance pattern, but data does exist regarding use of student services and student/faculty relations. The FOE team also examined minutes from the President's council and Learning Council to see if the
information from these results was used to improve institutional understanding. Individual department heads were also interviewed via e-mail as to the use of survey information regarding these dimensions to improve student success. The FOE team found that while departments are always trying to find ways to improve student success, this is generally not done within the context of specific survey results. The library in particular does a great job of documenting library services used and making specific refinements and improvements based on these results.
9.4 Strategies
Attendance at Higher Education Meetings
Two-thirds of the faculty reported that they engaged in attending conferences moderately to very high. The department heads reflected a wide variance in the percentage of individuals who attend conferences, etc. Clearly some department faculty members' attendance at such events (high three-year range 100%) is far more than others' attendance (low three-year range 22%). It should be noted that all faculty are requested to attend the "Back to Campus In-service Workshops." Many, if not all, of the conferences listed above are fully or partially engaged in topics regarding "student success."
Multi-Institutional Initiatives
Dual enrollment: The advantage to new students in participating in dual enrollment is that they start college with up to six hours credit. As the program exists now, dual enrollment prepares students for what to expect from college instruction.
RODP: Statistics on success rates and enrollment are not readily or routinely available to the college; the Nashville central office generates the administrative reports.
Institution-Wide Exposure to External Experts
Pellissippi does offer a wide exposure to external experts through out the academic year. However, faculty feels that there needs to be a better way to disseminate these events instead of using PSTCC email.
In addition, there is no specific plan that addresses specifically first-year students.
Recommended Grade: C-
Recommended Action Items:
English Composition I Instructional Strategies
None
New Student Advising
For New Student Advising, consider how best to use feedback from current instruments and/or to revise instruments to gather additional information on important but subtle dimensions of the advising process
Academic Skills Assessment/Developmental Studies Program
Use the revised survey and benchmarks (See PI 9.1 Action Plan) to improve or confirm existing practices in study skills.
New Student Orientation (NSO)
Adjusting the NSO agenda to better address concerns expressed by participants seems very appropriate. However, the agenda is not adjusted based upon success of participants, which is not tracked.
Scholarships
Attendance at Higher Education Meetings
Multi-Institutional Initiatives
Dual enrollment: Keep the college instructor face-to-face contact with students as a component of the course.
Institution-Wide Exposure to External Experts
Broad Exposure to Campus-Based Knowledge/Expertise about the New Student Experience
K.D. still needs to add to here
English Composition I Instructional Strategies
None recommended.
New Student Advising
Academic Skills Assessment/Developmental Studies Program
Revise the assessment used in the study skills course with a survey that is measurable and determine appropriate benchmarks
NSO
Need some instrument to track students after 1st semester or after completion of 30 hours to determine if the NSO was helpful for students and contributed to their success.
Scholarships
Top Improvement: Dimension Report | Performance Indicators
Dimension Reports: Philosophy | Organization | Learning | Campus Culture |
Transitions | All Students | Diversity | Roles & Purposes | Improvement
Home | Themes | Foundational Dimensions ® | Dimension Reports 2005 | Statement of Philosophy | First Year Council