3.4 Reality check: designing a new leadership program for the 21st  (Page 10/15)

Methods for evaluating and modifying program components

Program evaluation regarding the effectiveness of curriculum and field activities in meeting theneeds of their respective school districts is being conducted through surveys completed by program participants, mentors and/orclinical supervisors, and university faculty. Over time, school leadership success factors (e.g., professional developmentopportunities, student discipline, and teacher satisfaction) that are not measured by traditional accountability reports, notablystandardized test scores, are examined to determine the longitudinal impact of the school leadership preparation programand its prospective leaders. For each field-based project completed and implemented by candidates, a self-developed survey isdistributed to stakeholders (e.g., teachers, parents, students [as appropriate], administrators) to determine their level of satisfaction with the overall performance of the candidates duringthat seminar. Additionally, follow-up surveys are distributed to school and district administrators to determine their perceptionsof the preparedness and effectiveness of program participants involved in field experiences and internships at their respectivesites.

Each candidate is surveyed at the end of Seminars II and V to determine individual levels of satisfactionwith the program and the quality of instruction. These evaluations are used to provide the data necessary to monitor, evaluate, andmodify the program as needed. The data are collected through the PASS-PORT electronic assessment system. Other data, such as studentopinions of teaching and exit surveys are used to provide additional information regarding the quality of the program.

Analysis of the results is conducted by the university staff and members of the advisory council, who thencarry on the processes of program evaluation, formal discussion and dialogue, and collaborative decision making before makingrecommendations for program improvement. Approved changes are then systematically studied to measure their effects on the program and,consequently, on the leaders being produced by this program.

Results from Surveys and Assessments

The first two cohorts completed surveys, some individually and some in focus groups. Mentors and field siteschool principals completed an open-ended evaluation of the program. Superintendents wrote letters of opinion. Professorscompleted an electronic evaluation.

Cohort Surveys and Assessments

When asked if the first 6 hours met their expectations, students asked for more field experiences and lesslecture (face-to-face) time, additional assessment during the semester, and extra field time with their mentors. They felt a lackof district and school recognition, support, and approval. Some felt afraid to talk to their site principal, even when they had aschool leader mentor from another site. Some reported that other teachers who had completed another graduate program complainedabout the attention and release time for cohort members. They reported the need for more communication with mentors and releasetime for field experience. The artifacts generated by each student in their electronic portfolios have been, on the whole, judged tobe of high quality. (See Appendix D.)