Standard 4: Program Impact
The provider demonstrates the impact of its completers on P-12 student learning and development, classroom instruction, and schools, and the satisfaction of its completers with the relevance and effectiveness of their preparation.
IMPACT ON P-12 STUDENT LEARNING AND DEVELOPMENT
4.1 REQUIRED COMPONENT - The provider documents, using multiple measures, that program completers contribute to an expected level of student-learning growth. Multiple measures shall include all available growth measures (including value-added measures, student-growth percentiles, and student learning and development objectives) required by the state for its teachers and available to educator preparation providers, other state-supported P-12 impact measures, and any other measures employed by the provider.
INDICATORS OF TEACHING EFFECTIVENESS
4.2 REQUIRED COMPONENT - The provider demonstrates, through structured and validated observation instruments and/or student surveys, that completers effectively apply the professional knowledge, skills, and dispositions that the preparation experiences were designed to achieve.
SATISFACTION OF EMPLOYERS
4.3 REQUIRED COMPONENT - The provider demonstrates, using measures that result in valid and reliable data and including employment milestones such as promotion and retention, that employers are satisfied with the completers’ preparation for their assigned responsibilities in working with P-12 students.
SATISFACTION OF COMPLETERS
4.4 REQUIRED COMPONENT - The provider demonstrates, using measures that result in valid and reliable data, that program completers perceive their preparation as relevant to the responsibilities they confront on the job, and that the preparation was effective.
Standard 4 Narrative
4.1 IMPACT ON P-12 STUDENT LEARNING AND DEVELOPMENT
The Alabama State Department of Education does not provide EPPs access to state P-12 student learning data. Since 2016, Alabama State Department of Education has been led by four different State Superintendents of Education (three permanent and one interim). The state assessment system has also experienced an enormous amount of flux. The Alabama State Board of Education voted in 2017 to suspend administration of the ACT Aspire Test System as the state’s annual reading and math assessment for grades 3 through 8 in public schools. For the 2017-18 school year, public schools used Scantron formative data to produce a summative test score. Scantron was administered in the 2018-19 academic year. In July 2018, the State Board voted to approve a contract with Data Recognition Corporation (DRC) to design and implement a statewide assessment system for K-12 students with the goal of statewide computer-based administration in the 2019-2020 academic year based on Alabama’s college- and career-ready standards. Most recently, in March 2019, the Alabama Senate bill passed a bill to repeal the Alabama college- and career-ready standards, and it has now moved to the Alabama House of Representatives.
Since no state P-12 student learning data are made available to the EPP, it has created and begun implementation of a plan to obtain and analyze any student learning data made available by our major P-12 partners. The EPP contacted the Superintendent and central administration staff for each of the following P-12 partners (Huntsville City Schools, Madison City Schools, Madison County Schools, and Athens City Schools) to dialogue and ask questions to determine what measures of P-12 student learning data were available. While the data are very limited and focus only on elementary students, the districts were willing to assist the EPP in identifying any student data associated with identified completers currently employed in the district. Madison City Schools is able to offer STAR reading and math data for elementary students, Madison County Schools has DIBELS reading data for elementary students, and Huntsville City Schools has Scantron data for students in grades 3-12 (only administered for two years). It should be noted the Scantron assessment is not aligned with the Alabama college- and career-ready standards. Currently, Athens City Schools are still determining what data might be provided. A list of completers employed by each district has been provided to each one to assist with identification of available P-12 student data.
In addition to data provided by the P-12 partners which the EPP will review and analyze as a means of addressing CAEP Standard 4.1, the EPP has also determined the need to implement a case study approach with completers which will be piloted in the 2019-2020 academic year. (4.1 Impact on Student Learning and 4.2 Teaching Effectiveness Plan).
SUMMARY OF EVIDENCE AND SUPPORTING DOCUMENTATION
EVIDENCE 4.1 Impact on Student Learning and 4.2 Teaching Effectiveness PLAN
4.2 INDICATORS OF TEACHING EFFECTIVENESS
Alabama does not administer state-created student perception surveys or implement a statewide teacher observation system. The EPP met with each of its major P-12 partners (Huntsville City Schools, Madison City Schools, Madison County Schools, and Athens City Schools) to explore what district-specific data might be available concerning teaching practices of completers teaching in the district. The districts reported no measures were currently implemented to assess teaching practices, such as classroom observation instruments or student perception surveys. While the EPP utilizes EDUCATE Alabama with its interns, it is not typically used in the partner school districts other than to inform teacher-created professional learning plans in some districts. Given the lack of available data, the EPP has created a plan (4.1 Impact on Student Learning and 4.2 Teaching Effectiveness PLAN) which will be piloted in the 2019-20 academic year. The plan includes a purposive sampling of 6 completers in area public schools, along with administration of student perception surveys and focus groups of administrators currently employing completers. Data will include classroom observations of completers by faculty using the Danielson Framework instrument used with interns, along with administration of student perception surveys. Data and insights gleaned during the pilot implementation will inform further development and expansion of the plan in the 2020-21 academic year to include 12 completers.
SUMMARY OF EVIDENCE AND SUPPORTING DOCUMENTATION
EVIDENCE 4.1 Impact on Student Learning and 4.2 Teaching Effectiveness PLAN
4.3 SATISFACTION OF EMPLOYERS
Following the release of the CAEP standards, CAEP Coordinators from Alabama’s 27 member institutions of the Alabama Association of Colleges for Teacher Education (ALACTE), along with Alabama State Department of Education, met in July 2015 to initiate discussion and plans for Standard 4 reporting measures. In Fall 2015, an ALACTE task force was created and efforts began to create a statewide employer and alumni survey based on inTASC Standards. In Fall 2016, after a statewide stakeholder two-day meeting concluded, the survey format was refined and a content validation process was developed. It was determined the Alabama State Department of Education (ALSDE) would disseminate the statewide survey to employers and first-year completers. In Spring 2018, the ALSDE launched the surveys. Responses were distributed to each EPP. The second ALSDE survey launch is anticipated in April 2019 with data provided to the EPPs from the ALSDE by Fall 2019.
The UAH Employer Survey (2017-18) launched by the ALSDE yielded 16 respondents from administrators of teachers employed in Alabama public schools. Employers rated teachers on a descriptive scale with options of Ineffective Teacher, Emerging Teacher, Effective Teacher and Teacher Leader on 26 items. On 8 of the 26 items, one administrator responded the first-year completer was “Ineffective,” but all other response indicated UAH completers were considered to be Emerging, Effective or Teacher Leaders. The highest number of responses (n=4) for “Teacher Leader” was for the statement, “The teacher practices the profession in an ethical manner” followed by “The teacher understands the expectations of the profession including the Alabama Educator Code of Ethics, the NASDTEC Model Code of Ethics for Educators, (MCEE), professional standards of practice and relevant law and policy” (n=3). When reviewing the survey items, those with the highest percentage of responses indicating Effective Teacher or Teacher Leader were “The teacher practices the profession in an ethical manner” (n=13), “The teacher implements assessments in an ethical manner and minimizes bias to enable learners to display the full extent of their learning” (n=12), “The teacher understands the expectations of the profession including the Alabama Educator Code of Ethics, the NASDTEC Model Code of Ethics for Educators, (MCEE), professional standards of practice and relevant law and policy” (n=12) and “The teacher engages in continuous professional learning to more effectively meet the needs of each learner” (n=10).
SUMMARY OF EVIDENCE AND SUPPORTING DOCUMENTATION
EVIDENCE 4.3.A ALACTE Survey Development Process 2-28-18
EVIDENCE 4.3.B UAH ALSDE Report Card
EVIDENCE 4.3.C UAH Employer Summary Data
EVIDENCE 4.3.D UAH New Teacher Summary Data
EVIDENCE 4.3.E UAH Summary of Survey Results
4.4 SATISFACTION OF COMPLETERS
After several years of collaboration and development,The ALSDE launched the statewide in-service teacher survey at the end of the 2017-18 school year. The survey included 21 items to which respondents could reply Strongly Agree, Agree, Disagree, or Strongly Disagree. Sixteen UAH completers responded to the survey (10-elementary; 6-secondary) with twelve of the respondents indicating they were in year one of teaching and only one respondent indicating s/he was teaching outside the certification area. No items yielded responses of Strongly Disagree. One item, “My teacher preparation program prepared me to manage the learning environment to engage learners actively” yielded two responses of Disagree. Seven additional items yielded one response of degree. When analyzing responses of the 16 completers, every item had at least 81% of responses in either the Strongly Agree or Agree category, indicating completers’ satisfaction with the program.
Additionally the EPP surveys its candidates at the completion of internship each fall and spring semester to determine their perceptions of the educator preparation program and determine how well equipped candidates believe they are for classroom teaching. Fall 2018 data indicate those who completed the program felt best prepared to “create and maintain a safe and productive learning environment,” (item 12), “Develop as a professional,” and “think critically and self-reflect.” The areas where they indicated the educator preparation program could improve was in quality of placements (timely placements, opportunities to observe good models of teaching), quality of mentoring from university supervisors, and quality of mentoring from your cooperating teacher. The EPP noted this was the first semester edTPA was consequential in Alabama, and this might be reflected in some of the areas noted as opportunities for growth or improvement. (2.1.R Intern Survey)
SUMMARY OF EVIDENCE AND SUPPORTING DOCUMENTATION
EVIDENCE 4.4.A ALACTE Survey Development Process 2-28-18
EVIDENCE 4.4.B UAH ALSDE Report Card
EVIDENCE 4.4.C UAH Employer Summary Data
EVIDENCE 4.4.D UAH New Teacher Summary Data
EVIDENCE 4.4.E UAH Summary of Survey Results
EVIDENCE 2.1.R UAH P-12 School Partners (LEAs) and Major Employers of Completers
Standard A.4: Satisfaction with Preparation
The provider documents the satisfaction of its completers from advanced preparation programs and their employers with the relevance and effectiveness of their preparation.
Satisfaction of Employers
A.4.1 The provider demonstrates that employers are satisfied with completers’ preparation and that completers reach employment milestones such as promotion and retention.
The first M.Ed. in Differentiated Instruction degrees (advanced licensure) were awarded in Fall 2015. A comprehensive list of completers by semester has now been assembled and efforts are underway to identify where completers are employed. This is challenging since there is no state database, many of our candidates are female and often marry and change last names, and many move across schools, districts, and locations. The EPP is utilizing social media, last known email addresses, TEAC announcements, requests to local P-12 partner districts, faculty-candidate contacts, to determine where candidates are currently employed. Finally, candidates are also now informed they will be contacted at one year and three years after completion to obtain employer and completer information. The EPP requests advanced candidates provide two emails, two contact phone numbers and current place of employment when enrolled in ED 690 in the last semester of the degree program as a way of building an advanced candidate database. Utilizing a Google Doc, along with an improved database, the EPP is hopeful it will realize greater success in tracking advanced candidates after graduation. Additionally, the EPP has worked with the University Registrar and Graduate Admissions Office to develop accurate degree and concentration codes to support better identification and tracking of advanced candidates. Finally, the EPP recognizes the need to identify senior administrators in each school where advanced completers are employed on an annual basis since area schools in north Alabama tend to have quite a bit of turnover in school-based administrators within and across districts.
The EPP faculty and TEAC will identify common questions for the employer survey as well as questions specific to each concentrations or advanced licensure area based on program standards. All items will utilize a Likert rating scale (e.g. Mastery, Proficient, Emerging Competence, and Unsatisfactory) to facilitate comparisons across programs. Qualtrics or TK20 will be used to collect data from employers and facilitate analysis. In programs with smaller numbers of completers, some faculty may use an interview protocol to facilitate a focus group of employers. Faculty members will develop and validate the completer survey for advanced candidates in Summer and Fall 2019. The advanced completer survey will be launched in Spring 2020 and then annually administered by the EPP. The survey will be distributed electronically to school-based administrators with advanced completers in their building. Administrators will submit a separate survey for each advanced completer if multiple individuals are employed in one building.
Another method UAH will utilize to obtain employer satisfaction is to survey personnel directors or district personnel in area P-12 districts. Each year, the Coordinator of Field and Clinical Experiences and the Dean meet with area partner district leaders to better understand their needs and concerns. This, along with meetings twice a year with the Teacher Education Advisory Council (TEAC), provides opportunity to gather feedback from employers concerning their satisfaction with completers.
A.4.2 The provider demonstrates that advanced program completers perceive their preparation as relevant to the responsibilities they confront on the job and that the preparation was effective.
Following a process similar to the one outlined above, a completer survey will be developed and administered. As faculty and TEAC create and validate the completer survey, every effort will be made to assure it with the employer survey. Contact information will be gathered from Spring 2019 advanced level completers so the employee survey can be successfully piloted in the 2019-2020 academic year. Each completer will be sent an email with a hyperlink to the Qualtrics or TK20 advanced completer survey.
