Standard 5: Provider Quality, Continuous Improvement Capacity
Standard 5 Narrative
Quality and Strategic Evaluation
5.1 The EPP is committed to a Quality Assurance System (QAS) that is comprised of multiple measures to monitor candidate progress, completer achievements, and provider operational effectiveness.
UAH has experienced tremendous growth as it has launched numerous new degrees leading to initial licensure since its inception in Fall 2014 (e.g., B.S. in Secondary Education, B.S. in Early Childhood and Early Childhood Special Education; B.S. in Kinesiology - Physical Education Licensure Master of Arts in Teaching-Secondary Education; Master of Arts in Teaching-Elementary Education; Master of Arts in Teaching-P-12 Subject Areas).
Along with obtaining SACSCOC approval for each new degree program, the EPP also submitted each licensure program one for review by the Alabama State Board of Education through the new process known as Continuous Improvement in Educator Preparation (CIEP). The CIEP process, first launched in 2015, is modeled after the SPA process and focuses on alignment with standards, meaningful field and clinical experiences, and key assessments and detailed rubrics aligned with standards. It requires each program to be designed around comprehensive and intentional use of standards-based curriculum, integrated and meaningful assessments and field experiences, and analysis, summary and use of data to inform curricular and programmatic decision-making to foster continuous improvement of educator preparation programs.
Our Quality Assurance System (QAS) continues to emerge and evolve through our efforts to systematically and intentionally create effective programs designed to equip candidates for the demands and experiences of classroom teaching. Aligning institutional, state, and national standards has been a challenging and meaningful process for all faculty as field experiences, key assessments, scoring rubrics, crosswalks, and courses have been created. All programs have been submitted for CIEP review, and UAH anticipates applying for SPA approvals in Fall 2020. As program enrollments have grown, so has the need for gathering, reviewing, and analyzing data. UAH utilizes TK20 and is exploring how to harness its capabilities even further to facilitate data collection and analysis for key assessments and candidate data. A constant challenge is identifying the various data points required at both the program and EPP levels for the multitude of reporting requirements, many which are based on different timing cycles.
The EPP-created assessment instrument for dispositions was developed, used, and validated by faculty members and Teacher Education Advisory Council (TEAC) members using the Lawshe Method for content validity. The Associate Dean position was added in Spring 2018 to lend support to Data and Accreditation requirements and needs. Engaging in the CAEP and CIEP processes simultaneously, as well as ongoing state and national reporting demands, led to the realization that additional support was needed. A data analyst staff position was approved in Spring 2019 and will be advertised by May 2019. The addition of this individual will support efforts as we continue to build and develop a robust QAS.
The QAS is designed to intentionally assess candidates’ academic performance, dispositions, and experiences in classrooms and internships. Data are collected from national assessments (e.g., Praxis CORE, Praxis II exams, edTPA) as well as multiple data sources both internal (e.g., faculty dispositions checklist, Professional Development Plans, field experience placements) and external (mentor teacher surveys, Teacher Education Advisory Council, etc.). The faculty intentionally create and use multiple data sources and assessment strategies to regularly assess candidates’ knowledge, skills, and dispositions. Regularly monthly faculty meetings allow all EPP faculty to share insights, review data, and collaboratively design and implement assessments. For example, the current observation instrument (Danielson Framework) piloted by university supervisors in the internship semester in 2018-19 was reviewed and adopted by faculty in the 2017-18 year after recognizing the need for a valid and reliable instrument.
Multiple assessment techniques are used to evaluate candidates, program effectiveness, faculty effectiveness, and EPP operations. Data are collected, analyzed and used to improve candidate performance, curricula, instruction, delivery, and operations. Recently, in Fall 2018, the Alabama State Department of Education (ALSDE) issued an Educator Preparation Institutional Report Card for each college or university in the state. It is anticipated these will now be generated annually. The 2017-18 UAH Educator Preparation Institutional Report Card (4.4.B UAH ALSDE Report Card) issued by ALSDE documented program and candidate strengths. For example, 100% of initial Class B candidates passed the Core Knowledge/Applications Praxis II content exam on the first attempt and 90% of elementary education candidates passed the Elementary Education: Multiple Subjects Praxis II content exam on the first attempt.
UAH has proposed key assessments and data tables which would be implemented at the program level and reviewed through the Alabama State Department of Education CIEP process. Data will be gathered on academic year cycles. It is anticipated the Dean, Associate Dean, and Data Analyst will assist in summarizing and preparing data for review by program faculty. Additionally, edTPA and internship data will be prepared in consultation with the Director of Field and Clinical Experiences. Data will be reviewed by program faculty, as well as P-12 partners when appropriate, once each semester. Any possible changes to field experiences, courses, internship, handbooks, or unit policies or practices will be informed by data. Additionally, an EPP-wide data retreat will be conducted each fall to facilitate analysis and discussion of data and inform strategic planning and needed revisions, adjustments, and additions for specific licensure programs at the initial and advanced levels.
Candidate progress is assessed at several transition points throughout the program, including admission, program progress, internship, program completion and follow up. These transition points (3.4.A Decision Points) also require routine data, such as GPA analysis, Praxis CORE scores, edTPA, etc. Through monthly meetings of the teaching faculty, the EPP administers, coordinates, evaluates, monitors, reviews, and revises all programs. The Dean and Associate Dean, as well as the Director of Field and Clinical Experiences and teacher education advisors, actively contribute to and participate in the assessment process.
SUMMARY OF EVIDENCE AND SUPPORTING DOCUMENTATION
Supporting Documentation: 5.1.A UAH College of Education Timeline
EVIDENCE: 4.4.B UAH ALSDE Report Card
EVIDENCE: 3.4.A Decision Points
EVIDENCE: UAH Quality Assurance Plan
5.2 The Quality Assurance System (QAS) relies on relevant, verifiable, representative, cumulative and actionable measures, and is improving its ability to produce empirical evidence that interpretations of data are valid and consistent.
Our QAS, while still in its infancy in terms of design and implementation, is an integrated component of our College and institutional governance systems with intentional layers of review and feedback by varied entities and stakeholders. The Dean, in concert with the Associate Dean, department chairs, faculty, and Education Student Services staff members, develops and implements assessment policies and practices and monitors the QAS across the EPP. Faculty members have assumed leadership and responsibility for various key assessment data for specific initial licensure programs to support CAEP reporting outcomes, SACSCOC outcomes, SPA initiatives and CIEP state program reporting. The Dean also coordinates the efforts of the EPP with the UAH Office of Institutional Research and Assessment, University Registrar, Financial Aid, and other relevant units on campus.
Engaging in two continuous improvement processes simultaneously for both CAEP and CIEP (state program approval) shortly after establishing multiple new degrees, hiring new faculty, and adding new licensure options, has presented rich opportunities to engage in robust dialogue and creation of quality key assessments, scoring rubrics, and establishing of priorities and plan to further develop a systematic approach to continuous improvement informed by meaningful data. Given the rapid growth and change experienced in the last four years, several initiatives and objectives are emerging:
-
It became clear the EPP needed to identify key individuals with delineated responsibilities to support the focus on outcomes-based measures, intentional data collection and analysis, and the ability to guide and support program faculty in creation and implementation of key assessments aligned to standards. This has resulted in creation of an Associate Dean position and a Data Analyst position to provide additional supports.
-
The implementation of the new CIEP state program approval process, along with the hiring of many new faculty in the College, increased Praxis II qualifying score requirements, consequential statewide implementation of edTPA, and exploration of SPA requirements, highlighted the need to provide faculty and staff support as they learned about CAEP and CIEP standards, edTPA rubrics, construction of standards-based assessments and scoring rubrics, and curriculum mapped to standards. This resulted in faculty participating in workshops provided by Alabama State Department of Education focused on CIEP program submissions, edTPA regional conferences and local evaluation trainings, curriculum mapping and curricular revisions in departmental faculty meetings, and other professional development to support the creation of programs intentionally designed around a QAS. Faculty are integral members of the QAS efforts, and continuous improvement, reflective practice, and quality assurance are hallmarks of UAH programs.
Upon completion of the CIEP state program approval process, the EPP will continue its focus on developing and implementing a QAS using both proprietary assessments and EPP-created assessments. The EPP has successfully used the Lawshe method to determine content validity, and anticipates review of pilot data for implementation of the Danielson Teacher Observation Tool used in the 2018-19 academic year. Further, faculty are engaged in developing discipline-specific addenda for the internship instrument which will require methods to assure validity and reliability in the upcoming 2019-20 academic year.
Data measures often reflect a specific licensure requirement or specific professional knowledge or skills expected of candidates. These may be state-specific requirements or may be requirements established by the EPP. For example, the 2.75 cumulative GPA is both an admission and licensure requirement established by the State Board of Education while the required 2.75 GPA requirement for early childhood and elementary licensure candidates in the 48 required content-specific areas (e.g., 4x12) is a requirement established and monitored by the EPP.
Continuous Improvement
5.3 The EPP regularly and systematically assesses performance against its goals and relevant standards, tracks results over time, tests innovations and the effects of selection criteria on subsequent progress and completion, and uses results to improve program elements and processes.
The opportunity to imagine and create initial licensure programs and participate in the pilot implementation of the CIEP state program approval process has been instructive and rewarding. The challenge to design key assessments, scoring criteria or rubrics and courses with the end in mind, which are aligned to standards with meaningful field and clinical experiences imbedded, was embraced by the faculty even before candidates were enrolled in the program. The EPP is now poised to begin routinely tracking and analyzing data as candidates enroll in the newly created programs. In the spirit of continuous improvement, testing innovations is inherent within the College. ED 360, a practicum course for early childhood and elementary licensure students, was added to allow additional engagement in classroom instruction and interaction with learners in an intensive prolonged experience prior to internship. Similarly, the addition of a second methods course for secondary education candidates has been proposed in curricular revisions submitted in the CIEP proposals and the EPP anticipates tracking and testing that innovation upon approval in the 2019-20 academic year. Another recent innovation which is currently being tested in the 2018-19 academic year is use of a standard lesson plan format in all subject-specific methods courses to provide consistency, align with edTPA, and reflect best practices. The lesson plan format was collaboratively created by EPP faculty in the 2017-18 academic year as part of the CIEP state program approval process.
Faculty members are routinely and systematically assessed each semester by their students using the UAH Student Information Evaluation system, and data are reported at the individual, departmental, and College level by the Office of Institutional Research and Assessment to inform professional development and annual faculty evaluations. Program faculty continuously engage in dialogue, reflection, and adjust curriculum, candidate and program outcome measures, and key assessments in an effort to build a robust and rigorous educator preparation program. Additionally, candidate data is gathered, reported, and analyzed for CIEP state program reports, CAEP annual reports, Title II Federal Reports, University of Alabama Board of Trustees reports, licensure requirements and other measures.
The EPP utilizes TK20 as its data collection and analysis platform, along with data stored in the University’s Student Information System (Banner), and spreadsheets and data tables created by the EPP. Data are collected on candidates, cooperating teachers, faculty, GPA, licensure requirements and key assessments. With the advent of new reporting mechanisms established by our most recent CIEP submission, the entire data collection system will be “overhauled” in the Summer 2019 to support new reporting initiatives after an individual is hired for the newly approved data analyst position.
5.4 The EPP provides measures of completer impact that are analyzed, shared, and acted upon in decision-making related to programs, resource allocation, and future direction.
While the Alabama State Department of Education does not currently provide completer data to its EPPs, UAH has initiated conversations with each of its major P-12 partner LEAs to determine what measures of completer impact might be available. As outlined in Standard 4, the current measures are limited, but each LEA has offered to provide whatever data is currently available. Alabama anticipates administration of a new statewide assessment system, including formative and summative testing, which is currently being developed by the contracted agency, Data Recognition Corporation (DRC). The contract was approved by the Alabama State Board of Education in July 2018. The new test will be the first permanent one since the State Board of Education abruptly voted to stop using the ACT Aspire in June 2017 without a replacement identified. The EPP is hopeful the resulting assessment data will be disseminated or shared by LEAs or the Alabama State Department of Education as a measure of completer impact. Current measures of completer impact include first-year teacher survey data, employer and completer survey data, and case studies and focus groups of completers. Each LEA provides available data, such as DIBELS or STAR Reading and Math assessment data, but these are limited only to elementary education completers.
The EPP will continue to monitor overall EPP enrollment numbers to determine if data indicate a decline as is evident in national and statewide data. Most recently, faculty and staff have noted potential candidates are experiencing difficulty passing the Praxis CORE mathematics subtest which is now required for admission to Alabama educator preparation programs. With the growth in student enrollment and addition of new programs, resource allocation and future directions are routine topics. The College anticipates gaining additional physical space in the 2020-21 academic year when Roberts Hall 4th floor becomes available after the Morton Hall renovation is completed. As student enrollment grows and new licensure program options are added, faculty and staff needs are considered and requests made.
The edTPA became a mandatory licensure requirement in Alabama in Fall 2018 and increased qualifying scores for the Praxis II content exams also became effective. Candidate data for the fall 2018 and spring 2019 semesters will be shared with the appropriate faculty members in the content-specific disciplines as well as EPP faculty to identify successes and opportunities for continued growth and needed supports.
5.5 The EPP assures that appropriate stakeholders are involved in program evaluation and improvement.
Because UAH College of Education is a relatively new academic unit on the campus, engaging appropriate stakeholders often involves scheduling regular meetings with faculty in the College of Arts, Humanities and Social Sciences and College of Science to discuss programs, address concerns, and identify new initiatives. Academic Affairs hosts bi-weekly meetings of the Deans’ Council, monthly meetings with the Associate Deans’ Council, and bi-weekly meetings with Academic Advisors. The purpose of these meetings are to facilitate university-wide dialogue. The EPP has other direct meetings with specific Offices and Departments within the university as needed when determining any changes that need to be made to the curriculum or data systems. For example, with the adoption of TK20, multiple meetings and interactions occurred with OIT staff to coordinate and test data pushes from the Banner Student Information System to TK20. Similarly, ongoing coordination with UAH Registrar and Records Office has facilitated creation of appropriate degree, major and concentration codes as new licensure options are added.
The Teacher Education Advisory Council (TEAC) meets once each semester, and its members include central office personnel from the LEAs, completers, administrators and teachers. The Dean and faculty regularly meet with other stakeholders, such as Alabama Institute for the Deaf and Blind (AIDB) representatives, ARI and AMSTI personnel, CAPNA administrators (Head Start/Early Head Start), Office of Early Learning (State Pre-K), United Way of Madison County (Education Task Force) and other community agencies and stakeholders. The Coordinator of Field and Clinical Experiences and Dean meet each semester with each of the partnering P-12 school district superintendents and central office administrators to discuss new initiatives or opportunities, learn about local district needs, or share data.
SUMMARY OF EVIDENCE AND SUPPORTING DOCUMENTATION
EVIDENCE 5 UAH Quality Assurance Plan
Summary
In summary, both initial and advanced programs have identified multiple methods of performance assessment to ensure completers are effectively prepared and equipped to positively impact P-12 student learning in classroom settings and beyond. UAH completers enter the profession with an awareness of diversity within schools, the impact of technology on education, a strong knowledge base concerning teaching and learning, and a foundational belief and commitment to ensure all students will learn. As the EPP continues to grow, it seeks to intentionally design meaningful assessment systems and collect quality data on candidates, faculty, and programs to inform continuous improvement and quality assurance. When possible, the EPP considers all evaluation/assessment requirements as it designs a comprehensive approach to data collection and analysis to inform continuous quality improvement of programs while also responding to institutional, state, national, and other reporting entities. Collected data are regularly reviewed and analyzed to improve candidate performance, faculty performance, and program operations.
The EPP will continue to review and revise its data collection to assess program standards and outcomes in a way that will facilitate and support continuous improvement and provision of excellent educator preparation programs. The EPP has planned activities and timelines to assist in keeping it aligned with institution, state, and national standards as it further develops and refines its QAS.
