The policy paper, Holding Teacher Preparation Accountable, of the National Education Policy Center (NEPC), offers a researchers’ perspective on teacher preparation accountability efforts employed by the U. S. Department of Education, CAEP, NCTQ, and edTPA. It asserts “claims” for each of these accountability systems and describes the deficiencies in research support for those claims. The authors conclude that the “public summative evaluations” fail to “provide information useful for program improvement.” They contend that the Department of Education, CAEP, and NCTQ evaluations do not “acknowledge that inequality is rooted in and sustained by much larger, longstanding, and systemic societal inequalities.” In what follows, we comment on the observations offered in this paper, from the perspective of the organization that accredits teacher preparation programs, the Council for the Accreditation of Educator Preparation (CAEP).
The NEPC paper includes numerous points with which CAEP concurs. We agree that, “there is no direct evidence about the efficacy of CAEP national profession-managed accreditation.” This calendar year, 2016, is the first year when all providers seeking accreditation or re-accreditation must use the new CAEP standards. At this point (April 2016), no providers have undertaken the entire accreditation process—from CAEP Handbook guidelines and CAEP-trained reviewers through final Accreditation Council decisions. CAEP’s stance needs to be that we believe the combination of standards and evidence that comprise CAEP’s evidence-informed accreditation process will have a positive impact on teacher preparation—but we cannot yet point to places that exemplify that result.
Like others, we have long heard from researchers that teachers are the most significant in-school influence on student learning. For example in the frequently cited Darling-Hammond and Bransford 2005 National Academy of Education report, Preparing Teachers for a Changing World, we find:
“Although many have assumed that students’ backgrounds—such as income, parent education, and other family factors—are the major reasons for wide differences in student achievement, some studies have found that the quality of teachers can have an effect at least as large” (p. 14). “Many kinds of teacher knowledge and experience appear to contribute to this effect, including teachers’ (1) general academic and verbal ability, (2) subject matter knowledge, (3) knowledge about teaching and learning, (4) teaching experience, and (5) the set of qualifications measured by teacher certification . . .” (p. 15).
In light of these conclusions, and other empirical evidence (e.g., Effects of Teachers’ Mathematical Knowledge; How and Why do Teacher Credentials Matter for Student Achievement?; and In Most Schools, Teacher Quality Matters Most), it was surprising to see teachers dismissed in the NEPC paper as contributing “a relatively limited portion of overall variance in student achievement.”
The first assertion NEPC makes is that CAEP claims:
“...that a national accreditation system developed and managed by the profession is an effective mechanism for raising standards and thus improving the quality of the profession, defined primarily as graduates’ impact on students’ learning.” (Emphasis added to the original.)
Based on the experience of CAEP’s NCATE and TEAC predecessors, accreditation does have strong leverage. As data from TEAC programs show, Educator Preparation Providers (EPPs) want to respond to accreditation standards and to offer rigorous evidence for claims that their graduates are high quality teachers. That may be because they are located in one of the 19 states that require accreditation of some or all preparation providers; or because they prefer to seek the same accreditation recognition that their colleagues, peers, and competitors are receiving; or because they see an intrinsic value in the process of self-evaluation that is the hallmark of any accreditation procedure.
We agree that there should be explicit attention to P-12 student learning as one of the ways to judge effectiveness of teacher preparation. While TEAC and NCATE have been moving toward program outcomes, that evidence was not consistently and explicitly required in teacher education accreditation prior to CAEP. CAEP Standards require evidence of graduates’ successful teaching, and also of P-12 students’ learning, as an ultimate measure for evaluation of teacher education programs. But the NEPC assertion, quoted above, represents a narrow characterization of CAEP’s use of research to frame its standards, including the standard on evidence of P-12 student learning. CAEP does not agree with the NEPC statement that its “. . . policies themselves are not evidence-based.” Principal among the NEPC omissions in their citations of evidence are the following:
• The 2010 National Research Council (NRC) report, Preparing Teachers: Building Evidence for Sound Policy, concluded that “Existing research provides some guidance on three aspects of teacher preparation that are likely to have the strongest effects (on outcomes for students): content knowledge, field experience, and the quality of teacher candidates.” The Council report recommended these three large topics as appropriate starting points for additional research into how specific characteristics of teacher preparation programs improve student outcomes. CAEP used the NRC recommendations as a framework for three of its standards.
• A 2010 NCATE report, Transforming Teacher Education Through Clinical Practice: A National Strategy to Prepare Effective Teachers, recommended collaborative partnerships between providers, schools, and school districts as a path toward stronger clinical experiences. These ideas, that drew on numerous research studies, are embedded in the CAEP clinical experience standard.
• The CAEP standards incorporate, in full, the InTASC (2011) Model Core Teacher Standards created by educators and researchers for the Council of Chief State School Officers. These standards address content, including use of content in college and career-ready instruction; instructional practice; professional responsibility; and the learner and learning, including learner development and learning differences across diverse cultures and communities. Learners need supportive and safe learning environments to thrive.
• The Baldrige Education Criteria for Performance Excellence, and the concepts underlying “improvement research” as conceived by the Carnegie Foundation for the Advancement of Teaching, stimulated CAEP’s standards on quality assurance and continuous improvement and also on EPP monitoring of completer performance results.
While the NEPC paper finds that use of value-added measures has been “controversial,” the authors conclude that there should not be “too much weight” placed on value-added assessments. The report makes no mention of the section of CAEP Evidence Guide that addresses this topic, with extensive advice from psychometricians and educators, including counsel from the 2013 American Psychological Association report, Assessing and Evaluating Teacher Preparation Programs, and the 2013 National Academy of Education, Evaluation of Teacher Preparation Programs: Purposes, Methods, and Policy Options. There are three important points to be noted:
• NEPC is incorrect in asserting that CAEP “requires the use of statewide standardized achievement tests in determining teacher preparation program effectiveness.” Component 4.1 of the CAEP Standards requires EPPs to include, in their self-study reports:
“all available growth measures (including value-added measures, student-growth percentiles, and student learning and development objectives) required by the state for its teachers and available to educator preparation providers.”
However, EPPs are not required to use that information as evidence if they believe that other data are superior. The requirement is that EPPs demonstrate they are successfully preparing teachers who can help all children learn—i.e., “that program completers contribute to an expected level of student-learning growth.”
• If EPPs do include state student achievement data as evidence in their self-study reports, then CAEP expects that they will be well informed about how the data have been developed and reported. For example, EPPs would indicate their understanding, inter alia, of:
o “the psychometric soundness of the assessments taken by students and the alignment of those assessments with the state’s curriculum”;
o “the proportion of the EPP’s completers for whom P-12 student growth measures are available and the extent to which the reported completers are representative of all completers from the EPP programs”;
o “the degree of attrition from prior to current performance measures of P-12 students that would influence interpretations of the data”;
o “the manner by which pupil data are linked with teachers to judge the accuracy of the associated teacher data (scores should only be used for P-12 students who are actually taught by the EPP’s completers)”; and
o “the state’s practice in flagging possible biases or misrepresentation in the results.” (See CAEP Evidence Guide, pp. 31-34).
• For EPPs that do not have access to state data, or choose to build their own evidence of student learning and development, CAEP has listed the following examples of potential sources in its Handbook:
o Provider-conducted case studies of completers,
o Completer-conducted action research,
o Descriptions of partnerships with individual schools or districts (that link student learning information with teachers),
o Description of methods and development of any assessment used, and
o Use of focus groups, blogs, electronic journals, interviews, and other evidence.
Regarding the evidence base of CAEP standards, we agree that the field needs a great deal more research that rigorously studies what works in teacher preparation. We have instituted a CAEP Research Committee to try to address these lacunae. We encourage higher education researchers to conduct more of this critical empirical work, and urge funders to provide financial support to build the knowledge base that will help programs prepare the best teachers for all children.
The second assertion posited by NEPC is that CAEP claims:
“[in] the process of meeting standards for accreditation, programs will engage in ‘continuous improvement and innovation’ based on reliable and valid evidence about outcomes; this process will enhance teacher education and teaching quality.”
CAEP has no objection to this statement. We believe that the most important aspect of the CAEP process is that EPPs engage in continuous improvement and innovation to enhance teaching and student learning.
The third assertion from NEPC is that:
“An accreditor-created massive database containing systematically collected performance data will provide usable consumer information, thus restoring policymakers’ and the public’s trust in the teacher education profession.” (Emphasis added to the original.)
CAEP does aspire to develop a database about teacher preparation, over time. With the collaboration of EPPs and states, such a database would be comprised of information from self-study reports that describe, in commonly defined measures, aspects of preparation courses, and clinical and other experiences, as well as CAEP’s “8 annual reporting measures.” The annual measures include:
• “Impact” measures—
o Impact on P-12 student learning,
o teacher observation evaluations and/or student perception surveys,
o employer satisfaction with preparation and completer career progression, and
o completer satisfaction with preparation.
• “Outcome” measures—
o completer rates,
o licensure rates,
o employment rates, and
o student loan default and other consumer information.
Such a data resource would be available for research use, but the intent is primarily to support continuous improvement efforts of EPPs through CAEP reports of normative information, comparisons among institutions serving different missions and perhaps different geographic areas, and, eventually, performance benchmarks. The planned CAEP data resource would allow peer comparisons that associate commonly defined measures of preparation characteristics (e.g., qualities of clinical experiences or course requirements on assessment literacy) with the results of preparation. Currently, an individual EPP can associate its own measures of preparation characteristics with its own measures of results—but that is a significant limitation, and a needless one when so much potentially usable information is near at hand. Whether those potential uses, when put into place, result in achievement of NEPC’s assertion—to restore “policymakers’ and the public’s trust in the teacher education profession”—is the authors’ conjecture, not a claim that CAEP would advance.
CAEP shares the view that inequality “is rooted in and sustained by much larger, longstanding, and systemic societal inequalities,” and that teachers and schools are not alone the “solution to the equity problem.” But what is to be inferred from this statement? NEPC says that “Policymakers must acknowledge and address the multiple factors—in addition to teacher quality—that influence student outcomes.” We agree. While society addresses the “larger, longstanding, and systemic societal inequalities,” however, is the profession of teacher preparation to do nothing about holding itself accountable?
Fortunately, there is evidence that some school districts are reaching out to other local governmental units so that P-12 students can access health services, early childhood programs, food programs, and have safe places to be both during school hours and at other times of the day. These districts are not waiting for NEPC’s recommended policymaker acknowledgement and action. CAEP should not wait, either.
NEPC reminds us that researchers prefer to be explicit about what research tells us and what it may not. And it reminds us, as well—because some interpretations are “controversial”—that different researchers frequently reach different conclusions about what the research tells us. Abundant experience makes it clear that social science will not give us conclusive research on everything we need to know about teacher preparation. However, we believe we need to make the best choices, informed with the best information that we can marshal to enhance preparation of teachers.
We agree with NEPC’s recommendation that teacher preparation programs should, “Consider… teacher candidates’ performance (defined as knowledge, practice, commitments, and professional judgment as they play out in classrooms and schools), and students’ learning (defined as academic learning, social/emotional learning, moral/ethical development, and preparation for participation in democratic society).” CAEP looks forward to working with the many teacher preparation programs that aspire to achieve those goals, and to sharing the responsibility for achieving them.
CAEP is using available research, has done so in responsible ways, and will continue to adjust as new knowledge is accumulated. Moreover, CAEP has a responsibility to conduct its accreditation functions in ways that balance its goals for accountability to the public and for program improvement. Parents, the public, and policymakers care deeply about P-12 student learning and they expect that educators prioritize child and youth learning and development. If teacher preparation programs fail in their important fiduciary responsibility to graduate teachers who are ready to teach all children, then students are denied their basic civil rights. CAEP will continue to play the role that it can in helping to enhance the effectiveness of teachers preparing to teach in America’s increasingly diverse classrooms.