“In the traditional system, it can mistakenly feel more precise because we use mathematics to determine the grade. In the mastery-based system, we have to make sure we are as objective as possible – we have to be subjectively objective. We used to have teachers say that they wanted to give students who had worked hard the benefit of the doubt. Why is there any doubt? We need to have a system in which we can be confident of what students know.”
– Susan Bell, former Superintendent and David Prinstein, Principal, Windsor Locks Middle School, Windsor Locks School District, CT, 2016
Description
In competency-based systems, students advance upon demonstrated mastery of learning. In order to do so, those learning objectives must be clearly articulated and reliably understood by all. Moderation builds shared understanding of proficiency, and calibration creates consistency of grading practices to improve consistency in credentialing learning. Creating cross-district and cross-school clarity and consistency reduces variability in expectations. Systems of assessments are aligned with appropriate level of depth of knowledge as defined by the learning objectives.
Key Characteristics
- Valid and reliable. Districts and schools have accurate, standards-based definitions of proficiency. These definitions are transparent and available to all educators and students. Rubrics, examples of proficient student work and other tools are used to communicate proficiency.
- Authentic assessment. Systems of assessment are valid and reliable, and produce data that accurately assesses student mastery of standards. Assessment is also meaningful and valuable to the learning process by supporting reflection and guiding further instruction.
- Aligned to learning objectives. Systems of assessment are aligned to competencies and standards at the appropriate depth of knowledge.
- Assessment literacy. Teachers are supported in using different types of assessments and providing productive feedback to students. Teachers build capacity in assessing building blocks of learning, transferable skills and performance-based assessments.
- Moderated. Districts and schools have systems and processes to ensure consistency in the way that proficiency is understood across schools.
- Calibrated. Educators work together to ensure inter-rater reliability of grading of student work and assessments.
How Is Establishing Mechanisms to Ensure Consistency and Reliability Important to Quality?
Traditional education systems demonstrate high degrees of variability: they permit different understandings of what it means to be proficient between schools (higher-income communities often have higher expectations than lower-income communities), between educators (different definitions in every classroom or school) and between students (different definitions being applied to students, often based on their race, class and perceived ability). Many factors contribute to this variability, including educators working in isolation, A-F grading systems based on student behaviors, assignments and summative tests, biased educator perception and different expectations for students within and across schools. In these contexts, inequities are produced. Students are told they are proficient when they are not resulting in widening learning gaps. Neither students nor educators can access accurate information about what students know and can do to inform instructional decision-making. The results are many: each year teachers are challenged by the number of students with gaps in their knowledge from the previous year. Students without prerequisite knowledge and no avenue to build it become less engaged and motivation decreases. Students with high GPAs go off to college only to discover they need remediation, and parents and communities lose trust in the educational system.
By contrast, competency-based systems emphasize consistency and reliability. Rather than relying on seattime as a weak proxy for learning, competency-based systems develop structures to build confidence and transparency about student learning. Competency-based education systems value consistency and transparency as strategies that interrupt the replication of inequities. Quality and greater equity are rooted in evaluating student outcomes against a constant criterion—a standard with rubrics clearly outlining expectations for what evidence is needed for successful outcomes—rather than evaluating student outcomes against a single educator’s estimation of proficiency. Learning targets and proficiency determinations are transparent. Scoring proficiency is calibrated; educators work collaboratively to define what proficiency looks like using evidence of student work, use common rubrics and calibrated grading practices to increase inter-rater reliability of scoring. Student progress is measured based on outcomes demonstrating proficiency. The efforts of a few leading states to create proficiency-based diplomas is another strategic effort to create more consistency and confidence that students are mastering what they need to be successful in the future.
Creating consistency in teachers’ judgment of learning begins with the development of the common learning framework that identifies the learning targets, common rubrics for each performance level and example of proficient student work. From there several structures play key roles in creating consistency:
Balanced System of Assessments
Competency-based systems emphasize a balanced approach to assessment that drives powerful learning that leads toward common outcomes. Elements of a balanced system of assessment includes: strong emphasis on formative assessment for learning including productive feedback, multiple opportunities for students to reach proficiency, multiple measures used to determine proficiency, assessment aligned with depth of knowledge of learning targets including performance-based assessment and opportunities for students to pursue personalized strategies to provide evidence of learning.
Districts and schools integrate assessment and grading as part of the learning process: assessment illuminates what students need to know, provides students with low-stakes opportunities to practice and self-assess what they know throughout the learning cycle and develops feedback that students and educators can use to improve. The result is that students understand the role of assessment as meaningful to their learning. They see it as the doorway through which they are able to receive the feedback and differentiated instructional support to help them be successful. Assessment is the way teachers show they care for the student by wanting them to be successful, not something by which they are judged. Clear definitions and criteria to evaluate evidence of proficiency are core to a meaningful system of assessments. Validity refers to the degree to which assessments and evaluations measure what they are intended to measure (i.e., how well they are aligned with standards and curriculum). Reliability refers to the consistency and stability of results across student populations or across schools. Usability refers to how policymakers, school leaders and teachers make sense of and respond to assessment and evaluation results. Alignment of assessments and evaluations with standards and curriculum is crucial to usability.
In the most developed competency-based systems, summative assessments are organized to meet students where they are rather than based on pacing guidelines for covering grade-level standards. Students show evidence of learning or are assessed summatively after a teacher has determined that the student is proficient. Thus, summative assessments are designed to confirm proficiency as a form of quality control.
Assessment Literacy
Given the critical role assessment plays in the cycle of learning, competency-based systems invest in building assessment literacy throughout the districts and schools. Assessment literacy—the knowledge and skills to use the full range of types of assessment which are developmentally appropriate on behalf of helping students to learn—becomes a priority after the first stage of implementation. As districts and schools advance in implementation, attention to the system of and knowledge about appropriate assessments increases. Professional learning about assessment often includes attention to formative assessment including the use of learning progressions to better understand how students are solving problems. Student knowledge around self-assessment gains in importance. Districts and schools frequently invest in building the capacity and professional learning around assessment literacy, especially around performance-based assessment, if they do not yet have it integrated into their ongoing pre-service and in-service professional learning.
Moderation and Calibration
Two processes are critical for creating the consistency need for a high-quality, equitable competency-based system: moderation and calibration. Moderation is a process used to evaluate and improve comparability. The process involves having teachers (or others) work to develop a common understanding of varying levels of quality of student work. Calibration describes the process of creating consistent, shared understanding of what proficiency means for learning targets for specific levels of performance (or grade levels) and requires teachers to look at student work together. Moderation processes must take place within and across schools, and even across districts, to ensure that students are all held to high standards. Then, there is a need to calibrate the grading practices so that teachers can consistently determine proficiency and identify what students need to learn to reach proficiency. Calibration, like moderation, builds professional knowledge while also operating as a formal mechanism that ensures students are advancing upon mastery.
As schools begin to integrate rigorous deeper learning, moderation and calibration will be needed to help teachers consistently determine higher-order and transferable skills demonstrated through performance tasks, performance-based assessments, portfolios and capstone projects. In the future, it is likely that moderation processes will need to be expanded even further to support teachers in the process of understanding levels of development in the building blocks for learning such as metacognition, social and emotional skills, self-regulation and traits such as perseverance. Moderation processes can take place within schools, across schools and across districts in a state.
Proficiency-Based Diploma
Proficiency-based diplomas are being developed to create consistency in what students know and can do upon graduation. Essentially, the graduate profile drives alignment and also the requirements for graduation. When used as a high leverage policy, the introduction of a proficiency-based diploma can catalyze districts and schools to become more responsive to students so that they are fully supported in their learning starting in elementary school. However, if districts don’t make the necessary adjustments to ensure students are building mastery for all the critical learning objectives in the younger years, pressure builds at the high school level about how to respond to students with gaps in their learning within the four years, so that they can demonstrate mastery of all the graduation competencies.
It is by creating these structures that districts and schools can consistently know that students are learning, and credential learning authentically. The result is that teachers, students and parents can all have confidence that they know where a student is performing along the learning continuum (i.e., grade level) and growth (where they started and how they are progressing on their learner continuum).
Policies and Practices to Look For
- Structures and processes are in place to ensure that the instruction and assessments are fully aligned with the learning objectives and offer rich and frequent opportunities for students to perform at the highest possible depth of knowledge.
- Teachers engage in calibration or joint scoring of student work to ensure inter-rater reliability.
- Teacher-generated performance assessments are strengthened by engaging in task validation protocols.
- States, districts and schools establish moderation processes to ensure that levels of proficiency and mastery (application of the skills and knowledge) are aligned to state standards and shared among teachers.
- Professional learning communities seek to create consistency in determining learning. Teachers provide feedback to their colleagues if they credential students as reaching proficiency when they haven’t.
- Transparency in the learning cycle and grading provides feedback on student progress and is designed to recognize and monitor growth with improved consistency and reliability. Students are able to see examples of proficiency work on the walls of classrooms or in other resources.
- Districts and schools have mechanisms in place for quality assurance to ensure that variation is not creating situation of lower expectations for some students or students advancing without the opportunity to fully master skills.
Examples of Red Flags
Teachers are spending substantial time on unpacking standards and writing rubrics, without looking at student work to moderate their understanding. The development of the common learning framework with clear learning targets and rubrics can easily slip into a bureaucratic process rather than one focused on teaching and learning. Make sure teachers are spending time looking at student work, talking about what proficiency looks like, and building their assessment literacy. Manage refinements of documents on an annual basis so that it doesn’t take up too much of teachers’ precious time together. Great professional development can take place when teachers talk about student learning, instruction and assessment as they design and refine the learning continuum.
Standards-based grading is introduced too early without the structures for consistency in place. Many districts turn to standards-based grading too early in the process, often based on the misconception that by doing so they will be considered competency-based. The infrastructure of the learning framework to ensure consistency and mastery—aligned instruction and assessment, the mechanisms of moderation and calibration and flexibility for students to receive support when they need it—should all be in place before introducing grading practices organized around standards. Too often districts say they are doing standards-based grading with the intent to make sure every student fully masters the standards when they are actually using standards-referenced processes that provide feedback based on common standards without making the commitment to help every student achieve them. An additional risk is that students may only be receiving feedback based on grade level standards without attention to addressing gaps. Thus, students are not being held to same standards and false signals about student progress continue.
Students can tell you who are the “easy” educators and the “hard” educators in which the hard educators have expectations for students to master the knowledge and skills. In the traditional model, teachers have autonomy over grading and what they determine as proficient. Students know which teachers have high expectations and which ones don’t. The so-called “hard” teachers have high expectations and will make students stretch to receive a high grade. In competency-based systems all teachers should be “hard” holding high expectations for all students. High-achieving students in competency-based schools will often remark that they have to work harder because they are expected to demonstrate their learning, not just memorize for a test.
Source: Sturgis, C. & Casey K. (2018). Quality principles for competency-based education. Vienna, VA: iNACOL. Content in this book is licensed under a Creative Commons Attribution 4.0 international license.
