Making Competency-based Assessment Work: Macro-maneuvers
Know When You're Helping or Hurting
About six or seven years ago ‘design thinking’ found its way into the educator’s vernacular.
The d.school at my alma mater began to offer short professional development experiences to K12 educators to get people thinking like designers, using a process akin to one an engineer might use to solve a problem. Entire schools have been built around design thinking, and many classroom teachers now discuss their classroom experimentation and reflective practice in these terms.
We initially thought this small educational movement was terrific, but now we are more wary.
The scrawled Post-its on walls, half-used toolkits, notes from protocols, and completed graphic organizers have landed on the same island as other education initiatives: the place where products of ’discovery’ and ’ideation’ phases go to die.
I fear that in some places, competency-based assessment has not enjoyed leadership through the entire design process.
Indeed, you can’t fund, study and consult your way to new assessment models.
Someone has got to build the thing—and it’s not fair to teachers who already have full-time jobs and no experience in these uncharted territories.
Design work is difficult and time-consuming, even for people with loads of classroom experience and strong content backgrounds.
Yet this is what the field of education does: celebrate the big ideas, chant the jargon, and shove execution into a few PD days led by a consultant who has never designed anything more than PowerPoint slides and curriculum planning templates.
As someone who has delighted in the toils of building the thing, I’d like to share some key aspects of learning systems design that have kept our work moving forward constructively:
Remove as many constraints as possible for your design team.
There are often fewer constraints than people believe. There are many perceived constraints, and past precedents, on which people fixate. Interestingly, recent research shows that humans rarely think about removal or subtraction as a worthy problem-solving strategy. I can assure you, it is usually the best way forward.
Take time in the beginning to separate the wheat from the chaff (and document it).
Then avoid the temptation to produce plans, blueprints, frameworks, elements and checklists in the name of “guiding innovation.” Ideation phases often produce such things, and administrators believe they help distinguish their school’s work.
But if educators must use these lists of elements in their designs, they are nothing more than additional constraints.
Too many schools use academic frameworks as marketing tools before substantive programs are built, which, in turn, can cause communication or design problems later.
Design assessment criteria, performance tasks, and surrounding instruction together.
Designing the rubrics first and directing teachers to use them is less than ideal, and likely will cause teacher frustration (or screaming, or worse).
I realize that various non-profit and academic groups often share exemplar tasks, skills rubrics, or capstones for a single course or year.
Yet I always find myself asking, “How much instructional time is needed to prepare students to be successful, and how does this fit with other curricular features?” No one ever seems to know, which makes their examples minimally useful.
Remember that classrooms and those who inhabit them are complex and diverse.
We use ~500 variables in order to produce easy-to-use, cleanly-designed materials for daily classroom use. We achieved this by designing everything together, and by iterating multiple variables together.
The best systems in nature are those that co-evolve.
If you want different academic outcomes, take design work seriously.
I’m not sure when administrators and policymakers decided that innovative learning systems could be designed using the same methods used to create lecture-textbook-syllabus courses.
Learning systems design work requires a dedicated, full-time, year-round team.
Building one high-quality performance task can take nearly 100 hours of work, particularly for later years in our program when tasks demand multiple competencies that must be balanced with practice given through other tasks.
It is not fair to charge curriculum committees that meet for one week during the summer, and weekly for one hour during the school year, with major course overhauls.
If you want to liberate your teachers to modify and enhance curricula (the fun part), collaborate with an organization like ours who can see the big picture and confront the devil in the details while teachers tend to their full-time jobs inside classrooms.
Make Friends with Time
Here’s the thing about competency-based assessment: it takes time to deepen conceptual understanding, to practice skills, and to foster dispositions. So we can’t rush students to demonstrate proficiency before they are ready. But since time is one of the biggest design constraints educators face, competency-based assessment can prove very difficult to enact.
As discussed in a previous post, adult humans have a somewhat devastating relationship with time.
And adult educators love to tackle high school and higher ed. course design very academically, in outline form, as if they were writing a research paper or a textbook.
Content gets “chunked” within courses, and time gets “chunked” along with it. Start with a big topic or question, break it into component parts, lecture the hell out of it, and give a test at the end. In the lecture-textbook-syllabus model, the time it takes to finish the lectures and 1-2 substantive in-class activities is the time allotted to instruction.
This structure does not work in a competency-based model.
Since time is a major constraint, it is important to find as much of it as possible. This is why we focus on formal, required classes rather than electives.
We broke with past precedent to find the time needed to build an equitable assessment system for diverse learners.
Here are the key moves we made to befriend time on behalf of teacher sanity and student growth:
We removed single-year course structures. High school content area teachers are already pressed to finish the required curriculum in one year, and most don’t.
To establish a reasonable set of data points across enough competencies to make most educators happy, you need at least two years.
This is why the minimum program implementation for Integrated Science is two years, and we strongly recommend three– or four-year progressions. It’s successful even with a 15% annual student attrition or arrival rate, since transience is a reality.
We decoupled mastery from end-of-unit timeframes. In other words, students are not assessed on all concepts or skills practiced in a given unit by the end of it.
This is the shift that classrooms feel viscerally.
Presently, courses are designed through one-and-done units or projects in a serial fashion—and no, PBL does not correct this problem.
We figured out that you can keep serial units/projects as content containers, as long as you remember the role that content plays: as a foil for ongoing competency practice, to include conceptual understanding.
Content progressions interface with assessment progressions but they do so over many modules rather than one at a time.
As with many strong relationships, it is important for content and assessment to walk side by side without one controlling the other.
We removed end-of-unit summative assessments, midterms and final exams.
This was an obvious next step after the decoupling maneuver.
We included summative assessments in the professional development phase of our work from 2002-2008. During this period we observed that the assessments, no matter how efficiently designed, consumed far more time than we expected.
And the problems weren’t mechanical. They were psychological in nature.
High school tests send teachers and students into a different state of being, somehow.
So when we discovered that they weren’t truly necessary, we also discovered that the accompanying stress, anxiety and delays could be eliminated as well.
We honored the role sleep plays in learning.
Some schools have restructured time by cramming a year-long course into a single semester, often by doubling the time students spend in class on a weekly basis. This might work in a credit-bearing course model with traditional tests, where completion is prioritized over learning.
But in order to align with well-known neurological research on sleep and learning, we need to expand timeframes, not shrink them.
Sleep helps solidify and organize the various inputs a teen brain processes during the day, and sleep supports a long-term competency-based assessment system for this reason.
Teens also need to get enough sleep on a daily basis, which is why we removed studying from our assessment system.
After all, studying is just code for memorization.
Our students are allowed to use their resources for assessment tasks. They discover that consistent performance in class and during independent practice is the best path to success.
We’ll let you know when the next post is ready for you!
Built-in Accountability: Are Educators Ready?
We lived through the value-added teacher evaluation syndrome in the USA. We watched Charlotte Danielson’s body of work on teacher professional development contort into monetized multi-page checklists that principals used to evaluate one lesson.
Allow me to put forth another means of accountability: one that emanates from the design of the learning system itself.
Transparent accountability to oneself and one’s community forms a powerful nexus.
Our students are assessed individually in order to accumulate enough data points for each set of competencies cleanly and transparently. Many students find that the more data points they collect, the easier it is to succeed, which motivates them to attend class and complete work (and they need enough data to earn a quarterly rating).
Students are not dependent on the teacher’s direction to earn high marks. Rather, they proactively determine how they will attempt to exceed expectations, and discuss their proposal with the teacher in advance. As one student reflected, “You get out of it what you’re willing to put in.” That’s music to our ears.
For teachers, the multi-year progression demands their accountability to their students and peers. Preparing for class, delivering student feedback consistent with a stable set of competencies, and communicating with other teachers are imperative. In our lifelong learning model, the next year begins where the previous year concluded. This prevents gaps, and also encourages communication among teachers.
EduChange materials are Open Education Resources (OER) and everything may be modified. But if you decide to add or delete something substantive, you should share your rationale for these decisions. Teachers are most successful when they keep the lines of communication open, seek help from EduChange or peers when modifications get tricky, and include student voices in decisions with large implications.
After implementing with nearly 100 teachers across the world, we know that this accountability is novel, and palpable, for many.
Teachers experience agency but not anarchy, and collegiality rather than collective monologues.
No external evaluator is necessary to deliver these messages.
The system does that all by itself.