Designing a Functional Assessment System
Note to readers: I’m writing this blog serially for the purpose of describing the design of the 8 interacting systems of our Integrated Science Program for ages 13-20. It is possible for readers to enjoy a post as a stand-alone piece, but if you feel as though you want more context, previous posts might prove helpful. Each post is only 3 pages (see downloadable PDF) to avoid redundancy.
Recrafting the Messages
I chose the words for this post’s title carefully. I truly believe that 1) widespread conventional secondary assessment is not a product of intentional design; 2) cobbled-together reactions to grading policies, exam-based courses, and graduation requirements do not resemble anything functional; and 3) the processes, practices and loops experienced by teachers and students do not, and never have, comprised a real system.
It has been quite invigorating to turn the tables on each of these elements and design a functional assessment system from scratch.
The collection of structures and practices that most high school and university students experience as ‘assessment’ convey some pretty terrible messages about academic performance.
Masked as desirable attributes, the following terms flow freely from educators’ lips. Rarely are the lived messages articulated, so I define them below:
- Accountability is the relationship between the student and a very narrow set of task types and formats, grading policies, and subjective interpretation of the teacher, which may change with each new unit as the content changes. Students are accountable to the exam publisher or teacher, and their achievement is wholly dependent on these entities. In this way, the most accountable students are often the least autonomous learners.
- Achievement is a status, rank or grade, often earned on high-stakes exams, which may take the form of end-of-unit tests. Students have one opportunity per a given set of content or chapters to determine their level of achievement, which fixes the status, rank or grade. (Re-takes, test corrections, and other similar policies do not alter this definition—in fact they reinforce the message that the demonstration of content understanding takes place within a set timeframe).
- Studying is memorizing, and is done just before the need to regurgitate information bytes, parrot a published author’s or the teacher’s own reasoning/analysis, or compare definitions.
- A good student is one who studies well, achieves status, and exhibits strong accountability (dependence).
It was obvious that this collection of anxiety-provoking messages needed careful recrafting.
Any system behaves in a way that reflects what its designers value. The system itself is the messenger, not the stakeholders within it.
If students cannot establish accountability as defined above, if they fail time and again, or if they are unable to study well, teachers cannot console them with contradictory messages.
Students know better.
The lived experience of the stakeholders represents the values of the system.
For me, the most important part of our assessment system design was the identification of these foundational values.
I am proud that the central tenets of our assessment system silence messages of ‘performance by pain’ and replace them with more encouraging, equitable messages.
Messages and Mechanisms
Inside systems, messages are sent through mechanisms: the structures, processes and feedback loops that make them tick.
In this way, systems communicate through actions, which is probably why I am magnetically attracted to them. Actions are my preferred means of communication, too.
In this section I will articulate the main messages of our assessment system and the general mechanisms we use to deliver those messages. I’ll maintain the assumption that readers already have a feel for the ‘why’ behind the messages: why they are desirable in 2020, why they align with what the field has learned from the neuroscience of teen brains, why they improve upon our conventional practice.
I invite questions and comments below the post, so feel free to question this assumption and I’ll happily respond. All of these message-mechanism pairings are important, and I present them in no particular order:
- Message: Patterns of positive performance yield success. No singular moment defines you.
- Mechanisms: Competencies are assessed multiple times over multiple years, are weighted equally, and remain stable, to establish a reliable learner profile. There are no high-stakes assessments, and thus, no place for good ’test-takers.’ The system demands these patterns for both skills and conceptual understandings, which is accomplished through a complete overhaul of current course content. Individual and collaborative practice is essential, and provided in class and through homework. This practice is rewarded with assessments that invite revision and polishing of past work.
- Message: You are accountable for your evolving performance, and multiple supports are available.
- Mechanisms: Students are assessed individually, and all expectations are clear to students prior to attempting any assessment. Longer performance tasks often involve group collaboration and sharing, which mimics real-world settings. No memorization is required, and students may use any resources and assistive supports available to them. Teachers are available to clarify expectations ‘in the moment’ and tasks are designed to be completed comfortably within class time (or according to accommodations). Assessments are spaced strategically over the course of a given module to prevent pile-up. Students who wish to exceed expectations are responsible for determining how they will do so; they propose these demonstrations to the teacher for prior approval. This also mimics professional settings, where those who succeed take their own initiative and do not depend on bosses or supervisors to dictate ways to excel.
- Message: Different combinations of strengths can prove successful.
- Mechanisms: Tasks are assessed according to small rubrics developed from master rubrics that remain stable over multiple years. Different combinations of competencies are assessed in each task. The system is truly equitable because diverse performance pathways over an extended period of time, such as a semester, can lead to an “A.” The message of equity is one of the key reasons for expanding the timeframes over which learner profiles are established.
We’ll let you know when the next post is ready for you!
- Message: Feedback and reflection nourish growth.
- Mechanisms: Some teachers complain that students do not use their comments and suggestions to improve. But current assessment is one-and-done, making ‘improvement’ only an abstract concept. Feedback is genuinely useful to students in our program. Due to the stability of the competencies over multiple years, students know there will be a ‘next time.’ This helps concretize messages such as “learn from your mistakes” and “try, try again.” Students benefit from multiple sources of feedback: their own, their peers, their teachers, and occasionally outside experts. Competencies for giving and receiving feedback are defined and practiced. Students enjoy opportunities for peer review prior to the final submission of complex tasks, and time for revision is allotted.
- Message: There are no redos. Performance on a given assessment stands.
- Mechanisms: As discussed in a previous post, redos and retakes (and summative assessments more generally) create substantial delays in every unit. With high-stakes, one-and-done assessment practices, redos help students recoup points but little else. The field of education has failed to realize that a crucial design flaw of grading practices is the result of flawed content organization within the course or program. By reorganizing multiple years of content, we made redos the norm and eliminated wasted time at the end of units and projects. But while singular performances do not define them, students must take each assessment seriously because the rating will not be altered. This notion is familiar to student athletes and artists: there is always another game, but a particular game is played only once.
- Message: Learning is a lifelong process. Expertise continues to evolve as conditions and constraints change.
- Mechanisms: There is no cue that signals mastery, on purpose. Rather, assessments continue to increase in complexity and difficulty, seemingly without end in sight. Ultimately, the program ends (or high school ends) because time spent in school is finite. Students then take stock of their strengths and identify areas for improvement. This the mark of an expert learner. In the words of Anne Meyer, David Rose and David Gordon, authors of Universal Design for Learning: Theory and Practice:
“Expert learners know that ongoing practice and deep engagement bring about development. They know that learning is continuous, that there is no arrival point at which one is finally ‘expert’ and no longer needs to practice.”
Motivation, Grit, Resilience, Creativity
The desirable attributes and dispositions noted by the title of this section are not explicitly assessed through our rubrics. This might shock or confuse some readers, but after careful consideration of various characterizations and definitions, it became clear that no task could do them justice and there was no need to force the issue.
We once again relied on the internal messaging delivered by the system. By combining the messages and mechanisms described above into a powerful experiential package, students persevere and become resilient because the very design of the assessment system demands it.
Student motivation is the grease that turns the gears. Those with grit give an extra push even when energy is low.
In a similar vein, and after leading teams of people to create many somethings from nothing, I firmly believe that student creativity should never be measured. It is heartbreaking to think that a rubric or other mechanism would send a message like: “Pretty good, kid. But not that creative.” Ugh. Our assessment system covers a variety of competencies that, in different combinations, are likely to produce creative works. But we will never use a ruler against which degrees of creativity are determined.
Creativity is fostered through the content organization and instructional approaches of the designers, and within a safe classroom community facilitated by teachers and students. Designs that provide long incubation periods, and seed non-linear connectivity, help students deliver their creative work at the ‘just-right’ moment in their developmental journey.
If anything, it’s the designers who should be assessed for our ability to create the conditions within which student creativity can thrive. Some things shouldn’t be assessed, and it’s important to know when and where to draw the line.
Thanks for reading,