Transparent Design: Criteria for Success

The final step to creating a transparent design is to be explicit about the criteria, or standard by which the activity or assignment is judged or evaluated.  How will you know if the students “got it” and can you be explicit about how they will be evaluated?  Often, this will come in the form of a rubric, which is a scoring tool that describes expectations and outcomes.  However, there is a lot of flexibility in how you can determine the criteria for success.  In some situations, you will want a direct measure, such as a formal rubric, but in others, a more flexible approach will be appropriate.

The authors of Transparent Design in Higher Education Teaching and Leadership suggest providing students with multiple examples of real-world work in the discipline to model a successful outcome.  They found that the examples, coupled with a discussion of the criteria, was beneficial in that it allowed students to practice evaluating their peers’ work and then to apply the same technique to their own work (39).  For example, in the materials you are reading right now, each section asks you to compare a real-world example of an assignment that is more and less transparent.  The process of comparing and contrasting builds a foundation for evaluating your own work.

What are criteria for success?

Criteria for success are the ways by which you measure student learning.  They can fall into two categories: direct, or formal measures and indirect, or informal measures. There are many ways to conduct direct, formal measures of student learning.  For example, standardized tests are frequently used to directly measure learning in a formal way by designing consistent conditions, questions, interpretations, and scoring so that each student has the same experience.  Examining student work in this manner is a more rigid assessment that is often limited in giving feedback for improvement, among other things.  An indirect, or informal measurement may come from student surveys, focus groups, and reflection essays that ask questions based on your learning outcomes.  No one way is perfect.  Creating a rubric is a transparent design technique that is explicit about the criteria for success and allows for the student to understand how they know if they were successful.

Rubric Creation in 4 Steps

In Introduction to Rubrics, Stevens and Levi (2013) describe the basic definition of a rubric as being “a scoring tool that lays out the specific expectations for an assignment…(by) dividing an assignment into its component parts and providing a detailed description of what constitutes acceptable or unacceptable levels of performance for each of those parts.  Rubric creation can be broken down into 4 steps; reflect, list, group, and create.

First, reflect on your outcomes.

Using a piece of scrap paper, or your favorite recording device, brainstorm answers to the following prompts:

• What does a successful learning experience look like?
• How would you describe a “mission accomplished” scenario?
• What specific outcomes do you want students to exhibit?
• What skills should they take away?
• What does good instruction look like?
• What does learning/success look like?

In this step, consider practices, behaviors, strategies, and habits of mind.  You might rely back on any learning outcomes you’ve recorded or your purpose statement to gather ideas.

Then, reflect on these next questions:

• What are your expectations of students?
• What do the best students do with this assignment?
• What evidence can students provide that would demonstrate their learning?

Second, create a list.

Using your reflections, create a pile of post-its listing the practices, behaviors, strategies, habits of mind, and criteria that demonstrate learning.  As you create your list, consider the final reflection question: what evidence can students provide to show they have accomplished what you thought they should?  For each post-it, record only one idea so that you can be flexible and move them around in the next step.  It’s okay if this step looks a little messy, unorganized, and even repetitious because it will be refined as you go through the process.  As you create your list of post-its, consider adding a description of what the highest level of performance you would expect.

Third, group the post-its together by items you think are related.

Assign labels of 1-3 words that describe the content of each post-it grouping.  These labels will become the criteria for your rubric.  The Rails on Track, or Rubric Assessment of Information Literacy Skills, website has some helpful verbiage under their Categories heading.

The fourth and final step is to create the actual rubric by transferring the labels and groupings to a grid.

There are a lot of rubric templates available on the web, and they all follow the same basic rules.  The criteria for success are placed in the left column, while the top row is used to describe the different performance levels, or expectations.  Start with the 3 basic performance levels of Introductory, Developing, and Mastery, and expand them if necessary.  A helpful strategy here is to start with the mastery level, move to the introductory level, and then determine what are the stages that mark the evolution from introductory to mastery and use that for the developing level.  The University of Southern Maine has many examples of rubrics you can review and use as a starting point.

As you develop your rubric, reflect on how a student would read and interpret the information.  Is it written in student friendly language so that it is explicitly understood what is expected of them?  Is the rubric short, clear, and free of jargon?  Are the performance expectations consistent across performance levels, or do you need to apply parallel language?  Are the lower performance levels stated in a way that inspires students to do better, but not have negative feelings of failure?  Finally, is there a balance of quality versus quantity that motivates students in the right direction?

Other Criteria for Success

You may determine that a full rubric doesn’t fit your needs.  There are other, more flexible, ways to determine success in student learning outside of the strict rules of a rubric.  For example, if your purpose is to build confidence in your students while they use a certain tool, a rubric will not be the best tool to determine success.  Instead, you may opt for a Likert scale, or a reflective essay that asks students to write about how they developed their confidence.  Another scenario to consider is asking students to find an article relevant to their topic with the criteria for success being simply that they found something. Keep in mind there are many levels of assessment and therefore many ways to determine success.  The goal is that students will have a clear understanding of what those measures are, be they formal or informal, that those measures be clearly stated, and that when questions arise, there is a place to refer to for those answers.  When questions do arise, return to your materials and add clarifying details.