Fall 2021: More on Standards Based Grading

I'm in a book club with some colleagues this summer, and we are reading Grading for Equity by Joe Feldman. I've been meaning to write summaries of my thoughts after our assigned readings, but may wait to do a full book review. We're at Chapter 10 and it feels a bit of a cliff hanger; including any assignments or activities in a student's grade other than assessments of their understanding is inequitable. Our (seemingly) natural response has been to reject this as we have learned that high stakes assessments likely do more harm than good, especially for BIPOC students. The author has alluded to some solutions, which we will dive into in Chapters 11 (Practices that support hope and a growth mindset) and 12 (Practices that 'life the veil'). 

During our last meeting standards based grading came up, and most people seemed interested but intimated. In previous posts I've mentioned how I am balancing flipping my classroom and creating a standards based grading scheme that I'll test out behind the scenes for AY 21-22. Two concerns people had were creating the standards and levels, and the logistics of both recording student's progress and allowing for retakes of assessments. 

Creating the standards can absolutely be a difficult task, but for most textbooks there are standards available. This is absolutely becoming a requirement of the job for many faculty, the ability to create and align standards, and it would be nice to see more institutional and department support be the norm for those efforts. It can take quite a while to do this for any course, especially for courses that have more than five faculty teaching those courses.

For corequisite courses this is a bit trickier as you have to identify prerequisite topics that support the course-level standards. This took our department many, many months (if not years) and not something to be taken lightly. If you are thinking of doing something similar the Dana Center has great resources and tools to support faculty in this effort.

I'm finding that the levels for standards isn't as difficult, mostly because I know the course material, and have experience teaching so I know what areas most students struggle with. For example, to create the following levels I started with the Proficient level as meeting the Learning Objective (LO). From there I went up to Expert and included an additional aspect of this LO that I like students to be able to do, but it isn't necessary. After that I went to Developing, knowing that students struggle with the idea that a function can be defined only by a set of points, and that verbal descriptions of functions can be challenging. Lastly for Novice I knew that students struggle with functions that are not continuous, but I couldn't use that word so I decided on "for functions that are defined for all x."

Learning Objectives	Expert	Proficient	Developing	Novice
Determine the domain, range of a function.	The student can identify the domain and range of a function graphically, tabularly, and by a description, and do so for applications.	The student can identify the domain and range of a function graphically, tabularly, and by a description.	The student can identify the domain and range of a function graphically, but may struggle doing so tabularly, and by a description.	The student can identify the domain and range of functions graphically for functions that are defined for all x.

One thing that is really helping me get through writing these LOs and levels is the fact that I'm not going to be using them for grading this year, and I'll have time to refine them. I plan on using these LOs and standards to create my flipped materials and activities, but for AY 21-22 I won't use them for grades but will still record student progress as if I did. This would (hopefully) provide me feedback as to what standards should be adjusted or removed. I could also test sharing these levels with students, possibly using them for review exercises, editing any that students struggle with reading and understanding... In fact, creating a card sort from these would be pretty easy, save me time, provide good feedback, AND might help students review and understand terminology. 

The logistics aspect is absolutely a little difficult, and I plan on going back to the initial presentation I saw for ideas about how to share information about grades using these LOs and levels. One idea is to create a Canvas Group for each student. This way Canvas will generate a page for that group/student, and I can post their progress.

Another idea is to put it on the student. Give them a chart at the start of the term, explain how to fill it out, and make them responsible for them telling me what they need. Granted I'd have a spreadsheet of my own, but I would hope that by giving them the responsibility of maintaining a record of their progress AND grade the student would see how they are connected. 

In terms of assessments, my first thought is to copy Lewis and Ramos' plan; have a quiz where students can make their first attempt at showing mastery of the LOs for the week. After that have students try again. I'm resistant to using class time as I'll be flipping the course material and will likely need that time to help students. Using office hours sounds good, but maybe doing so in the classroom after class might be a better idea. (A colleague (Hi Hannah!) has been talking about having faculty volunteer time in a classroom to make a grassroots testing center. I like the idea but need to figure out the COVID-related logistics. It would certainly help me!)

I feel like I'm making good progress, but August is already here and I don't have ANY videos or activities made, and am only half-way through completing the LOs and levels. I feel like I have to make a solid effort towards flipping my class soon and get at least 50% of the videos and activities done in August or I might have to scrap it. I have previous homework assignments and materials I can use, but it'd be nice to have most of the course developed by then.

Fall 2021: Thinking about modalities

 A colleagues (If you're reading this, hi Allie!) has been asking some really good questions about how we'll return to campus, and what the experience of everyone teaching online might mean for our face-to-face (F2F) courses. In responding to one of her emails I started the following chart comparing different aspects of online, F2F, and flipped classes. I've since added a few more items. 

	Online	F2F	Flipped
Timed Assessment Logistics	Range of times, online, could be submitted on paper, some auto-evaluated and some faculty evaluated.	Specific time and place, on paper, there could be some variation in questions among students, faculty evaluated.	Same as F2F
Timed Assessment Questions	By necessity of online honesty could be more conceptual, but likely includes computational questions.	More computational, but could shift towards more conceptual if more online assessments include them.	Same as F2F
Learning	Self-taught with reading, videos.	Through in-class lectures, activities.	Partially self-taught with readings and videos, more difficult topics in-class.
Online Discussions	Bread and butter of student and content presence	Almost nonexistent	Same as F2F
Activities	A good number of online classes use activities in groups or individually	About 1/3 of f2f instructors use group-based activities	Unknown
Homework	Required after learning, online discussions, and activities. Mostly auto-evaluated	Required after in-class sessions. Some auto-evaluated, some faculty evaluated.	Required before, after, and during in-class session. Most auto-evaluated, some faculty evaluated.
Projects	Used for some courses and summative assessments.	Very few use projects, possibly upper-level courses and stats.	Same as F2F

It does seem like every implementation of the flipped classroom is a bit different, but I stuck to the common elements; some instruction and/or homework before in-class sessions, doing some (if not all) homework during class, etc. One thread you'll see running throughout is the use of auto-evaluated or faculty evaluated assessment or homework questions. This is becoming a bigger theme for faculty to consider for various reasons. 

• Auto-evaluated work is by definition evaluated by a computer and answers are generated by a computer, meaning students can use computers (Desmos, Photomath, etc.) to answer them. 
• Most of the time the only feedback students get with auto-evaluated work is whether the student is right or wrong. Yes, some systems provide some feedback, but as far as I am aware no system can detect what error was made and provide specific feedback on how not to make that error. Faculty evaluated work on the other hand can provide specific feedback.
• Faculty can save a huge amount of time using auto-evaluated work. 

To balance these concerns of academic honesty, feedback, and time savings it seems some middle ground should be established, like using these systems for specific functions like additional practice on computational skills.

In thinking through the feedback that online homework systems do provide, there is a trend towards using more general growth-mindset type feedback. Having seen some of these examples I worry that they appear tone deaf, and do more harm than good. If a student answers a question in correctly, seeing "Keep trying. We learn from mistakes." auto-generated by the system might not have the same effect as a human providing that feedback AND identifying what the mistake was and how to prevent it in the future. Another colleague is very interested in growth-mindset based feedback, and it seems clear that you have to do a lot of thinking about the student, their work, and what message would support the student to provide effective feedback. An automated message does not seem comparable. 

What do you think? Are online courses going to start resembling f2f courses? Are f2f courses going to start resembling online courses? What is your stance on auto-evaluated work or faculty evaluated work? I'd love to hear what you think. 

Pre-Pre-(Pre?)-Planning: Fall 2021 - Standards based grading, flipped classroom, corequisite support, and maybe group work?

After putting together a grant proposal and wanting to do something other than the list of tasks on my plate left over from last term, I thought I'd spend today prepping for my Fall 2021 course; face-to-face business math. I was really inspired by Matthew Lewis & Michal Ramos from Yakima Valley College, and their talk Thinking About Assessment and Grading Differently: Standards‑Based Grading in Precalculus at WAMATYC this year. Students can get so focused on points and grading, that they act as grades aren't related to understanding. In watching their talk I am hopeful that tying a student's grades directly to their understanding of specific learning objectives, they will be more motivated to do things that actually impact their grade.

This chart really hit home how standards based grading could be accomplished, which I have shared in a past blog post. 

In reviewing their materials it seems that I have a few choices to make. 

• How should I assess the different levels of understanding?
  While they focused on online assessments (using WAMAP) being an online course, I'm thinking of moving some of them to face-to-face assessments. Their plan to have students complete the different levels of understanding of an objective sequentially (An Expert assessment doesn't open up until the Proficient assessment is 100%.) is well done in the system, and the questions themselves are great! Of the Expert questions I looked at, they included things like randomly generated values (a standard in WAMAP),  analyzing an incorrect answer, and short question path with question parts that opened up based on previous responses. All of these things avoid the 'static' questions I see in so many online systems, and would help prevent sharing of solutions and cheating. 
  
  My current thought is to have a similar system in WAMAP where students can complete assignments (homework really) aligned to the Novice and Developing levels of each objective, and students can retake them as much as they want with no penalty. This is also a corequisite support course, so expanding these novice and developing levels to include prerequisite knowledge would seem appropriate. Expert and Proficient levels of objectives would be assessed through face-to-face weekly quizzes, and some other types of assignments, possibly projects, discussion forums, or something else, I'm not sure. I like to limit quizzes to 30 minutes, 5-6 questions, and at the low end this means that I'm limited to something like 25 objectives (5 questions per week * 10 weeks * 1/2 for the number of objectives (Expert and Proficient) I can assess) which seems doable. That leads me to...
• What objectives are you going to focus on?
  Since assessments are directly related to objectives, I'm constrained by how many objectives to include. These would absolutely need to be student readable, so including vague or abstract objectives that 'cover' a lot of content would not be good.
• How to handle retakes?
  Students are not going to get things right the first time, and I need to figure out when this is going to happen. Do I plan on some time at the end of class (10 min) where students can complete a question?
  
• What will feedback look like?
  In focusing on the specific learning objective, and in creating thoughtful rubrics, I am hoping my feedback will be more targeted about the specific area of confusion. Obviously I'll keep providing process feedback, and discussing ways for students to improve their work. 
• No due dates?
  Really? Am I going to be ok with that? In making the assessments the main way students can reach higher levels of each objective I'm hoping they nudge students into preparing for them. At the same time I need to be deliberate and focus their attention on completing their work on a regular basis. 

Because this is a corequisite support course I have a number of success skill assignments tied to different components of the course. The time management assignments are ok, but I feel students don't connect when they should complete their work. I'll be trying to work backwards from the assessments, to the assignments, to what they should be doing each day. I have other such assignments on growth mindset, deep versus shallow processing, etc. but I'm wondering if maybe I should focus on time management as the central skill, and bringing up other affective domain skills through it. For example, sure you block out time for homework, but what should you be doing (and not doing) during that time? If you're feeling unmotivated to complete your schedule, why, what are those unresolved feelings that are motivating your procrastination. 

The course includes eight hours of instruction (5 college-level, 3 pre-college level) but I am only meeting for five hours a week, with the other three being online. In flipping some of that material, especially the prerequisite pre-college material, I'm hoping to make sure students are prepared for class sessions. In not having due dates I'm a little concerned about students not completing the flipped instruction, how else am I going to get them to do it?

For now I feel like I have more questions than answers, but over the summer I'm hoping to dial in a course plan I can use for the year. Here is roughly what I'm thinking;

- July: Decide on course objectives, levels, rubrics, and create some of that material in WAMAP. 

- July 20, 22, 26: I am taking the Flipped Plus Model PCC continuing education course run by Wendy Fresh and Jessica Bernards.

- August: Create flipped instruction videos and linked class materials.

- September: Revisit Success Skill assignments, and decide what would be helpful to students. 

Given my other projects this feels manageable if I spend 10 hours a week on it.

Your thoughts on any of this? Something I'm not seeing or overlooking? Am I going overboard? Let me know in the comments.