Pre-Pre-(Pre?)-Planning: Fall 2021 - Standards based grading, flipped classroom, corequisite support, and maybe group work?

After putting together a grant proposal and wanting to do something other than the list of tasks on my plate left over from last term, I thought I'd spend today prepping for my Fall 2021 course; face-to-face business math. I was really inspired by Matthew Lewis & Michal Ramos from Yakima Valley College, and their talk Thinking About Assessment and Grading Differently: Standards‑Based Grading in Precalculus at WAMATYC this year. Students can get so focused on points and grading, that they act as grades aren't related to understanding. In watching their talk I am hopeful that tying a student's grades directly to their understanding of specific learning objectives, they will be more motivated to do things that actually impact their grade.

This chart really hit home how standards based grading could be accomplished, which I have shared in a past blog post. 

In reviewing their materials it seems that I have a few choices to make. 

• How should I assess the different levels of understanding?
  While they focused on online assessments (using WAMAP) being an online course, I'm thinking of moving some of them to face-to-face assessments. Their plan to have students complete the different levels of understanding of an objective sequentially (An Expert assessment doesn't open up until the Proficient assessment is 100%.) is well done in the system, and the questions themselves are great! Of the Expert questions I looked at, they included things like randomly generated values (a standard in WAMAP),  analyzing an incorrect answer, and short question path with question parts that opened up based on previous responses. All of these things avoid the 'static' questions I see in so many online systems, and would help prevent sharing of solutions and cheating. 
  
  My current thought is to have a similar system in WAMAP where students can complete assignments (homework really) aligned to the Novice and Developing levels of each objective, and students can retake them as much as they want with no penalty. This is also a corequisite support course, so expanding these novice and developing levels to include prerequisite knowledge would seem appropriate. Expert and Proficient levels of objectives would be assessed through face-to-face weekly quizzes, and some other types of assignments, possibly projects, discussion forums, or something else, I'm not sure. I like to limit quizzes to 30 minutes, 5-6 questions, and at the low end this means that I'm limited to something like 25 objectives (5 questions per week * 10 weeks * 1/2 for the number of objectives (Expert and Proficient) I can assess) which seems doable. That leads me to...
• What objectives are you going to focus on?
  Since assessments are directly related to objectives, I'm constrained by how many objectives to include. These would absolutely need to be student readable, so including vague or abstract objectives that 'cover' a lot of content would not be good.
• How to handle retakes?
  Students are not going to get things right the first time, and I need to figure out when this is going to happen. Do I plan on some time at the end of class (10 min) where students can complete a question?
  
• What will feedback look like?
  In focusing on the specific learning objective, and in creating thoughtful rubrics, I am hoping my feedback will be more targeted about the specific area of confusion. Obviously I'll keep providing process feedback, and discussing ways for students to improve their work. 
• No due dates?
  Really? Am I going to be ok with that? In making the assessments the main way students can reach higher levels of each objective I'm hoping they nudge students into preparing for them. At the same time I need to be deliberate and focus their attention on completing their work on a regular basis. 

Because this is a corequisite support course I have a number of success skill assignments tied to different components of the course. The time management assignments are ok, but I feel students don't connect when they should complete their work. I'll be trying to work backwards from the assessments, to the assignments, to what they should be doing each day. I have other such assignments on growth mindset, deep versus shallow processing, etc. but I'm wondering if maybe I should focus on time management as the central skill, and bringing up other affective domain skills through it. For example, sure you block out time for homework, but what should you be doing (and not doing) during that time? If you're feeling unmotivated to complete your schedule, why, what are those unresolved feelings that are motivating your procrastination. 

The course includes eight hours of instruction (5 college-level, 3 pre-college level) but I am only meeting for five hours a week, with the other three being online. In flipping some of that material, especially the prerequisite pre-college material, I'm hoping to make sure students are prepared for class sessions. In not having due dates I'm a little concerned about students not completing the flipped instruction, how else am I going to get them to do it?

For now I feel like I have more questions than answers, but over the summer I'm hoping to dial in a course plan I can use for the year. Here is roughly what I'm thinking;

- July: Decide on course objectives, levels, rubrics, and create some of that material in WAMAP. 

- July 20, 22, 26: I am taking the Flipped Plus Model PCC continuing education course run by Wendy Fresh and Jessica Bernards.

- August: Create flipped instruction videos and linked class materials.

- September: Revisit Success Skill assignments, and decide what would be helpful to students. 

Given my other projects this feels manageable if I spend 10 hours a week on it.

Your thoughts on any of this? Something I'm not seeing or overlooking? Am I going overboard? Let me know in the comments. 

Faculty and Data: Threading the needle

Last spring my department did an assessment project to determine if our corequisite support courses (which I am the coordinator of) are effective in helping students learn the course material. My home institution has been running corequisite support courses (Business Mathematics with Support, College Algebra with Support, and College Trigonometry with Support) for more than a year and it seemed like a good time to assess whether these courses were effective. I wrote assessment questions aligned to our course outcomes and asked for faculty feedback, which turned into really interesting and engaging conversations. I also wrote rubrics with granular criteria (53 in total) and asked for faculty feedback, which turned into even more interesting conversations. Faculty in all of the support courses and most of the standard courses included these questions (a feat by itself), and after a day of about two dozen instructors grading the 220 student artifacts we had solid data showing that there was no significant difference in student understanding of the course outcomes.

As a result of this project I was asked to became the Outcomes Assessment Liaison for the Math Department at my home institution. I've also joined the Outcomes Assessment Committee, and it feels like I'm gearing up for another phase of my work. The conversations I had with faculty showed me that I have a lot more to learn about assessment, rubrics, and measuring understanding. Granted I've made decisions about what questions to write to assess outcomes and how to grade them. However seeing how faculty make these decisions for their own courses, and then the effect these decisions have on students makes me wonder whether faculty can (or should) have conversations about the decisions they are making. Why decide on a five point scale versus a ten point? How would different faculty grade the same student work? What are the best practices of math assessments senior faculty have and junior faculty crave?

With these questions the idea of collecting data on questions, rubrics, and course success rates is fairly natural, but like venomous animals, natural doesn't mean safe. Faculty are (rightfully) concerned that this data would be used as a hammer to shape everyone into the mold of the faculty member with the highest course success rates. Note these questions are separate from the expectations of most administors; How can we measure program-level outcomes? How can we show student progress to accreditors? How can we determine the areas in a program that need support? Yes, administrator's questions are absolutely related to faculty decisions, but their focus seems to be on the system as a whole. (I could be wrong in that last point, but being in a service department I don't have students wanting to go into my discipline, and therefore I don't have a lot of control over a specific program.)

Another point to consider is the college's Guided Pathways initiative. Like other community colleges we are undergoing reforms to clarify our pathways, get students onto pathways, keep students on pathways, and to determine whether these pathways are effective. The college created four Guided Pathway Pillars for each of these tasks, and I was (am?) on the pillar tasked with keeping students on the path. In that work I looked at both automated and manual methods for determining students who are 'at-risk' of not being successful in their courses. One dataset that was always asked for was access to the live student grades in our learning management system, Canvas. The idea is that with actual grade data our student services staff could conduct interventions to help students get back on track.

I am not opposed to the idea that student grades could be used to determine how outreach should happen, but that use of grade data needs to be restricted in a very specific way. Granted, administrators can get access to course success rates but live student grades during a term feels like a different beast altogether. If a course had a 50% pass rate half-way through the term and an administrator heard about that, is it out of the realm of possibility that the administrator would consciously or unconsciously apply pressure to that faculty member to increase their pass rate? Sure, lots of hypotheticals and unknowns there, but there very well could be other impacts I'm not thinking of.

This leads me to two thoughts on how we can create an environment of trust among faculty to use data;

1. A memorandum of understanding of how course data is to be used. There needs to be a line between faculty and student services use of the data, with administration accessing the data. Yes, administration needs some data, and that would be outlined in the memo. After the current union negotiations are completed I'm hoping to get facetime with our union president to propose such a document.

2. A faculty learning community where membership means you share your course data with the other faculty members. I really like this idea as it creates and builds community, while taking ownership for what we are doing. Starting from a place of looking at equity data we could address reforms that would have the biggest impact on all students.

Have you tried any of this? What are the successful data projects you have been a part of, or heard about?