Why online math homework isn't living up to its potential.

In planning my math classes for the fall, I am rethinking the use of online math homework systems. There are some powerful uses of these systems, and I want to take some time to think through some of the hidden costs and missed opportunities they misdirect us from. The following are my own thoughts, and I have yet to dig into the research. If you have any studies or articles that address these issues, especially if they counter my thinking, please send them my way. 

Practice vs. Scores

The biggest benefit of online homework systems is to help students practice math concepts and computations. By providing questions, showing correct answers, linking to full solutions and videos, and allowing for multiple reattempts, these systems allow students to practice iteratively. Ideally students use these answers and solutions to identify their errors, correct them, and confirm their new understanding with another question. 

An assumption in the above is that students will use this set of questions to practice in this focused way, and subsequently learn the underlying concept. After getting a question incorrect they will take a moment to review the course material, watch a provided video, or some other intervention to address a misconception or computational error. Educators assume students are using these systems in a thoughtful, rational, and metacognitively informed way.

Maybe it was never true that students used these systems in this way, but my colleagues and I have certainly worked with students who seem to have earned a perfect score on the online homework, but cannot answer basic questions that were on the homework in person. These systems track the amount of time students spend on an assignment, and when they complete twenty questions in  less than 10 minutes it seems clear that they are completing these assignments in a way that does not align with our assumptions above. When this happens most of us believe students are copying and pasting questions into AI.

Granted, when we ask students to do anything educators should ask "How do they know how to do this?" and think about how to scaffold the activity or action. This can certainly be done by modeling during class, instructional videos, specific instructions, and questions within the assignments asking students for more detailed analysis. Yet essay questions must be graded manually by faculty, which works against the 'efficiency' of these systems. 

Isolated Algorithms vs. Deep Connections

When creating assignments within an online homework system, faculty know the broad topics and connections they want students to practice and subsequently understand. Because of the nature of these systems we have to select a finite number of questions, with answers that are algorithmically determined by internal coding. This coding allows for the benefits above, that answers can be marked correct or incorrect, and students can reattempt similar questions with different values. 

In presenting questions individually this setup leans into the (false) idea that answering a math question is a matter of answering disconnected questions, like a hellish version of Jeopardy. This is not the intent, as faculty want students to understand topics broadly and build connections. Yet this atomization of individual questions does not give students much of an opportunity to identify connections between questions, and subsequently topics. 

Having a limited number of questions to answer gives students a false sense that once they complete the assignment they know the material. While many of us give further directions on when, how, and what to practice, many students will see the 100% in the gradebook and assume they have learned everything they need to.

Being able to reattempt a similar question just with different numbers also supports the idea that each math question can be answered by 'doing something' to the numbers provided. Many students interpret this as a pattern matching exercise, where they need to identify the pattern of what to do with each number, without any attempt at understanding the underlying computation or topic.

Focused Practice vs. The Open Internet

Being online these systems are accessible to students at whatever place and time they want, just an internet-connected device is needed. This allows for a huge amount of flexibility for students, and faculty, to complete assignments and grading.

Being on the open internet a student can very quickly choose to do something unrelated to the course. Anything from streaming a movie, scrolling social media, or playing a video game are a few clicks away. This is before we talk about AI use for homework, which is a plague on its own. This temptation is great for working adults, let alone students who are still developing their ability to learn.

Faculty 'Offloading' vs. Engagement

An issue that I have experienced both in my teaching and as department chair, is when faculty offload much of their thinking and teaching to students on online homework systems in place of real student and content engagement. These systems do allow for lots of data about student work time, question completion, grades, and usually have ways for students to message their instructor about a question. This last feature is invaluable, otherwise many students don't provide sufficient information for us to help them. With a message within the system linking to the question itself, we can go to the question and quickly identify what their answer was. 

In order to be useful, the data these systems collect must be reviewed. It is very easy, in the variety of tasks a faculty member has to do, to forget to review student data. I have had success in setting up time blocks each week where I do just that, and sometimes they don't happen because of a fire I have to put out. Ideally, in reviewing this data faculty get to see the topics that students are struggling with, and offer interventions during class or online.

It sometimes happens that a faculty member doesn't review this data, and works on the assumption that as long as students are earning high grades on assignments they are learning. Faculty are often surprised when students perform poorly on assessments, and in looking at this data realize that many students really weren't engaging with the course concepts, rather completing these assignments in a way that does not increase understanding. Setting these systems on 'autopilot' is easy to do, even accidently.

To be fair, we all know of lecture faculty in our academic lives who did not review homework or practice assignments, and focused solely on assessments. Yet I think the difference here is that with those faculty there is no assumption that someone is reviewing your practice work. With online systems there very well could be an assumption of students that their faculty member is reviewing their work, the data the system collects, and messages. If this does not happen, then everyone is put in a difficult spot; students don't get the support they believe they are getting, and faculty don't have data to base interventions on.

Questions to Consider

1. Do any of these issues resonate with you or your students? 
2. Are there assumptions I am not stating about these systems? 
3. What are ways that online math homework systems could be changed to address these issues?
4. Would going back to pen and paper homework, spot checking 1-3 questions at the start of class, address all of these issues? Would it be better?

Hopefully the above shares how these online math homework systems are supposed to work, and what behaviors I am actually seeing in students.  I'd love to hear your thoughts, or responses to the Questions to Consider. Feel free to message me directly, comment below, or share your anonymous response in the Applied Abstractions - Questions to Consider form.

The Endowment Effect and the AI Inverse

The endowment effect is when we overvalue what we own, compared to equivalent items we don't. Think of your cousin who claims their copy of Amazing Spider-Man Annual Vol. 1 #21, The Wedding! issue where Peter Parker marries Mary Jane, is worth more than yours. While not a universal example, you've encountered people who hang on to what they own, not satisfied with any price other than their unrealistic one. This is also true of ideas. You only have to talk to an educator for half an hour, and they will share their solutions to common classroom management issues and instruction choices.

In reading The Elements of Choice: Why the Way We Decide Matters by Eric J. Johnson and  Quit: The Power of Knowing When to Walk Away by Annie Duke, I came across this idea in both books. Eric Johnson points out that the endowment effect can make decisions harder, and that people who design choices can leverage it through default choices. He mentions studies where a choice can be changed after new information is given, and most people will generally stick to their original choice. This aligns to how Annie Duke discusses the issue, that we are so tied to our own choices that we have a hard time quitting, as you would be 'quitting' yourself in a way. There are a variety of factors she mentions in why we don't quit a course of action, from our choices informing our identity, to the 'katamari' effect (sunk cost fallacy) of past decisions building to today's state, and the endowment effect.

I've talked before about my reasons for reading these two books, and their intertwined ideas about making choices and 'unmaking' them (quitting) keep pointing to new questions to mull over in relation to my teaching. 

• What choices and beliefs do students have that they 'overvalue' instead of others?
• What behaviors and practices are students tied to that don't support their learning?
• Are there new 'identities' students can take on in my class that will help them see themselves as learners and researchers? (Others have talked about a person's 'math identity' in persuasive ways.) 

At the same time I am seeing many students doing the opposite; overvaluing what AI will produce instead of their own thinking and learning. There is a simplistic idea that AI will produce correct and accurate results, which isn't true. The efficiency trap comes into play, as many students need to get a variety of tasks accomplished, and see AI as an efficient solution to many of them. Unfortunately learning only happens when we truly engage with concepts, not pass them off to other entities to do our thinking for us. 

So why don't students prioritize their own thoughts over AI's? As efficient actors students are looking to conserve time, money, and energy. Thinking requires all three, so it isn't a matter of having something, but rather doing something. That is where insecurities about doing the 'right' thing, looking 'dumb', and a host of other fears come up. Those are social, interpersonal, and intrapersonal dynamics that I don't think the endowment effect gets to.

So are there things educators can do to get students to 'quit' unproductive behaviors that they may be tied to? I think the answer is yes, but likely requires both a goal and a belief that what you are doing will get you to that goal. This is something I talk about with students, as humans won't do hard things if they don't know why, or where this is going. Being transparent about what topics and computations will be necessary in future courses seems to address some of these fears. At the same time students who are motivated by a future state, who know what program they are pursuing, what they want to do, generally don't need me to help frame what we learn or motivate them. 

Default choices are really our past choices writ today. What are things I can do in my classes to change the default choice to one where students are sitting down after class and answering a few math questions, or reading a textbook? One idea from The Elements of Choice is 'streaks', doing an action or activity and recording doing it in the same place. Over time you will have completed it on consecutive days, providing a sense that a person can't break the streak. (This seems tied to the sunk-cost fallacy, that all those past days would be 'worthless' (not true) if you don't continue the activity.) How could I include something where students record doing something related to the course each day of the week? (I have thoughts and will share them in a future post.) What does an AI-free streak look like? (Unfortunately no thoughts there.)

What are other ways educators both push students away from unhelpful behaviors, and pull them towards their future self? Both seem hard given the distractions they have on a daily basis, and the future looking a little grim right now. Hope springs eternal, and I hope you can share some of yours. 

Hard Thoughts: My BIG Question and Next Steps

I had the great opportunity to talk to a storied and respected educator in POGIL circles about potentially leaving teaching. I've known this person for about six years, and they have helped me grow my teaching and reflection practices in ways I am incredibly grateful for. We talked for about an hour and a half, and while I still have questions, our conversation has helped me pose and reframe some of my questions in prior posts. 

• What homework practices can I implement that increase students' chances of practicing the skills and concepts they learn in class? I'm concerned I've relied a bit too heavily on technology and by extension students' self-regulation to complete those assignments. In assigning online homework there are assumptions that each student has a stable, quiet place to complete coursework, that students know how to focus on these assignments and not be distracted with the entirety of the Internet, and that they know how to use these technologies for learning. I have some thoughts on how to help students focus on the core of what I need them to do after class, practice, and will share them soon. 
• What are ways I can scaffold my activities to help students who do not have sufficient prerequisite knowledge successfully learn the course material? Sure, I can do a bit at the very start of the term to make sure they are in the right class, and doing sufficient review of prerequisite topics is a necessity in any math class. We talked about different POGIL models, or different questions, and I wonder if there are other things I can do to increase student 'flow' through an activity. Differentiation is a core skill in the K-12 classroom, and I wonder how much of those skills I can use in mine. 
• Can I continue to make the decision to meet students where they are, and support them towards the course outcomes? 

This last one is THE question. Right now I think my answer is yes, I still have things I want to try. When I cannot continue to make that decision and have things I want to try, I think that is the moment I start planning my exit. After all, if I quit I can't answer the "What if?" questions, and if they run out I think I have my answer.

That being said I think this process over the last few months of examining my teaching practice and desire to continue it has pointed to some other opportunities. In talking to other educators they agree that now is a good time to consider alternatives. The landscape of higher education is anything but certain, and having a back-up plan (or two) is a good hedge if things go south, say by runaway automation, employment opportunities drying up, or worse. By having a plan of what else I can do I am doing this work on my own terms, and not waiting for external factors to dictate when and how I have to change. After all, No Effort is Wasted, so says Hank Green. 

I am planning to put a few eggs in other baskets, away from my home institution. Nothing major, and I do not plan on slowing down on any of my commitments. (I have a reputation for being an annoying but helpful cuss to maintain after all.) No, I think I have to put some small investments into other ideas, thought experiments, potential lives I want to live. At the start of this summer I'd like to take some time and explore a few big ideas. 

• Cal Newport had a podcast recently titled Should I Press Pause?, and shared a three step process to identify small actions you can take towards big changes for your life. It sounds like it will be helpful; identify the big 'join the circus' dreams you have, and backwards design them to small actionable things you can do today.
• I found this older TEDx talk 5 steps to designing the life you want by Bill Burnett and it has some elements I want to think about (gravity problems, choice overload, etc.) but not sure if I want to follow it to the letter. 
• In Getting Things Done there is the concept of a Year End Review, and it seems like it makes more sense for me to do this at the end of the academic year, as opposed to my birthday, April 28th. Its just an awkward time during the school year.

In the podcast Newport talks a lot about going somewhere new and interesting to go through these three steps. Unfortunately I am a little tight on time and money at the start of this summer, so I'm planning to do it during jury duty that I have next week. I may be able to get some time afterwards to go to a nice coffee shop or our downtown library, but it may just have to be some municipal buildings for me. I am hoping the change in scenery will give me some of what he's going for, and that being elbow to elbow with other potential jurists won't be too distracting. 

How are you thinking about your big life goals? Is there a question or practice that helped you clarify what you wanted to 'be' when you grew up? How do you balance your professional obligations with investments into different futures?

A Simple Path Forward: An Analog Solution for a Digital World

In getting through the audiobook of Elements of Choice by Eric J. Johnson Chapter 5 discusses how defaults work, specifically through three channels; ease, endorsement, and endowment. The book goes through a variety of situations and examples of people making choices, for good and bad, and uses them to examine the ways and reasons we make decisions. Default choices are a way organizations and people can frame choices to others, like an opt-in to a mailing address when you order something from a store.

A default makes a choice easier in that people are unlikely to make a change. The example of setting a 3% withholding for a retirement account is shared in the book, and how few people increase or decrease that savings rate. The default endorses the specific choice, for good or bad if the choice giver is looking to capitalize on that default. There is also an endowment effect that happens, where after a period people feel attached to the default as if it were their choice.

I am reading the book to figure out a way to help students make better choices around completing coursework, and to practice the skills and knowledge in my classes. My class success and DFW rate that around half of my students are not making the decision to practice coursework. There could be a range of reasons for this, some I can address but many I can't;

• Student has more obligations than they have time for. There isn't a lot I can do about this, but I can be clear with class expectations on the first day of class to ensure they are in the right class. 
• Something comes up in a student's life that increases their obligations. Again, not much I can do about this reason, but if a student misses a class I can make sure they have what they need to address missing the class.
• Student does not know the prerequisite material. I can make sure that students are ready for the course by checking prerequisites and by providing a prerequisite check on the first day of class. Sure, it's a bit intimidating on the first day, and I expect students to get comfortable demonstrating what they know. 
• Student encounters difficulty in sitting down to work on course work. Alternatively they may encounter difficulty in getting into the online textbook and homework. 

That last reason has been gnawing at me; What if I can organize my class materials and assignments to reduce the friction so that students make the choice to complete coursework? Whatever it is needs to be self-contained (within reason), something I can include in grading (students don't do optional) and therefore is done on paper to reduce AI usage, and would support students making better choices. 

I have some thoughts on how to do the above, but what do you think? Are there structures, documents, processes, routines, or something I am not thinking of that can support students in making choices that support their learning?

Reading: The Elements of Choice and Quit

 I started reading The Elements of Choice: Why the Way We Decide Matters by Eric J. Johnson a while ago to better understand how to support students in making better choices around their courses. It certainly helped in writing up my thoughts in A Simple Path Forward: Trading digital clutter for physical simplicity, thinking through how to reduce student choice in the LMS and focus their attention on course concepts and ideas. The idea of having a specific list of tasks to complete before the next class session, focused on practice on paper without online tools, certainly reduces student choice however it does feed into the 'efficiency' trap many in education fall into; that faculty should tell students what to do and that should be enough. I'm now wondering if scaffolding this list so that the first three weeks includes these lists, the next three weeks includes half of this list, and the last three weeks students are expected to fill in the list would meet students in the middle. 

The other book I'm reading (based on conversations after posting Hard Thoughts: Is it time to move on or did I have a rough year?) is Quit: The Power of Knowing When to Walk Away by Annie Duke. It examines why and when we quit things in our life, and why we don't. So far the focus has been on things you might expect; expected value, sunk cost fallacy, etc. It also includes a wide range of examples and situations where people quit while ahead and were ridiculed, or where people didn't quit and ended up broke, unhappy, or dead. There are a lot of persuasive framings of quitting, that Americans especially romanticize keep going despite the odds, quitting and keeping going are two sides of the same coin, and other really useful ways of rethinking what it means to quit. 

I'm about a third of the way done with both, and I'm seeing a pretty deep connection between the two; by thinking through my choices I can (hopefully) make a better decision to keep on my current career path, to quit to do something else, or to do something in between where I iterate (literally quitting some things) and refocus on aspects of a job I enjoy, want to do, and allow me to get paid. To help I am considering a decision matrix with a variety of options and weighing them on a few factors.

What factors should I include in this expected value calculation? Is there something you decided that in hindsight you wish you considered something else?

A Simple Path Forward: Trading digital clutter for physical simplicity.

I talked to my 'rabbi' the other day, a senior faculty member in the department, and we talked over his observation of my class. He said a lot that stuck with me, but the thing that rings out now is my class has a lot of 'bells and whistles' in the number and type of assignments I have. Maybe its time to reduce all of them for something more simple, for both my students and myself.  

My colleague and I also kvetched about the problem of students just disappearing. A good 10%-30% of my class has just stopped showing up after a few weeks, and in talking to others this isn't unusual. I wonder what I could do to persuade more students to not withdraw. Is there some phrase, some way of describing learning that I can share that makes them more receptive to sticking with the struggle of learning? Some change to assignments, framing of practice, a culturally relevant example or application that will inspire them to stay? If I were to quit teaching I wouldn't be able to answer these questions, and would be left wondering 'What if?'

In listening to the audiobook of Elements of Choice by Eric J. Johnson I wonder if I can shape a student's plausible path towards them choosing to lean into learning the course material. Can I reduce the choices students have to one where they complete assignments? I keep circling back to the following course structure for next Fall;

• No online assignments. The LMS is a place just for grades, announcements, and files.
• After each class is a list of two to three things students should do before the next class session, such as;
• Complete any remaining questions from the in-class activity from Monday's class. 
• Complete the following Exercise questions from Section 1.6
• Read and take notes on Sections 2.1 and 2.2
• Review your returned Assessment 5 and complete the Assessment Reflection Form for up to two questions you want to reattempt.
• Check that students completed this list of tasks for 10% of their course grade. (Maybe during the Weekly Assessment?)
• Weekly Assessments with 5-6 questions from the prior week, that comprise 60% of the course grade, allowing reattempts for up to two questions which amounts to around 40% of each assessment. They can't ignore the material, they have to know something by the Weekly Assessment. 
• The Final Assessment will comprise 30% of the course grade, no reattempts.

I can hear my more senior colleagues cackle in delight as I essentially 'reinvent' college courses from 30 years ago. Yet really stripping down a course, getting away from all the distractions of the online environment, would address some of what I am worrying about. The LMS has this headspace of "I should be doing something here." yet to many students it isn't clear what they should be doing. The ten to thirty percent I lose each term may be leaving because they don't know what to do, and by providing clear directions of what to do will hopefully give them the specific guidance they need. Also, taking the uncertainty of online assignments away and putting the work in front of them on pencil and paper will get them to the work quicker, and in a more focused way.

Hard Thoughts: Answers you have to hear again and again.

My last post got some attention, and after talking to others and thinking through the questions I posed at the end (still more to do to really answer them in full) I thought I'd share where I'm at. This is also a letter to my future self for when I have similar thoughts, as I am noticing a cycle of disillusionment and resolve that seems to happen every 2-3 years. 

Yes, teaching with active learning is difficult. You have chosen to teach on hard mode, as you are not only trying to help students learn math but also how humans learn; through experiences, reflection, and engaging with others. Yet humans also want things to be easy, so posing learning as something you have to actively struggle with is going to get apathy, resistance, and even anger. There may be students who can't do what you ask, which leads to another hard question; are you ready to change your teaching practice to be more inclusive of students who may not be prepared mathematically, or who may not be ready to be put in charge of their learning? The answer has to be yes if you want to help students move from where they are to learn the course outcomes, and this change should refine what you want students to struggle with. 

So what can I do? I can address students who are not mathematically prepared by confirming prerequisites are fulfilled, running a prerequisite assessment at the start of the term. Taking time to carefully grade these assessments will allow me to identify unprepared students I need to counsel to take another course. (I don't believe it is equitable to let a student continue in a course if a faculty member has evidence that the student will be unlikely to be successful. Every student deserves a reasonable chance of success at the start of the term.) This assessment would also let me identify where to offer targeted practice of these skills when they are needed for underprepared but qualified students. Throughout the term I can also point to prerequisite skills students should have before starting topics that require them. I can address students who may not be ready to be put in charge of their learning by being explicit with how learning happens in the course at the start of the term, and following through on that model throughout. A good practice has been to share with students how many of them have completed the homework before giving out the assessment over the material from that assignment.

Still, you're not going to reach everyone. One of the hardest parts of the job right now is the DFW rate in my current courses. For my last three courses it ranged between 50% to 60%. Meaning more than half or the students who are enrolled in my courses after the first week end up with a non-passing grade. I know there are many forces acting on students, and they come from inequitable systems, yet I am tasked with helping students learning the course material. In talking to others these rates are not unusual, but I am driven to be good at my job.

Do what you can with who you have. A poignant moment happened the prior Friday during an active learning community of practice I run. I don't have the number of people I once had, but the quality of our conversations has certainly deepened. Two educators with more years than I shared that we can only reach the students who are in the classroom. If students choose to withdraw from a class it isn't clear if that is something I can control. I can certainly ask them which requires having a copy of the initial course roster with their email addresses, and messaging them once I notice that they have dropped. 

You feel stuck because the system is not setup to support students, and you want to. Decreasing state support of higher education, the demand to be more 'efficient' in education, increased income inequality, decreases in both numeracy and literacy, and other forces beyond your control are making it difficult for students to spend the necessary time and energy on earning a college degree. You can either fight these tides, or use your energy and attention to support the students you can, while advocating for change. 

Hopefully the above will help my future self not get so frustrated, and move on to solutions I can't see right now. What are cycles you repeat? What would you say to your future self?