Why online math homework isn't living up to its potential.

In planning my math classes for the fall, I am rethinking the use of online math homework systems. There are some powerful uses of these systems, and I want to take some time to think through some of the hidden costs and missed opportunities they misdirect us from. The following are my own thoughts, and I have yet to dig into the research. If you have any studies or articles that address these issues, especially if they counter my thinking, please send them my way. 

Practice vs. Scores

The biggest benefit of online homework systems is to help students practice math concepts and computations. By providing questions, showing correct answers, linking to full solutions and videos, and allowing for multiple reattempts, these systems allow students to practice iteratively. Ideally students use these answers and solutions to identify their errors, correct them, and confirm their new understanding with another question. 

An assumption in the above is that students will use this set of questions to practice in this focused way, and subsequently learn the underlying concept. After getting a question incorrect they will take a moment to review the course material, watch a provided video, or some other intervention to address a misconception or computational error. Educators assume students are using these systems in a thoughtful, rational, and metacognitively informed way.

Maybe it was never true that students used these systems in this way, but my colleagues and I have certainly worked with students who seem to have earned a perfect score on the online homework, but cannot answer basic questions that were on the homework in person. These systems track the amount of time students spend on an assignment, and when they complete twenty questions in  less than 10 minutes it seems clear that they are completing these assignments in a way that does not align with our assumptions above. When this happens most of us believe students are copying and pasting questions into AI.

Granted, when we ask students to do anything educators should ask "How do they know how to do this?" and think about how to scaffold the activity or action. This can certainly be done by modeling during class, instructional videos, specific instructions, and questions within the assignments asking students for more detailed analysis. Yet essay questions must be graded manually by faculty, which works against the 'efficiency' of these systems. 

Isolated Algorithms vs. Deep Connections

When creating assignments within an online homework system, faculty know the broad topics and connections they want students to practice and subsequently understand. Because of the nature of these systems we have to select a finite number of questions, with answers that are algorithmically determined by internal coding. This coding allows for the benefits above, that answers can be marked correct or incorrect, and students can reattempt similar questions with different values. 

In presenting questions individually this setup leans into the (false) idea that answering a math question is a matter of answering disconnected questions, like a hellish version of Jeopardy. This is not the intent, as faculty want students to understand topics broadly and build connections. Yet this atomization of individual questions does not give students much of an opportunity to identify connections between questions, and subsequently topics. 

Having a limited number of questions to answer gives students a false sense that once they complete the assignment they know the material. While many of us give further directions on when, how, and what to practice, many students will see the 100% in the gradebook and assume they have learned everything they need to.

Being able to reattempt a similar question just with different numbers also supports the idea that each math question can be answered by 'doing something' to the numbers provided. Many students interpret this as a pattern matching exercise, where they need to identify the pattern of what to do with each number, without any attempt at understanding the underlying computation or topic.

Focused Practice vs. The Open Internet

Being online these systems are accessible to students at whatever place and time they want, just an internet-connected device is needed. This allows for a huge amount of flexibility for students, and faculty, to complete assignments and grading.

Being on the open internet a student can very quickly choose to do something unrelated to the course. Anything from streaming a movie, scrolling social media, or playing a video game are a few clicks away. This is before we talk about AI use for homework, which is a plague on its own. This temptation is great for working adults, let alone students who are still developing their ability to learn.

Faculty 'Offloading' vs. Engagement

An issue that I have experienced both in my teaching and as department chair, is when faculty offload much of their thinking and teaching to students on online homework systems in place of real student and content engagement. These systems do allow for lots of data about student work time, question completion, grades, and usually have ways for students to message their instructor about a question. This last feature is invaluable, otherwise many students don't provide sufficient information for us to help them. With a message within the system linking to the question itself, we can go to the question and quickly identify what their answer was. 

In order to be useful, the data these systems collect must be reviewed. It is very easy, in the variety of tasks a faculty member has to do, to forget to review student data. I have had success in setting up time blocks each week where I do just that, and sometimes they don't happen because of a fire I have to put out. Ideally, in reviewing this data faculty get to see the topics that students are struggling with, and offer interventions during class or online.

It sometimes happens that a faculty member doesn't review this data, and works on the assumption that as long as students are earning high grades on assignments they are learning. Faculty are often surprised when students perform poorly on assessments, and in looking at this data realize that many students really weren't engaging with the course concepts, rather completing these assignments in a way that does not increase understanding. Setting these systems on 'autopilot' is easy to do, even accidently.

To be fair, we all know of lecture faculty in our academic lives who did not review homework or practice assignments, and focused solely on assessments. Yet I think the difference here is that with those faculty there is no assumption that someone is reviewing your practice work. With online systems there very well could be an assumption of students that their faculty member is reviewing their work, the data the system collects, and messages. If this does not happen, then everyone is put in a difficult spot; students don't get the support they believe they are getting, and faculty don't have data to base interventions on.

Questions to Consider

1. Do any of these issues resonate with you or your students? 
2. Are there assumptions I am not stating about these systems? 
3. What are ways that online math homework systems could be changed to address these issues?
4. Would going back to pen and paper homework, spot checking 1-3 questions at the start of class, address all of these issues? Would it be better?

Hopefully the above shares how these online math homework systems are supposed to work, and what behaviors I am actually seeing in students.  I'd love to hear your thoughts, or responses to the Questions to Consider. Feel free to message me directly, comment below, or share your anonymous response in the Applied Abstractions - Questions to Consider form.

The Endowment Effect and the AI Inverse

The endowment effect is when we overvalue what we own, compared to equivalent items we don't. Think of your cousin who claims their copy of Amazing Spider-Man Annual Vol. 1 #21, The Wedding! issue where Peter Parker marries Mary Jane, is worth more than yours. While not a universal example, you've encountered people who hang on to what they own, not satisfied with any price other than their unrealistic one. This is also true of ideas. You only have to talk to an educator for half an hour, and they will share their solutions to common classroom management issues and instruction choices.

In reading The Elements of Choice: Why the Way We Decide Matters by Eric J. Johnson and  Quit: The Power of Knowing When to Walk Away by Annie Duke, I came across this idea in both books. Eric Johnson points out that the endowment effect can make decisions harder, and that people who design choices can leverage it through default choices. He mentions studies where a choice can be changed after new information is given, and most people will generally stick to their original choice. This aligns to how Annie Duke discusses the issue, that we are so tied to our own choices that we have a hard time quitting, as you would be 'quitting' yourself in a way. There are a variety of factors she mentions in why we don't quit a course of action, from our choices informing our identity, to the 'katamari' effect (sunk cost fallacy) of past decisions building to today's state, and the endowment effect.

I've talked before about my reasons for reading these two books, and their intertwined ideas about making choices and 'unmaking' them (quitting) keep pointing to new questions to mull over in relation to my teaching. 

• What choices and beliefs do students have that they 'overvalue' instead of others?
• What behaviors and practices are students tied to that don't support their learning?
• Are there new 'identities' students can take on in my class that will help them see themselves as learners and researchers? (Others have talked about a person's 'math identity' in persuasive ways.) 

At the same time I am seeing many students doing the opposite; overvaluing what AI will produce instead of their own thinking and learning. There is a simplistic idea that AI will produce correct and accurate results, which isn't true. The efficiency trap comes into play, as many students need to get a variety of tasks accomplished, and see AI as an efficient solution to many of them. Unfortunately learning only happens when we truly engage with concepts, not pass them off to other entities to do our thinking for us. 

So why don't students prioritize their own thoughts over AI's? As efficient actors students are looking to conserve time, money, and energy. Thinking requires all three, so it isn't a matter of having something, but rather doing something. That is where insecurities about doing the 'right' thing, looking 'dumb', and a host of other fears come up. Those are social, interpersonal, and intrapersonal dynamics that I don't think the endowment effect gets to.

So are there things educators can do to get students to 'quit' unproductive behaviors that they may be tied to? I think the answer is yes, but likely requires both a goal and a belief that what you are doing will get you to that goal. This is something I talk about with students, as humans won't do hard things if they don't know why, or where this is going. Being transparent about what topics and computations will be necessary in future courses seems to address some of these fears. At the same time students who are motivated by a future state, who know what program they are pursuing, what they want to do, generally don't need me to help frame what we learn or motivate them. 

Default choices are really our past choices writ today. What are things I can do in my classes to change the default choice to one where students are sitting down after class and answering a few math questions, or reading a textbook? One idea from The Elements of Choice is 'streaks', doing an action or activity and recording doing it in the same place. Over time you will have completed it on consecutive days, providing a sense that a person can't break the streak. (This seems tied to the sunk-cost fallacy, that all those past days would be 'worthless' (not true) if you don't continue the activity.) How could I include something where students record doing something related to the course each day of the week? (I have thoughts and will share them in a future post.) What does an AI-free streak look like? (Unfortunately no thoughts there.)

What are other ways educators both push students away from unhelpful behaviors, and pull them towards their future self? Both seem hard given the distractions they have on a daily basis, and the future looking a little grim right now. Hope springs eternal, and I hope you can share some of yours. 

Hard Thoughts: My BIG Question and Next Steps

I had the great opportunity to talk to a storied and respected educator in POGIL circles about potentially leaving teaching. I've known this person for about six years, and they have helped me grow my teaching and reflection practices in ways I am incredibly grateful for. We talked for about an hour and a half, and while I still have questions, our conversation has helped me pose and reframe some of my questions in prior posts. 

• What homework practices can I implement that increase students' chances of practicing the skills and concepts they learn in class? I'm concerned I've relied a bit too heavily on technology and by extension students' self-regulation to complete those assignments. In assigning online homework there are assumptions that each student has a stable, quiet place to complete coursework, that students know how to focus on these assignments and not be distracted with the entirety of the Internet, and that they know how to use these technologies for learning. I have some thoughts on how to help students focus on the core of what I need them to do after class, practice, and will share them soon. 
• What are ways I can scaffold my activities to help students who do not have sufficient prerequisite knowledge successfully learn the course material? Sure, I can do a bit at the very start of the term to make sure they are in the right class, and doing sufficient review of prerequisite topics is a necessity in any math class. We talked about different POGIL models, or different questions, and I wonder if there are other things I can do to increase student 'flow' through an activity. Differentiation is a core skill in the K-12 classroom, and I wonder how much of those skills I can use in mine. 
• Can I continue to make the decision to meet students where they are, and support them towards the course outcomes? 

This last one is THE question. Right now I think my answer is yes, I still have things I want to try. When I cannot continue to make that decision and have things I want to try, I think that is the moment I start planning my exit. After all, if I quit I can't answer the "What if?" questions, and if they run out I think I have my answer.

That being said I think this process over the last few months of examining my teaching practice and desire to continue it has pointed to some other opportunities. In talking to other educators they agree that now is a good time to consider alternatives. The landscape of higher education is anything but certain, and having a back-up plan (or two) is a good hedge if things go south, say by runaway automation, employment opportunities drying up, or worse. By having a plan of what else I can do I am doing this work on my own terms, and not waiting for external factors to dictate when and how I have to change. After all, No Effort is Wasted, so says Hank Green. 

I am planning to put a few eggs in other baskets, away from my home institution. Nothing major, and I do not plan on slowing down on any of my commitments. (I have a reputation for being an annoying but helpful cuss to maintain after all.) No, I think I have to put some small investments into other ideas, thought experiments, potential lives I want to live. At the start of this summer I'd like to take some time and explore a few big ideas. 

• Cal Newport had a podcast recently titled Should I Press Pause?, and shared a three step process to identify small actions you can take towards big changes for your life. It sounds like it will be helpful; identify the big 'join the circus' dreams you have, and backwards design them to small actionable things you can do today.
• I found this older TEDx talk 5 steps to designing the life you want by Bill Burnett and it has some elements I want to think about (gravity problems, choice overload, etc.) but not sure if I want to follow it to the letter. 
• In Getting Things Done there is the concept of a Year End Review, and it seems like it makes more sense for me to do this at the end of the academic year, as opposed to my birthday, April 28th. Its just an awkward time during the school year.

In the podcast Newport talks a lot about going somewhere new and interesting to go through these three steps. Unfortunately I am a little tight on time and money at the start of this summer, so I'm planning to do it during jury duty that I have next week. I may be able to get some time afterwards to go to a nice coffee shop or our downtown library, but it may just have to be some municipal buildings for me. I am hoping the change in scenery will give me some of what he's going for, and that being elbow to elbow with other potential jurists won't be too distracting. 

How are you thinking about your big life goals? Is there a question or practice that helped you clarify what you wanted to 'be' when you grew up? How do you balance your professional obligations with investments into different futures?

A Simple Path Forward: An Analog Solution for a Digital World

In getting through the audiobook of Elements of Choice by Eric J. Johnson Chapter 5 discusses how defaults work, specifically through three channels; ease, endorsement, and endowment. The book goes through a variety of situations and examples of people making choices, for good and bad, and uses them to examine the ways and reasons we make decisions. Default choices are a way organizations and people can frame choices to others, like an opt-in to a mailing address when you order something from a store.

A default makes a choice easier in that people are unlikely to make a change. The example of setting a 3% withholding for a retirement account is shared in the book, and how few people increase or decrease that savings rate. The default endorses the specific choice, for good or bad if the choice giver is looking to capitalize on that default. There is also an endowment effect that happens, where after a period people feel attached to the default as if it were their choice.

I am reading the book to figure out a way to help students make better choices around completing coursework, and to practice the skills and knowledge in my classes. My class success and DFW rate that around half of my students are not making the decision to practice coursework. There could be a range of reasons for this, some I can address but many I can't;

• Student has more obligations than they have time for. There isn't a lot I can do about this, but I can be clear with class expectations on the first day of class to ensure they are in the right class. 
• Something comes up in a student's life that increases their obligations. Again, not much I can do about this reason, but if a student misses a class I can make sure they have what they need to address missing the class.
• Student does not know the prerequisite material. I can make sure that students are ready for the course by checking prerequisites and by providing a prerequisite check on the first day of class. Sure, it's a bit intimidating on the first day, and I expect students to get comfortable demonstrating what they know. 
• Student encounters difficulty in sitting down to work on course work. Alternatively they may encounter difficulty in getting into the online textbook and homework. 

That last reason has been gnawing at me; What if I can organize my class materials and assignments to reduce the friction so that students make the choice to complete coursework? Whatever it is needs to be self-contained (within reason), something I can include in grading (students don't do optional) and therefore is done on paper to reduce AI usage, and would support students making better choices. 

I have some thoughts on how to do the above, but what do you think? Are there structures, documents, processes, routines, or something I am not thinking of that can support students in making choices that support their learning?

Reading: The Elements of Choice and Quit

 I started reading The Elements of Choice: Why the Way We Decide Matters by Eric J. Johnson a while ago to better understand how to support students in making better choices around their courses. It certainly helped in writing up my thoughts in A Simple Path Forward: Trading digital clutter for physical simplicity, thinking through how to reduce student choice in the LMS and focus their attention on course concepts and ideas. The idea of having a specific list of tasks to complete before the next class session, focused on practice on paper without online tools, certainly reduces student choice however it does feed into the 'efficiency' trap many in education fall into; that faculty should tell students what to do and that should be enough. I'm now wondering if scaffolding this list so that the first three weeks includes these lists, the next three weeks includes half of this list, and the last three weeks students are expected to fill in the list would meet students in the middle. 

The other book I'm reading (based on conversations after posting Hard Thoughts: Is it time to move on or did I have a rough year?) is Quit: The Power of Knowing When to Walk Away by Annie Duke. It examines why and when we quit things in our life, and why we don't. So far the focus has been on things you might expect; expected value, sunk cost fallacy, etc. It also includes a wide range of examples and situations where people quit while ahead and were ridiculed, or where people didn't quit and ended up broke, unhappy, or dead. There are a lot of persuasive framings of quitting, that Americans especially romanticize keep going despite the odds, quitting and keeping going are two sides of the same coin, and other really useful ways of rethinking what it means to quit. 

I'm about a third of the way done with both, and I'm seeing a pretty deep connection between the two; by thinking through my choices I can (hopefully) make a better decision to keep on my current career path, to quit to do something else, or to do something in between where I iterate (literally quitting some things) and refocus on aspects of a job I enjoy, want to do, and allow me to get paid. To help I am considering a decision matrix with a variety of options and weighing them on a few factors.

What factors should I include in this expected value calculation? Is there something you decided that in hindsight you wish you considered something else?