Summer 2020 Thoughts - College Trigonometry

For various reasons I am teaching three classes this summer, and because of the coronavirus pandemic I'm teaching them all online through Canvas. Having taught online courses off and on since 2012 I'm comfortable teaching in this modality, but as is my nature, I'm looking to improve the student experience and their ability to learn the course material. I'm writing down the following thoughts so I can look back on my rationale for making certain decisions, to share them with others, and to backtrack on any coding ideas I'd like to follow up on.

Homework

I've taught College Trigonometry a number of times before and have a well developed set of homework questions through WAMAP, which I'll use again. I would like to align homework assignments and questions to course outcomes, and WAMAP has a way to track this so I may give it a shot this term. My goal here would be to give students specific feedback on the areas they need to work on, and to give me a dashboard of where students are doing well, and need help with.

Discussion Forums

I also created a number of group-based active learning discussion forums last term in the switch to remote learning. Below is an example.

I really like having having a first post where students explore an idea or concept through simple exploratory questions. This is in keeping with the POGIL activity writing framework, and works well to get students started thinking about these ideas. I also like that students must share their first post without seeing the work other's did to get a sense of what others are thinking, and to show that incorrect answers are completely normal. What didn't work so well last term was getting students to discuss their answers to these questions. I believe this was partially because their second post was to answer a number of questions in a longer activity, with many students seeing this activity as a 'worksheet' to complete like an additional homework assignment.

Over time my directions to both of these two initial posts changed, making sure to indicate that saying "I don't know." is a perfectly fine answer, but then students would just write that and not go back to the discussion forums. There were situations where students did have engaging discussions starting from these activities, but the students who had these discussions usually had very high affective motivation. The groups that I struggled most with were students who either had low affective motivation, or those who had a very solutions oriented view of mathematics. I will likely need to make explicit the purpose of these discussion forums, and to model what a good discussion forum looks like.

Subsequent replies were supposed to be completed with the following directions;

The above roles were a slight modification of my face-to-face roles, and after the last term it seems clear that I need to make a few changes;

• Roles need to be tied into a deliverable, some 'thing' students do or say at the end of the discussion that shows that they fulfilled the role. For Presenters this could be a group answer or solution to a question where students have to use what they learned in the activity (a POGIL application question), for Reflectors this could be a deeper reflection on the group's work together, for Quality Controller this could be a series of questions about the quality of the group's work, and for Managers this could be a set of questions about who worked well in the group and what students could do to improve. 
• At the end of each week last term I had students complete a group reflection by Thursday so I could post answers to any questions on Friday. For the summer term I think I could have students answer role-based reflection quizzes.
• I assigned roles at the start of each week in a 'Meet your group' discussion forum, but wonder if there are more automatic ways of doing this? I would also really like some kind of notification or color to be the background of discussion forums where they need to post in their roles. CSS maybe?

With multiple deadlines of posts students found it difficult to know when to post. To address this I added reminders in the course calendar of when to post. The reminders are just text, but I figured out you can add links to the assignments themselves. Here is what it looked like in the Canvas calendar;

In playing with the Canvas API I did find a way to create calendar events, and may play with that over the term.

Closer to the end of the course, once groups were well established I included Group Help Discussion Forums, like the following;

This format was borrowed from one of my colleagues Kayoko Barnhill, and is a really nice low effort way to get students talking. For the summer term I may alternate the discussion forums between one to two group-based activity discussion forums, and one of these group help discussion forums. There may be more cognitive load in following different sets of directions at first, but I think students may appreciate the reduced time commitment.

Quizzes

Each week I had a quiz over the course material with five questions and spent the plurality of my time each week grading them and providing feedback. This term I'll be doing similar quizzes, but only providing detailed feedback on two of them. Here are the directions I ended the term with and plan on using this term;

Directions

Submit your work for the quiz. Note the following;

• Your work submission should be the work you did in getting your answers to quiz.
• You will earn partial credit by showing your work, step by step.
• You will not earn any additional credit for submitting the incorrect answers in the quiz and submitting the correct answer in your work.
• You will not earn credit for questions that you answered correctly with no work.
• You will not earn credit for solutions that utilize technology (graphing calculators, WolframAlpha, Desmos, etc.) unless specified to in the question.

I will review all of their questions this term for showing work, but will only grade two of their questions in detail.

Final Assessment

I had a final assessment consisting of twenty questions, ten True or False, and ten similar to the homework. A few of my colleagues had questions about the efficacy of True or False questions in a final assessment, pointing out that they should be fairly easy to determine by students. I agreed, but argued that we believe they are easy to determine if the statement is correct or not, and that is not that purpose of these questions. The purpose is to determine if students know how to determine if the statement is correct or not. Below are three questions and their results;

While all students were able to answer the last question above correctly, a number had difficulty with the first two. This suggests to me that students don't have a developed understanding of either inverse functions, or what cos(-x) means. If that is the case then I would be very concerned about their ability to do well in calculus, and subsequently by not answering this question (and others) incorrectly their chances of passing this class decrease. And really, what is the point of a final assessment in a math class other than to determine if a student is ready for the next class?

I did have students write down their work and submit it to me, and overall the quality and work was consistent with their quiz work submissions. Reviewing all of their work took quite a bit of time, but it was made easier by WAMAP's ability to review individual questions, even though I randomized the order the questions students were given. I did not provide as detailed feedback, but did award partial credit where warranted. 

I did not use proctoring software, and actually gave students a few days to take the final with a time limit of 2 hours 30 minutes. They were required to complete an academic affidavit (below) before accessing the final, which was to automatically release after signing the affidavit.

Overall I'm pretty happy with the final assessment from last term, and will likely keep the format, change some of the questions, and ask for online proctoring if available. 

Weekly Overviews

Each week I also wrote a weekly overview page that included student-friendly learning objectives, their course tasks, additional resources, and how to get help. I may keep these, making sure to include them in student's To-Do list (a feature in Canvas that adds content pages to their course calendars), but may also include some of this information in a Monday email or at least summarize it. I've read over the fall term plans of faculty at other institutions and they seem to like multiple communication methods, or at least find that students respond to them differently. 

In writing down the above I've come to a few tasks for myself;

• Rework the discussion forums to include one to two group activities, and one group help discussion forum each week.
• Reformat reflections to be role-based, possibly sharing the results anonymously with students to further our conversations about group work, what high quality work looks like, and to reflect on our experiences. 
• Try to complete the course outcomes in WAMAP.
• Find a way to automate discussion forum reminders. I'm still learning the ins-and-outs of the Canvas API and of Python, so this may be a goal through the term.
• Update Weekly Overviews and include them in a Monday Week-in-Preview email. On second thought I'm not sure I like having separate ways of communicating with students, I'd rather have everything done through Canvas. Sure, students can opt to get information sent to their email or text, Canvas has pretty robust notification settings. At the same time I've had to deal with multiple communication methods in work place settings, and it can be bewildering to figure out where something came from. It generally seems to run counter to Quality Matters, and the best practices I have seen for online education. I could try it... I'll have to mull it over more. 

Writing this out has given me a good idea of what I need to do today, and I'll try to write something similar for my other two classes this summer Finite Mathematics and Business Math, and Business Calculus. I've already decided I'll be doing something less involved for these two classes, and have quite a few materials already developed.

As always I appreciate any feedback, suggestions, questions, ideas, recommendations for books or articles, accusations, or anything else you'd like to share.

Creating an activity about Bayes' Theorem from a blog post

When I teach a topic I haven't taught for a while, I usually refer to some old texts, my course notes, and the internet for new ways of presenting the topic. In my statistics class we were to cover Bayes' Theorem, a topic I have always enjoyed presenting, but never felt that I got quite 'right'. Students seemed disconnected from the idea, and weren't able to answer basic questions about the idea during the first lecture.

To address this I wanted to create an activity where students were to apply Bayes' Theorem in a relatively simple way. Searching the internet I found the article (an essay really) An Intuitive Explanation of Bayes' Theorem by Eliezer S. Yudkowsky, and thought it did a good job explaining the basic idea, and even includes different presentations of the same example. These different presentations are used to discuss innumeracy in health professionals, but provided me a variety of ways of presenting this example.

After some self-editing and debate, I settled on using the simplest presentation of this example; Statistics I Activity - Bayes' Theorem. While the questions aren't directly about Bayes' Theorem, it gets students more familiar with conditional probabilities and how to compute them.

Creating an activity from a blog post or article has some advantages I didn't realize until I presented the activity to students:

• Students are able to confirm their answers by reading the article, a noble goal by itself. 
• I don't mention where I pulled the example from, so they have to search through the article to find the 'answers'. If they read through parts of the article by accident, even better.
• The content becomes richer by pulling from outside resources. I dislike the idea that a course is just about what I, as the instructor, want from students. The ideas and concepts we are talking about are greater than just me, the textbook, and the student. Using someone else's perspective on the topic makes the course 'bigger'.

If you have any questions, comments, or suggestions, feel free to share them below.

Part III - Forming an interesting in-class Statistics activity

I've discussed what I wanted for this experiment activity, what my plan was, and now I'll talk about how it went and what to change in future activities. Overall I thought it went pretty well, but there were a few major changes that had to be made on the fly. The biggest one being the math tests themselves; I completely underestimated my student's basic math ability and we came up with useless data. Someone mentioned that they thought it was part of the activity, and was a great way to show how 'messy' statistics really is. I'm glad they thought that, because I really didn't anticipate it.

The initial discussion of how to construct this experiment was useful and demonstrated a number of ideas we discussed in class. Controlling for certain variables turned into a big part of the discussion, namely how to control for people with natural math ability. We decided to do a paired sample, pairing those people of the same math ability by their score on the first test. I asked if this was really the best measure, and we had a good conversation of how to measure someone's math ability, and how for some people, that's their job.

To control for some of these variables, and to construct a basic demographic survey, I had students develop a few survey questions that may help explain some variation in math ability. This discussion included what to ask, how to ask it, and what kinds of variables (categorical, numerical, etc.) we were measuring. I suggested a question about how long it has been since you took a math class, and some students wanted to do a categorical variable of 0-6 months, 7-12 months, etc. I responded with the question "Is it easier to turn numerical data into categorical data or categorical data into numerical data?" and we talked about converting from one data type to another, and how we were going to use the data.

We also talked about what would happen if a person took two similar math tests back to back. One student mentioned that people would become fatigued,  and rightly so. To limit this I asked what we could do to limit the fatigue, and we discussed the pros and cons of long and short tests. I also mentioned the idea of activating previous knowledge and that after seeing the first test, students would remember how to complete the questions for the second test. We 'settled' on giving both tests to both control and experimental group, and 18 basic math questions... since thats what was in the packet. I know this isn't in the true spirit of exploratory activities, and some people might deride me for exerting this amount of control over the process. I want students to explore this material and engage with it, but doing everything on the fly doesn't seem conductive to these aims. Without some kind of structure students get bored, annoyed, disengaged with the learning process. I can deal with the first two things (barely) but the third I can't.

Students then took the survey we constructed together (number of months since last math class, sex, age, handedness, work status) and the first test. I took them all, handed them back out randomly, and we graded them. I then had students come up to the computer to enter in the information in Excel. This was a good step since it showed that data entry is an important step, one people take for granted. It is time consuming work, and must be done accurately. This was at the 1-hour mark and once they entered the data we had a 5-minute break.

Once the data was entered there were some survey responses that didn't make sense. Instead of age, one person put 'old'. For the number of months since last math class, someone put the categorical variable response 0-6 months even though we settled on a numeric one. I then discussed data cleansing and that our simple decisions on how to handle these discrepancies has real impact on our data. For the 0-6 month responses, we inputted 3 and included an asterix. For the 'old' response we took the oldest age in the data set (26) and replaced 'old' with that number, including an asterix.

After grading and inputting the data, all but two people received perfect scores on the first test. We discussed how we couldn't use this data since we are looking for improvement in math ability after attempting this puzzle, and we can't show improvement if everyone scored perfectly the first time. I quickly made another test that was more difficult (basic algebra, roots, percentages), had students complete it, graded it, and the scores were much more varied.

This turned out great, even though my veins went to ice when I looked at the initial scores. Students saw that our question couldn't be answered with the data set we so carefully constructed and we had to start again. This demonstrates the 'messiness' of statistics I try to get across to them and how you really have to rely on sound statistical principles, and your understanding of the context to get good data.

Creating the sample was now fairly simple, we paired people based on their initial scores and randomly assigned one to the control group and one to the experimental group. In the pairing we noticed that we had an odd number of people. I asked if there was an observational unit (person) that their initial test and survey information seemed to be outside everyone else's. We decided to remove the 'old' entry from above, since it did not seem comparable with the others. Once we did that we created each of the groups.

The experimental group then had 10 minutes to work on the puzzle. I did not say they had to complete it, just that they had to attempt it. The control group worked on the part of the activity that was to be turned in, descriptive statistics and a box-and-whisker plot of the four data sets (pre-test/post-test, control/experiment). Once the time was up I made another quick test, administered it, we graded it, and I collected the scores.

Getting all the data together there was a distinct difference in the control and experimental group. Averaging the before and after puzzle scores for both groups, it was clear that the experimental group's averages were about ten percent higher than the control group's. I then found the differences in scores, averaged those, and found the percent increase to be about six percent. These two averages came out to two different numbers, but I asked which one would I pick to use to sell my puzzle? We then had a conversation about these sorts of averages, how to compute other similar numbers, and how marketers do similar things in their promotional materials.

Overall I thought it was great, and would probably keep most of it, including the too-easy of tests. I would be a little more prepared and have some back-up tests to use, but it really demonstrated that your initial plan sometimes doesn't work. I would like to include scatterplots and linear regression models next time, but it is not included in our Statistics I course.

I could have done a better job of demonstrating how to control for a variable, and used some basic descriptive techniques to do so. For example, breaking the control and experimental groups into sex, handedness, or work status could show if there were any significant differences in these groups.

Thank you for reading these posts about me struggling through the planning, development, and execution of this activity. If you have any thoughts or questions feel free to post them below.

Forming an interesting in-class Statistics activity - Part II

After the last post I spent some time thinking about my options for an experiment to do in-class that fulfilled all of my requirements. After some thought I decided on the following plan.

• Claim to the class that I have a logic puzzle that I believe will help people with their math skills. I want to be able to put something on my website saying that this puzzle helped people with their math by some incredible percentage.
• Walk students through how I was to do this, including samples, factors to control for, and how to create the tests. I guided the conversation so that we would also collect information on how long ago they took their previous math class, to ensure that the control and experimental groups had a variety of math ability levels (measured by an initial test), to anonymize the results, and discuss how we would use the class setting in the most appropriate manner.
• I created a packet with all the necessary tests, the puzzle, and data recording forms. When printed I assigned random numbers to each packet. I would hand out these packets at this time, asking students to remember their number.
• Students took the first test, I collect it, hand it back out randomly, and we graded the results.
• Collecting the data in Excel on the projector, we would then make our sample based on the survey questions we agreed on and the scores. At this time we would compare different sampling methods and how to make them work.
• Have the students in the experimental group work the logic puzzle, and the control group start on computing descriptive statistics of the pre-puzzle scores. I would later ask all students to compute descriptive statistics for both tests, both samples, as what they were to turn in.
• After 10 minutes, have all students take a similar test, collect them, hand them back out randomly, grade them, and collect their scores.
• Having collected this information we would then do some basic statistical analysis on each group, comparing means, medians, standard deviations, and possibly five-number summaries. Comparing the means and medians of each group, and comparing the difference in the two scores and then their means, I would create two different reasonable measures for what I could include on my website. I would like to do some linear regressions, but that material isn't covered in this class but in Statistics II. 
• Wrap-up with a discussion on how to make this experiment better. Guide the conversation to the placebo effect (I did mention to everyone that I thought these puzzles would help.), blinding, sources of bias, and anything else students mentioned. 

Here is the packet I put together, Statistics I In-Class Activity: Experiment. Tomorrow's post will be about how the activity worked, what didn't, and how I'd like to change it in the future.

If you have any suggestions, or you use this or a similar activity in your Statistics class, feel free to share by commenting below. Thanks!

Forming an interesting in-class Statistics activity.

This Wednesday I've scheduled an in-class activity that relates some of the ideas we've discussed in my Statistics I class. So far they include surveys, descriptive statistics (mean, median, quartiles, and standard deviation), sampling, experiment design, and observational studies. Writing this on Tuesday morning, I have nothing written down but I have a couple ideas of what I want this activity to include:

• Perform an experiment and/or an observational study. I covered what these are in a lecture, but I really want them to attempt one (or both) on their own so they get a concrete idea of what they are. Doing both would be best since they would then be able to compare the methods of both.
• Take appropriate samples based on the research question. Ideally this would be done with students in the class so they can see how to take appropriate samples.
• Compute and use descriptive statistics to compare and contrast different samples. Quantitative reasoning and analysis are a core focus in statistics. 
• Talk about statistics. Whether it be in groups or in a presentation, forming ideas and communicating them to others is another core skill in statistics.

I also know what I don't want:

• A rote activity that requires no input from students. I want them to struggle with questions about sampling, which statistic to compute, what to do next. Through this struggle I want them to appreciate principles like variability, controlling for certain variables, and how to construct arguments for one action or another.

For an experiment I have a few ideas:

• Performing an intervention dealing with basic math skills. The idea would be to assess if a certain intervention (a logic puzzle, or a game) has any effect on student performance of a basic math test. We would discuss how to create a control, how to create the samples, how to administer the tests, etc. While the direct applicability of this example may be a stretch, it would be a constructed activity that students could perform.
• Have the students develop their own question and proceed from there. This has some issues, primarily because of the unending and uncertain nature of such a question. Without some guidance the possibilities are a bit too large, which generally leaves students not choosing anything at all.
• Provide a context (marketing manager, nurse's aide, etc.), and have students develop an interesting question that can be answered in-class. While being more specific we may not be able to answer such questions in-class. For example, if we wanted to do A/B testing of a website, we would have to construct this example fairly quickly, and perform the experiment on people who weren't part of the development of the website.

For an observational study:

• Given a data set have students develop an appropriate research question, and try to answer it using the data. This could be generalized further by having multiple data sets and have students pick one, or assign different groups different data sets.
• Taking the survey data collected at the beginning of the course, ask an appropriate research question that can be answered either by the previously collected data, or by another survey in class. The survey data is anonymous so it would be difficult to answer some questions.

You can probably guess which way I'm leaning, but I'll post the completed activities tomorrow. Feel free to post in the comments if you have any ideas or suggestions.