The unauthorized A.I. use long game OR How not to burn out my empathy circuits in an online stats course

 It is Week 1 in my online introduction to statistics course, and I've already detected unauthorized A.I. use in discussion forums. I'm not going to share the details of how I know, but in addition to technical details the writing is competent, factual, and bland. In just reading the suspected posts, they included course concepts we have yet to talk about, not really address the points being asked to discuss, and were just sad. 

In looking at many of these student's posts and in the online discussions I'm having with other faculty, I get the sense that many were using A.I. because they didn't feel like their honest writing would be appreciated, or awarded credit. This bums me out, as the struggle to make yourself understood is something every human engages in, and in statistics, it is a central problem. By using A.I. to write their posts for them, these students are missing the opportunity to struggle through these ideas. 

For example, I posted the following in a general announcement;

Data

A source of data that I really enjoy teasing apart is you! Each interaction with Canvas leaves behind some data, and as your instructor for a statistics course, I enjoy diving into that data and thinking about what it tells me about you. One interesting data set records how much time students actively spend in Canvas. Below is the data set and a few questions you may want to consider. The data is formatted so that each row is one student, the first column is the number of hours they have spent actively in our Canvas course, and the number of minutes is the number of minutes (minus the number of hours) they have spent in our Canvas course. So for example a 1 and then 51 would represent a student who spent 1 hour and 51 minutes in the course. 

Hours	Minutes
1	11
1	14
1	1
2	10
1	30
0	48
2	0
0	14
0	41
14	40
2	59
4	5
0	12
0	50
2	47
0	47
2	24
0	38
1	35
5	37
6	25
1	51
0	25
0	41
1	23

• Do there seem to be 'unusual' values in this data set?
• Do some values seem more likely than others? 
• If you have spent more than the average amount of time in the course than your colleagues, what might you do (if anything) differently?
• If you have spent less than the average amount of time in the course than your colleagues, what should you do differently?

Of the responses I got, one had the technical and linguist hallmarks of A.I. use. Again, this bummed me out, as asking about these ideas of what values seem 'unusual' or seem more likely than others leads directly to measures of center and variability. I am offering students the opportunity to think through these ideas, building a need for mean and standard deviation, but in using A.I. students don't get any of that. They don't build the foundation for these ideas, and therefore likely won't understand these ideas, turning to more A.I. use. 

In the same announcement I shared the following;

If you find yourself using A.I. for course assignments, please talk to me. I am concerned that if you start using A.I. now you may not be able to stop throughout your college career, as you won't have the prerequisite knowledge to complete future courses. A college degree is not just about getting a better job, it is meant to be a marker of understanding about the world and your discipline in a meaningful way. Using A.I. for course assignments takes that opportunity away from you. 

To be clear, if I discover unauthorized A.I. use in any of our graded assignments, I'll award a zero (0) for the assignment the first time, and a failing grade (F) for the course the second time. This is not the outcome I want for any student, and I will do the necessary work of maintaining the clear academic standards I have communicated to all students in any course I am the instructor of record for. 

I know, reads a lot like Abe... Or maybe Skinner.

But look, I'm not budging on expecting students to write their own thoughts, as a form of learning. That's the job. I believe in their ability to grow, and capacity to learn statistics. If I have to do that in spite of students, I'm ok with that. I'd rather not. I'd rather students understand the utility of statistics and math, and use them as tools to make better decisions. It certainly feels like I have to determine the level of this I am willing to put up with, and invest a corresponding amount of time.

On that note, I don't think I'll be able to investigate each and every instance of unauthorized A.I. use. There is just enough plausible deniability of using A.I. in many assignments, that I can't 'prove' A.I. use in all cases. 

Note-Taking Assignments – Given that students are expected to hand write these notes, it is unlikely that they will use A.I. Unlikely, but not impossible.

Student ability to submit A.I. generate work - Unlikely
Ability to confirm unauthorized A.I. use with technical means - No
Ability to confirm unauthorized A.I. use with textual means - Unlikely or difficult

Discussion Forums  - This area seems ripe for students to use A.I., as most of these will either be a regular Statistics in the News discussion forum, or a Homework Help discussion forum. 

Student ability to submit A.I. generate work - High
Ability to confirm unauthorized A.I. use with technical means - Some
Ability to confirm unauthorized A.I. use with textual means - Unlikely or difficult

Homework – I use a MyOpenMath related site (WAMAP) where many of the questions were written by faculty from across Washington state. It is already known that many of the questions are on websites like Chegg and MyCourseHero. With A.I. it is trivial to copy and paste the questions into a chatbot, and even for questions that use a graph a screenshot of the question into the chatbot will suffice. There is only one technical way I know of determining if a student is using A.I. or not. 

Student ability to submit A.I. generate work - High
Ability to confirm unauthorized A.I. use with technical means - One method that can be unreliable.
Ability to confirm unauthorized A.I. use with textual means - None

Quizzes - Again I am using the same site for homework, so it is likely that a student will be able to use A.I. for these quizzes. For some questions I will require students to submit written work, so that may reduce the amount of A.I. use, however it will work counter to the one technical method I have for detecting A.I. use. 

Student ability to submit A.I. generate work - High
Ability to confirm unauthorized A.I. use with technical means - One method that can be unreliable.
Ability to confirm unauthorized A.I. use with textual means - None

Excel Skills/Applications - This is the biggest question mark as I used these assignments before November 2022. In Excel Skills assignments I provide students with an Excel file, and on different worksheets walk them through a concept or computation, and on other worksheets ask them to apply these ideas. In Excel Applications I have three worksheets that have students answer question using the skills they learned in the Excel Skills assignments, with what we have learned in class. 

Student ability to submit A.I. generate work - Unknown
Ability to confirm unauthorized A.I. use with technical means - Unknown
Ability to confirm unauthorized A.I. use with textual means - Unknown

Course Project - This is where I have a plan to detect A.I. use, and I am unsure of how students will react. The idea is to provide each student with their own Google Document, where they will draft their project submissions. 

Student ability to submit A.I. generate work - High
Ability to confirm unauthorized A.I. use with technical means - One method that I have yet to explore.
Ability to confirm unauthorized A.I. use with textual means - Unlikely or difficult

From that analysis I come to the following table;

From this I am feeling the most confident about identifying A.I. use in the Course Project, and possibly the Excel Skills/Applications. Homework and Quizzes. I am thinking of using my detection of A.I. use in Discussion Forums as a kind of signal of students who may be prone to use it in other assignments. If and when I submit a zero for an assignment I will likely do with a package of evidence that they have used A.I. for other assignments.

To be clear, this sucks. Constantly playing a Turing test on every student submission is not what I signed up for, and wonder if I just don't teach online after this year. 

What do you think? Am I being overly prescriptive? Paranoid? Is there a way to actually get students to understand that learning takes effort, and the smoothness of A.I. use is limiting their own growth?

Three ideas of how to help students use AI that do not include copying and pasting my assignments directly into a chatbot

 In my last post I shared my AI policy (in short: red light) and that I included the following in my syllabus. 

To be clear this means using generative A.I. for assignments is not allowed, and there are other aspects of learning and studying a student could use these tools for. To help guide students towards useful and acceptable uses of this technology, each Weekly Overview will contain a recommended activity students could use this technology for, in addition to other information.  

I have to write these anyway, so here are the first three ideas I have. Again, this is for an introduction to statistics course.

Learning Outcomes Help

In past terms I have tried standards based grading to mixed results. However in doing that work I have a pretty good set of learning outcomes I want students to accomplish each term. They were initially taken from our current textbook, and over time I realized which ones I wanted to assess students on. This summer I spent time aligning my homework questions, activities, and assessment questions on these outcomes. 

A persistent issue with learning outcomes is that, unless used to generate students grade, many students don't know what they are, and what to do with them. The ideal would be that students refer to them regularly as they are learning the course material over the week, and as the week progresses they assess themselves as whether they know how to accomplish each objective. Unfortunately humans are generally task oriented, and in turn students believe doing well in their courses is a matter of accomplishing tasks, and not developing the understanding of course concepts. I get the sense students 'get it' in an abstract sense, that they are there to learn, but there is something tangible about a task that is not inherent to an outcome or an objective. To be clear I do not judge students for this, and I take pains to explain how the assignments (tasks) build on their understanding.

This is where I think AI can come in and help both myself and students, by having students complete the following task. 

This week I am going to suggest that you use AI like a test prep tutor. Our first assessment will contain questions you have not seen before, and to prepare for that assessment you should probably do the same, answer new questions. After you complete the assignments for the week, for each learning outcome this week, ask your generative AI of choice for three questions aligned to this outcome, one easy, one of medium difficulty, and one of a hard difficulty. Then try answering those questions to determine if you really can do the learning outcome. 

2A Describe how the selection of an appropriate graphical display depends on the number of variables in the data set, the data type, and the purpose of the graphical display. 

2B Construct graphical displays for categorical data (bar charts and comparative bar charts) and for numerical date (dotplots and comparative dotplots, histograms).

2C Describe the distribution of a numerical variable in terms of shape, center variability, gaps, and outliers. 

2D Describe similarities and differences in the data distributions of two or more groups. 

2F Describe trends over time based on a time series plot. 

2H Critically evaluate graphical displays that appear in newspaper, magazines, and advertisements. 

For example this would look like "For the following learning outcome, write an easy, medium, and difficult question. 2A Describe how the selection of an appropriate graphical display depends on the number of variables in the data set, the data type, and the purpose of the graphical display." After attempting these questions yourself you can ask for solutions, and then compare your solution. If these answers are radically different then you may want to review the prior assignments, your notes, and/or ask for further clarification from myself.

Do be critical of the questions that are posed. If any seem odd or strange, you are welcome to share them with me, and your thoughts on them and their solutions. 

Essentially they are using AI to generate assessment questions aligned to these outcomes. I am curious that if students do the above to see what kinds of questions they will get, whether they will ask for variations, and if they will be more focused on the outcome instead of individual questions. 

Textbook Reading Help

As mentioned in my previous post, I have students take handwritten (paper or digital) notes on the textbook, and have had them do this for about two years now. One persistent issue is that students will sometimes come to a phrase or term, and don't know what they mean. For the second week this is what I am thinking of asking students. 

This week we will be learning about a variety of computations with specific notation, terminology, and descriptions. As you complete the Note-Taking Assignments for Chapter 3, I want you to be on the look out for confusing terms and phrases. Circle these, or put a start next to them on your notes. After our Tuesday class, go back to your notes and enter these terms and phrases into your AI of choice, and ask it to explain what they mean. For example you could enter the following into a chatbot "What does it mean to be a measure of center? Can you use a baseball example to describe it?" 

 As you read these examples, refer back to your Note-Taking Assignments, in-class activities, and homework to see how well they match what we are learning. 

Correlation and Causation Help

In the third week I talk about correlation and causation within the context of linear regressions. This is usually the moment where students start seeing statistics that are beyond descriptions of groups or trends, and how 'messy' statistics can really be. Sometimes there isn't a clear statement you can draw from a two-dimensional numerical data set, and that frustrates students to no end. 

In short we generally can't determine if one variable causes and effect of another variable based solely on them being correlated. The website Spurious Correlations has myriad number of variables that are correlated, but do not have a causal relationship. To determine that we would need to perform an experiment, controlling for other variables. 

Anyway, that's a long way to saying that students struggle with this idea. Similar to the last suggestion, I am thinking of the following. 

This week we will be discussing the difference between correlation and causation, which can be a confusing concept for many students. After attending Tuesday's class, consider entering the following into your chatbot of preference; "Provide three examples of varying levels of causation given a correlation between two variables." You are welcome to specify a discipline (business, biology, etc.) or a personal interest for these examples.

Again, as you read these examples, refer back to your Note-Taking Assignments, in-class activities, and homework to see how well they match what we are learning. 

There is an aspect of culturally relevant curriculum that I struggle with in some of these suggestions; to integrate student interests within instruction. I struggle with this because a big part of college is being exposed to new ideas, and to stretch what 'interests' us. For students who will be more motivated by examples relevant to their interests, I can AI doing that work fairly well.

You can also see me asking students to be critical of AI, and you can guess as to why. Some of these systems produce inaccurate facts, and can engage in 'sycophantic modes' that I believe are incompatible with critical thought.

Thoughts? Suggestions? Accusations? Other ideas on how students could use AI for learning? Again, I am a fulsome 'no' on having students copying and pasting my assignments into AI. I'm giving one strike before an 'F'. At the same time I do see potential for AI, if it used the right way. Hopefully the above will have students try it in a way that doesn't take their learning away from them. 

Grading, Late Work, and AI in an Online Stats Class: What I am trying.

With another academic year comes another set of questions I have to answer; What assignments do I want students to complete? How? When? Why? What if they use AI? How can I structure my courses to reduce the likelihood of students turning to AI? In this post I'll share three decisions I've come to for the online statistics course I am teaching this fall; what and how I am grading, what are my late policies, and what are the consequences of unauthorized AI use?

How and What I am Grading

Below is what I have written in my syllabus for what I'll be grading. Much of this is informed by Grading for Equity by Joe Feldman, and Uncommon Sense Teaching by Oakley, Rogowsky, and Sejnowski. 

Grade Composition

Grades are generated from assignments and assessments that have students apply course information, in order to develop their understanding of statistics. Parentheses () include the percentage of the total course grade this component makes up. 

This preamble seemed important to me to clarify that the goal is to have students apply statistics. I don't want or need AI to answer any question. 

Note-Taking Assignments (17%) – To start the learning process students are asked to take handwritten notes on the textbook before class, as this has demonstrated greater learning gains than typed notes.[1] Students may elect to take handwritten notes on paper or on a digital device, however no screenshots and/or copying and pasting of text is allowed. You’ll be asked to try different note taking methods, answer questions, and find ways of taking notes that work best for you. Grading will be done according to a rubric that is focused on taking good notes, not being perfect or correct.  Due Tuesdays and Thursdays by 12:00 PM (noon).

To get students thinking about the course material I have them take handwritten notes from the textbook. Is this antiquated, old-fashioned, and time consuming? Yes, and that's the point. Learning is slow, reading takes time, and the pace of both compliment each other. 

Discussion Forums (17%) - To continue the learning process, students will engage with the course material in a collaborative setting. A variety of discussion forums will be presented throughout the term, and students will be expected to discuss the course topics with other students and the instructor. Grading will be done according to a rubric that is focused on collaborating and learning with your colleagues, not being perfect or correct. Initial and response posts due alternating Wednesdays by 10:00 PM.

I'm a little unsure what I want these discussion forums to look like. I would like at least one asking students to find statistics in an article or video they find interesting, and then have students ask questions in replies.  

Homework (17%) – To practice applying course concepts, and recall of course ideas, students will complete homework assignments on WAMAP. Grading is done on correctness of answers and solutions. Due Mondays by 10:00 PM.

Starting with a homework set in WAMAP from a colleague (Thanks Allie!) I've aligned them to the learning outcomes from the textbook that I am assessing. I do have a running list of questions I need to write and add, but that is something I can juggle later on.  

Quizzes (10%) - You will have five open-book, open-note quizzes throughout the course. Some questions will ask you to write and submit your work.  These may be taken twice and the highest score is recorded. Grading is done on correctness of answers and solutions. Due every other Friday by 10:00 PM. 

Professor Tip: It will be important to keep current with the course material to do well on quizzes. If you don’t do well on a quiz take this as a signal that you should change something about your approach to the course.

The major assessment of the course is project based (more below) and I'd like students to answer a few questions to apply what they have learned. I am using this format from a colleague (Thanks Kate!).  

Excel Skills/Applications (15%) - This section (version) of MATH&146 is intended to help Business Administration, Accounting, and other business students develop their skills with Excel. This set of ten (10) Excel assignments were developed in collaboration with business faculty, and will have students answer statistics questions with Excel. The assignments consist of an Excel Skills assignment that has students use and explore various functions within Excel, and then Excel Applications, having students use Excel formulas and features to answer statistics questions. Due weekly on Fridays by 10:00 PM.

I really like these assignments, and have yet to run them through AI... 

☆ Course Project (24%) - To make course concepts and principles relevant students will pose a question, and spend the term answering it using statistics. There are five parts to the course project, with each part building on the last. 

• Course Project Part 1 (2%) - Explore the Data & Variables
• Course Project Part 2 (3%) - Statistics & Graphs
• Course Project Part 3 (4%) - Relationships Between Variables
• Course Project Part 4 (5%) - Confidence Intervals
• Course Project Part 5 (5%) - Peer Review with Video Presentation
• Course Project (5%) - Culminating Project with Video Presentation

Students will be sent a single Google Doc to write their submissions. Only that file will be accepted for grading. It is expected that students will add onto their project parts for each submission. At the end of the term students will have a completed article describing a research question, detailing their attempts at completing it, and summarizing their statistics and conclusions. 

Thanks to Jennifer Ward, Allie Dykes, and Kate Cook for sharing this project and their iterations. I will be creating a single Google doc using my own account for students to write their submissions. Revision history will be used if I suspect AI use. 

Recommended Questions – Within each of the Weekly Overview pages on Canvas, there will be a set of recommended questions for each Learning Objective.

Professor Tip: Practice makes progress. While the above assignments do not calculate into your course grade, they will be graded to provide feedback as to your understanding of the course material, and progress towards learning mathematical notation and formatting. Failure to complete them will negatively impact your ability to participate in class activities and complete assessments.

This is something I'll create on a weekly basis in order for students to get additional practice.  

Late Work Policies

I've spent a lot of time thinking about late work policies and how to be equitable. After conversations with colleagues (Thanks Hannah and Michelle!) I am instituting a few hard due dates for discussion forums and quizzes. In both cases they will be open for quite a bit of time, and students can reattempt Quizzes. 

Late Work Policies

• If students do not complete an assignment by its due date, it is LATE. For all LATE assignments, a zero (0) will be entered into the gradebook. This is to ensure that students understand their current grade, and can take informed steps to improve.

One thing I see many instructors get in trouble with in Canvas is not entering zero grades after a due date. Canvas can only record a student's grade based on what is entered, so if you only enter a 100% on that intro assignment from Week 1 a student will (wrongfully) think they have a 100% in the course in Week 10.  

• Students will have until Wednesday, December 10th at 10:00 PM to complete LATE Note-Taking Assignments and Homework. Note that it is in your best interest to complete assignments on time, as other course components will rely on knowing this information. Any submission after this date will be marked zero (0).
  Professor Tip: The previous two points might seem to contain contradictory messages (allowing  and “complete assignments on time”) but I do not see them that way. I want to provide students guidance as to when to complete assignments, and I want to give you some flexibility IF you need it. Using this late work policy 2-3 times during a term is fine. Using this late work policy once a week or more is a signal that something you are doing needs to change.

This is what I have used in the past and am wondering if drawing attention to this fact helps or hurts students. I want to provide flexibility, and many students need firm deadlines... To be honest I don't feel like I should be responsible to 'motivate' students to complete assignments through deadlines, but here we are. 

• Students may not complete LATE Discussion Forums, given that the learning activity is meant to be completed in collaboration with other students with their current level of understanding. Any submission after the due date will be marked zero (0).

Because discussion forums are asynchronous with a day limit, I feel confident in setting this deadline. The purpose isn't the post, it is the conversation with colleagues, the sorting out of ideas by discussing them is the goal. 

• Students may not complete LATE Quizzes, given that they will have a week to complete it and will have two attempts. Any submission after the due date will be marked zero (0).

We'll see how this goes.  

• Students may have an additional two (2) days (as in 48 hours) for LATE Excel Skills and Application assignments. Any submission after this date will be marked zero (0).
• Students may have an additional two (2) days (as in 48 hours) for LATE Course Projects. Any submission after this date will be marked zero (0).

I've built this into my grading so I don't mind if students are a day or two late.  

• All submissions must be complete by Wednesday, December 10th at 10:00 PM.

All prior due dates will be by Monday, December 8th at 10:00 PM, so students can have a the extra two days for the project and homework.  

AI Policy

None of this is revolutionary, and no, I don't want or need students to use AI in our course. I will be giving some ideas of what they could use AI for (writing practice questions, taking apart learning objectives, etc.) to provide some level of guidance. 

Generative Artificial Intelligence (Gen A.I.) Policy

I expect that all work students submit for this course will be their own. The use of ChatGPT, Google Bard, or any other generative artificial intelligence (AI) tools at all stages of the work process, including preliminary ones such as creating drafts, generating ideas, etc., are not allowed. Violations of this policy will be considered academic dishonesty. Note that different classes at Clark could implement different AI policies, and it is your responsibility to understand the expectations for each course. 

To be clear this means using generative A.I. for assignments is not allowed, and there are other aspects of learning and studying a student could use these tools for. To help guide students towards useful and acceptable uses of this technology, each Weekly Overview will contain a recommended activity students could use this technology for, in addition to other information.  

Writing a syllabus feels like writing a check that you need to cash by following through and making all of these assignments, rubrics, assessments, and then grading them all. I will try to reflect on these policies in December, after the term ends, as I am trimming the Christmas tree.