Chat GPT, Stats, and Math: What are we going to do?

Along with everyone else freaking out about Chat GPT (John Dickerson did a nice segment on CBS) I thought I might test it out with a few basic questions from classes I teach. This year my focus is on introduction to statistics, so I thought I would start by asking it a few open-ended and computational questions. 

Education Research Writer: Dual-classing Science Writing and Education Writing

As an educator for a few years (uh...16?) I'm seeing a kind of 'blank spot' in the education landscape; individuals who take education research, and write articles and produce other media promoting the teaching practices from this research. 

Contrast that with science writers who make up a whole industry of reading current science research, and writing for publications like Popular Science, National Geographic, and other national magazines. Sure, we might have organizations or institutions (PERTS, the Rand Corporation, Dana Center, etc.) that create initiatives to support the implementation of one set of best practices or another, but we don't have individuals writing about the most recent advances in education research. Granted, there are breakthrough topics every so often that get printed, seemingly everywhere, like the growth mindset based on Dr. Carol Dweck's work. At the same time education topics are not a regular part of most American's media diet. 

An obvious reason might be that science is very accessible and visually impressive to more people than education research. I mean, who wouldn't want to watch a video of rockets taking off, over watching stock footage of students taking a test? I would! It could also be that the American attitude towards education is transactional; you educators give us knowledge, and in turn we pay your salaries. Suffice it to say, I don't agree with this attitude, and believe it likely hurts our educational outcomes compared to other nations'. 

Back to science writers, what do they do? This list of responsibilities from Indeed.com might come in handy:

• "Writing research-based pieces of content to discuss scientific theories, science trends and news
• Adhering to a particular style guide to make sure all content pieces follow company standards
• Conducting interviews with industry experts, including medical doctors, scientists and academic professionals
• Planning content creation and publications
• Keeping updated on breaking news in science and the types of content that an audience wants to read more about
• Proofing written content before publishing
• Selecting supporting documentation to accompany a news story
• Attending conferences, events and meetings to learn about breakthroughs in science"

The job seems bifurcated: half the time you are collecting information through interviews, reading, and additional research, while the other half the time you are creating content. So how might we revise this list of responsibilities for an education research writer? (Unfortunately we can't just say 'education writer' as that includes diverse jobs such as curriculum writer, journalist who reports on education news (not research), among others.) Doing a simple find and replace of 'science' with 'education' on the science writer list of responsibilities we get:

• "Writing research-based pieces of content to discuss education theories, education trends and news
• Adhering to a particular style guide to make sure all content pieces follow company standards
• Conducting interviews with industry experts, including teachers, educators and academic professionals
• Planning content creation and publications
• Keeping updated on breaking news in education and the types of content that an audience wants to read more about
• Proofing written content before publishing
• Selecting supporting documentation to accompany a news story
• Attending conferences, events and meetings to learn about breakthroughs in education "

Changing six words seems to do the trick, but is such a job in demand? Can someone make a living doing this? I have some more thoughts to share on this, and might try it out in the near future. Do you know someone who currently has this job? Let me know in the comments.

Comment on Joe Pitkin's blog post The School Down the Hill From the Ivory Tower

 I wrote a comment responding to my colleague Joe Pitkin's blog post The School Down the Hill From the Ivory Tower. In it he talks about the democratization of information through the internet, and the role community colleges play in higher education. Here is my comment, mainly so I can have it in an accessible spot for me to reference later. 

Thank you for sharing your thoughts Joe! I’m having similar ‘big’ thoughts about community colleges and higher education in general. You’re right, our commitment to providing a high-quality, personalized education to anyone who walks in our doors is our value proposition, and we need support from the state, administrators, and colleagues to do that. Too often I see people from other departments operate under a scarcity model of resources, that to get ‘mine’ I must take from ‘yours’. That isn’t going to get us anywhere.

Where I see the biggest opportunities for growth in higher education is the mismatch between the new role of educators and the instructional methods that have been brought over from that previous age. As information disseminators, sure, lecturing makes sense in that we are to convey information quickly and efficiently. But if our role isn’t to disseminate information, rather to guide students to grow their own understanding of the world and develop skills, then our instructional methods must change.

The part that really bums me out is the fear that many faculty display when asked to do something other than lecture. Yes, it is absolutely scary to do something new professionally, but we can’t keep lecturing. Besides not aligning to our new roles, too much research has been done to show lecturing isn’t an effective instructional method, along with the negative effects it can have on BIPOC and first generation students.

If we don’t adapt to this new role, using effective instructional methods and technologies, then the private market is going to eat our lunch, ala https://www.outlier.org/.

Fall 2021: More on Standards Based Grading

I'm in a book club with some colleagues this summer, and we are reading Grading for Equity by Joe Feldman. I've been meaning to write summaries of my thoughts after our assigned readings, but may wait to do a full book review. We're at Chapter 10 and it feels a bit of a cliff hanger; including any assignments or activities in a student's grade other than assessments of their understanding is inequitable. Our (seemingly) natural response has been to reject this as we have learned that high stakes assessments likely do more harm than good, especially for BIPOC students. The author has alluded to some solutions, which we will dive into in Chapters 11 (Practices that support hope and a growth mindset) and 12 (Practices that 'life the veil'). 

During our last meeting standards based grading came up, and most people seemed interested but intimated. In previous posts I've mentioned how I am balancing flipping my classroom and creating a standards based grading scheme that I'll test out behind the scenes for AY 21-22. Two concerns people had were creating the standards and levels, and the logistics of both recording student's progress and allowing for retakes of assessments. 

Creating the standards can absolutely be a difficult task, but for most textbooks there are standards available. This is absolutely becoming a requirement of the job for many faculty, the ability to create and align standards, and it would be nice to see more institutional and department support be the norm for those efforts. It can take quite a while to do this for any course, especially for courses that have more than five faculty teaching those courses.

For corequisite courses this is a bit trickier as you have to identify prerequisite topics that support the course-level standards. This took our department many, many months (if not years) and not something to be taken lightly. If you are thinking of doing something similar the Dana Center has great resources and tools to support faculty in this effort.

I'm finding that the levels for standards isn't as difficult, mostly because I know the course material, and have experience teaching so I know what areas most students struggle with. For example, to create the following levels I started with the Proficient level as meeting the Learning Objective (LO). From there I went up to Expert and included an additional aspect of this LO that I like students to be able to do, but it isn't necessary. After that I went to Developing, knowing that students struggle with the idea that a function can be defined only by a set of points, and that verbal descriptions of functions can be challenging. Lastly for Novice I knew that students struggle with functions that are not continuous, but I couldn't use that word so I decided on "for functions that are defined for all x."

Learning Objectives	Expert	Proficient	Developing	Novice
Determine the domain, range of a function.	The student can identify the domain and range of a function graphically, tabularly, and by a description, and do so for applications.	The student can identify the domain and range of a function graphically, tabularly, and by a description.	The student can identify the domain and range of a function graphically, but may struggle doing so tabularly, and by a description.	The student can identify the domain and range of functions graphically for functions that are defined for all x.

One thing that is really helping me get through writing these LOs and levels is the fact that I'm not going to be using them for grading this year, and I'll have time to refine them. I plan on using these LOs and standards to create my flipped materials and activities, but for AY 21-22 I won't use them for grades but will still record student progress as if I did. This would (hopefully) provide me feedback as to what standards should be adjusted or removed. I could also test sharing these levels with students, possibly using them for review exercises, editing any that students struggle with reading and understanding... In fact, creating a card sort from these would be pretty easy, save me time, provide good feedback, AND might help students review and understand terminology. 

The logistics aspect is absolutely a little difficult, and I plan on going back to the initial presentation I saw for ideas about how to share information about grades using these LOs and levels. One idea is to create a Canvas Group for each student. This way Canvas will generate a page for that group/student, and I can post their progress.

Another idea is to put it on the student. Give them a chart at the start of the term, explain how to fill it out, and make them responsible for them telling me what they need. Granted I'd have a spreadsheet of my own, but I would hope that by giving them the responsibility of maintaining a record of their progress AND grade the student would see how they are connected. 

In terms of assessments, my first thought is to copy Lewis and Ramos' plan; have a quiz where students can make their first attempt at showing mastery of the LOs for the week. After that have students try again. I'm resistant to using class time as I'll be flipping the course material and will likely need that time to help students. Using office hours sounds good, but maybe doing so in the classroom after class might be a better idea. (A colleague (Hi Hannah!) has been talking about having faculty volunteer time in a classroom to make a grassroots testing center. I like the idea but need to figure out the COVID-related logistics. It would certainly help me!)

I feel like I'm making good progress, but August is already here and I don't have ANY videos or activities made, and am only half-way through completing the LOs and levels. I feel like I have to make a solid effort towards flipping my class soon and get at least 50% of the videos and activities done in August or I might have to scrap it. I have previous homework assignments and materials I can use, but it'd be nice to have most of the course developed by then.

Fall 2021: Thinking about modalities

 A colleagues (If you're reading this, hi Allie!) has been asking some really good questions about how we'll return to campus, and what the experience of everyone teaching online might mean for our face-to-face (F2F) courses. In responding to one of her emails I started the following chart comparing different aspects of online, F2F, and flipped classes. I've since added a few more items. 

	Online	F2F	Flipped
Timed Assessment Logistics	Range of times, online, could be submitted on paper, some auto-evaluated and some faculty evaluated.	Specific time and place, on paper, there could be some variation in questions among students, faculty evaluated.	Same as F2F
Timed Assessment Questions	By necessity of online honesty could be more conceptual, but likely includes computational questions.	More computational, but could shift towards more conceptual if more online assessments include them.	Same as F2F
Learning	Self-taught with reading, videos.	Through in-class lectures, activities.	Partially self-taught with readings and videos, more difficult topics in-class.
Online Discussions	Bread and butter of student and content presence	Almost nonexistent	Same as F2F
Activities	A good number of online classes use activities in groups or individually	About 1/3 of f2f instructors use group-based activities	Unknown
Homework	Required after learning, online discussions, and activities. Mostly auto-evaluated	Required after in-class sessions. Some auto-evaluated, some faculty evaluated.	Required before, after, and during in-class session. Most auto-evaluated, some faculty evaluated.
Projects	Used for some courses and summative assessments.	Very few use projects, possibly upper-level courses and stats.	Same as F2F

It does seem like every implementation of the flipped classroom is a bit different, but I stuck to the common elements; some instruction and/or homework before in-class sessions, doing some (if not all) homework during class, etc. One thread you'll see running throughout is the use of auto-evaluated or faculty evaluated assessment or homework questions. This is becoming a bigger theme for faculty to consider for various reasons. 

• Auto-evaluated work is by definition evaluated by a computer and answers are generated by a computer, meaning students can use computers (Desmos, Photomath, etc.) to answer them. 
• Most of the time the only feedback students get with auto-evaluated work is whether the student is right or wrong. Yes, some systems provide some feedback, but as far as I am aware no system can detect what error was made and provide specific feedback on how not to make that error. Faculty evaluated work on the other hand can provide specific feedback.
• Faculty can save a huge amount of time using auto-evaluated work. 

To balance these concerns of academic honesty, feedback, and time savings it seems some middle ground should be established, like using these systems for specific functions like additional practice on computational skills.

In thinking through the feedback that online homework systems do provide, there is a trend towards using more general growth-mindset type feedback. Having seen some of these examples I worry that they appear tone deaf, and do more harm than good. If a student answers a question in correctly, seeing "Keep trying. We learn from mistakes." auto-generated by the system might not have the same effect as a human providing that feedback AND identifying what the mistake was and how to prevent it in the future. Another colleague is very interested in growth-mindset based feedback, and it seems clear that you have to do a lot of thinking about the student, their work, and what message would support the student to provide effective feedback. An automated message does not seem comparable. 

What do you think? Are online courses going to start resembling f2f courses? Are f2f courses going to start resembling online courses? What is your stance on auto-evaluated work or faculty evaluated work? I'd love to hear what you think. 

Planning for the Fall: Flipping my class and shadow grading

 In a previous post (Pre-Pre-(Pre?)-Planning: Fall 2021 - Standards based grading, flipped classroom, corequisite support, and maybe group work?) I discussed my plans for the summer. I just completed the course Flipped Plus Model taught by Jessica Bernards and Wendy Fresh, and my plans for my fall course have changed slightly. This course I am teaching is a corequisite support version of our 100-level business math class, and contains both prerequisite math topics, and success skill assignments. This will be the first time I have taught a face-to-face course since March of last year, and for various reasons 5/8 of the course (100-level math content) will be face-to-face, with the remaining 3/8 (prereq math topics and success skill assignments) online.

In that previous post I discussed standards based grading, and while it is something I'm still interested in I think I am going to focus for the month of August on 'flipping' my course. This is half inspired by Flipped Plus Model and the wealth of tools and thoughts the instructors shared during the workshop, a quarter inspired by a past colleague who used the method to great effect (Hi Joan if you're reading this!), and a quarter inspired by the fact that for this coreq course I have been using an adaptive homework system for the prerequisite content that I have not been entirely happy with. On this last point, I felt a little too 'locked in' to their content and while I had the option to ask for additional standards and materials I feel more comfortable creating my own content and changing it as needed.

So does this mean I'm going to drop the standards based grading component? For students, yes. For myself, no. To determine what videos to create I am going to continue creating my own student learning objectives (SLOs) for the course, delineate the expert, proficient, developing, and novice levels, and build my videos, lessons, activities, and homework around them. Below is an example of drafts of two SLOs and their levels. 

1.1 Find the equation of a line given information about its graph.

Expert - The student can find the equation of a line given a variety of information about its slope, if the line is parallel or perpendicular to other line(s) or points (individual or intercepts), or the graph of the line. 

Proficient - The student can find the equation of a line given a variety of information about its slope, if the line is parallel or perpendicular to other line(s) or points (individual or intercepts), or the graph of the line. Student may still struggle with finding the equation of vertical or horizontal lines.

Developing - The student can find the equation of a line given the graph of the line. Student struggles with finding the equation of the line given information about its slope, if the line is parallel or perpendicular to other line(s) or points (individual or intercepts). Student still struggles with finding the equation of vertical or horizontal lines.

Novice - The student can sometimes find the equation of a line given its graph. 

1.1 Graph a line given its equation, points, table, or other information.

Expert - The student can graph a line given its equation, points, table, or a verbal description of the line.

Proficient - The student can graph a line given its equation, points, or a table of values.

Developing - The student can graph a line given points, or a table of values, and can sometimes graph a line given its equation.

Novice - The student can sometimes graph a line given its equation, points, or a table of values.

By the end of August I hope to have all of these SLOs created (100-level and prereq topics), videos supporting them, learning assignments students should complete before class (part of the 'flip'), activities for the start of class to dial in the misconceptions or difficulties with the material, practice assignments (homework) for students to complete during and after class, and an alignment of learning assignments, activities, practice assignments, and homework to the SLOs. 

I'll use a conventional weighted grading system (25% final, 25% quizzes, 25% assignments, 15% activities, 10% success skills) BUT if I have the SLOs aligned to all of these assignments I could shadow grade them according to a standards based scheme. Doing this for a year would have a number of benefits;

• I don't have to convince students of two new things I am doing; flipping and standards based grading. Having taught with POGIL in the past I am experienced with addressing student concerns proactively around new instructional methods, but I am still uncertain of the specific issues students will have with either of these systems. Doing one at a time I can fully address the concerns of one before moving on to the next. 
• I can use the year's data to convince students next year that this grading method would either improve their grades, or not harm them. 
• While a bit of extra work (two grading schemes) if I get all my materials prepped I can used that saved time towards this additional grading. I'm not entirely convinced that I have to grade throughout the term, I could do this additional grading at the end of the term. 

With the fall return I would like to have as much of my course 'figured out', and the Flipped Plus Model workshop absolutely helped with that. Below are slides from my first day presentation I've updated after the workshop, hopefully explaining to students what we'll be doing. 

[Figure 1: Flow chart and bulleted list of course assignments and activities meant to answer the question "How are we going to learn?"]

[Figure 2: A table showing the hours between 11:00 AM and 2:30 PM of Monday, Wednesday, and Friday, filled out with the various assignments and activities meant to answer the question "When are we going to learn?"]

So where is this going to break? What is going to happen to unravel all this careful planning? Could it be student resistance? I'm hoping by giving a good 4/9 of class time to homework students will be motivated to try to complete it in order to reduce their workload outside of class. Granted getting them to complete assignments an hour before class might be a bit of a challenge. This 'flipping' of when students complete their homework is entirely the point, but I will have to work a bit at selling it. 

I could also break during August, trying to get all this done with my other work. I'm still late on a few projects from Spring, which I hope to wrap up this week. Having the list of SLOs is going to be really helpful, that way I have a list of specific topics to address and I don't have to worry about what I'm creating videos on. I'm not terribly concerned with the how of creating videos, I've been using OBS for quite a while now and have a good idea of what I want to create.

The only thing I'm still struggling with is the recommendation from the Flipped Plus Model to create guided notes for my videos. If I use Google slides to create them, and use those slides to create the videos that will cut down quite a bit on production time. I guess my resistance is more a gut feeling about 'spoon feeding' students, and yet a number of my colleagues have successfully used similar guided notes. I suppose I'll just have to let my ego take a hit if it means supporting students. 

What am I not seeing? Where are things going to go wrong? Be my Karnak and help me find the weakness in this plan. 

Pre-Pre-(Pre?)-Planning: Fall 2021 - Standards based grading, flipped classroom, corequisite support, and maybe group work?

After putting together a grant proposal and wanting to do something other than the list of tasks on my plate left over from last term, I thought I'd spend today prepping for my Fall 2021 course; face-to-face business math. I was really inspired by Matthew Lewis & Michal Ramos from Yakima Valley College, and their talk Thinking About Assessment and Grading Differently: Standards‑Based Grading in Precalculus at WAMATYC this year. Students can get so focused on points and grading, that they act as grades aren't related to understanding. In watching their talk I am hopeful that tying a student's grades directly to their understanding of specific learning objectives, they will be more motivated to do things that actually impact their grade.

This chart really hit home how standards based grading could be accomplished, which I have shared in a past blog post. 

In reviewing their materials it seems that I have a few choices to make. 

• How should I assess the different levels of understanding?
  While they focused on online assessments (using WAMAP) being an online course, I'm thinking of moving some of them to face-to-face assessments. Their plan to have students complete the different levels of understanding of an objective sequentially (An Expert assessment doesn't open up until the Proficient assessment is 100%.) is well done in the system, and the questions themselves are great! Of the Expert questions I looked at, they included things like randomly generated values (a standard in WAMAP),  analyzing an incorrect answer, and short question path with question parts that opened up based on previous responses. All of these things avoid the 'static' questions I see in so many online systems, and would help prevent sharing of solutions and cheating. 
  
  My current thought is to have a similar system in WAMAP where students can complete assignments (homework really) aligned to the Novice and Developing levels of each objective, and students can retake them as much as they want with no penalty. This is also a corequisite support course, so expanding these novice and developing levels to include prerequisite knowledge would seem appropriate. Expert and Proficient levels of objectives would be assessed through face-to-face weekly quizzes, and some other types of assignments, possibly projects, discussion forums, or something else, I'm not sure. I like to limit quizzes to 30 minutes, 5-6 questions, and at the low end this means that I'm limited to something like 25 objectives (5 questions per week * 10 weeks * 1/2 for the number of objectives (Expert and Proficient) I can assess) which seems doable. That leads me to...
• What objectives are you going to focus on?
  Since assessments are directly related to objectives, I'm constrained by how many objectives to include. These would absolutely need to be student readable, so including vague or abstract objectives that 'cover' a lot of content would not be good.
• How to handle retakes?
  Students are not going to get things right the first time, and I need to figure out when this is going to happen. Do I plan on some time at the end of class (10 min) where students can complete a question?
  
• What will feedback look like?
  In focusing on the specific learning objective, and in creating thoughtful rubrics, I am hoping my feedback will be more targeted about the specific area of confusion. Obviously I'll keep providing process feedback, and discussing ways for students to improve their work. 
• No due dates?
  Really? Am I going to be ok with that? In making the assessments the main way students can reach higher levels of each objective I'm hoping they nudge students into preparing for them. At the same time I need to be deliberate and focus their attention on completing their work on a regular basis. 

Because this is a corequisite support course I have a number of success skill assignments tied to different components of the course. The time management assignments are ok, but I feel students don't connect when they should complete their work. I'll be trying to work backwards from the assessments, to the assignments, to what they should be doing each day. I have other such assignments on growth mindset, deep versus shallow processing, etc. but I'm wondering if maybe I should focus on time management as the central skill, and bringing up other affective domain skills through it. For example, sure you block out time for homework, but what should you be doing (and not doing) during that time? If you're feeling unmotivated to complete your schedule, why, what are those unresolved feelings that are motivating your procrastination. 

The course includes eight hours of instruction (5 college-level, 3 pre-college level) but I am only meeting for five hours a week, with the other three being online. In flipping some of that material, especially the prerequisite pre-college material, I'm hoping to make sure students are prepared for class sessions. In not having due dates I'm a little concerned about students not completing the flipped instruction, how else am I going to get them to do it?

For now I feel like I have more questions than answers, but over the summer I'm hoping to dial in a course plan I can use for the year. Here is roughly what I'm thinking;

- July: Decide on course objectives, levels, rubrics, and create some of that material in WAMAP. 

- July 20, 22, 26: I am taking the Flipped Plus Model PCC continuing education course run by Wendy Fresh and Jessica Bernards.

- August: Create flipped instruction videos and linked class materials.

- September: Revisit Success Skill assignments, and decide what would be helpful to students. 

Given my other projects this feels manageable if I spend 10 hours a week on it.

Your thoughts on any of this? Something I'm not seeing or overlooking? Am I going overboard? Let me know in the comments.