I've been home for a few days and I think I'm just now getting my equilibrium back. I've had quite a few meetings with faculty and administrators, am in the middle of grading an assessment, am trying to put together the rationale for why we're updating our coreq support courses, trying to train new coreq instructors, and make sure to spend time with my family. I'm hoping to get through the rest of the conference by December? January? Don't quote me.
11:30 AM - On the Use of Reflective Writing Assignments
Anurag Katyal from Palm Beach State College had a great presentation on his use of a variety of readings, prompts, and quotes to get students to reflect on their own math experiences. He referenced Reflective Writing in Mathematics Education Programmes by McNaught (2010), so I'll have to read that at some point. Anurag discussed a variety of assignment logistics (time, length, etc.) he used as part of his continuous improvement of his teaching practice. I'm glad to hear I'm not the only one who tinkers.
The big takeaway was Anurag's rubric that he developed with colleagues in the English Department. I'm looking forward to reviewing it (as of 11/21 I haven't gotten a copy yet) and trying to borrow what I can. As I have no training in evaluating written work, my focus has always been evaluating a piece of writing for the student's intent. Are they looking to get this done as quickly as possible, or was the assignment done with sufficient completion, a term I've used in the past to mean:
"Making
a good faith attempt at answering all questions in an assignment, showing your
work, and completing the entire activity to the best of your ability. If you do
not meet this criteria for an assignment that uses sufficient completion you
will earn a zero (0) for that assignment. "
I was also impressed by the variety of readings Anurag asked students to complete and reflect on. I'm hoping to get that list and see how I can incorporate them into my courses. I routinely use my LMS's quiz feature for Check-Ins, short assignments asking students for feedback, to reflect, and to answer mathematical questions. Most of the feedback is centered around group work, and most reflection is around questions like "What was the stickiest point?", "What do you wonder?", "What is a question that would build understanding?", "What remaining questions do you have?" I've included some longer form reflections that require a reading, but these are kind of sporadic. It may be nice to organize these smaller reflections, interleaved with these larger reflections in a consistent way, possibly every Friday a bigger reflection....
12:00 - Mathematics Anxiety: What is it? Who has it? What can be done?
There was a good amount of references to look over (math anxiety was first discussed in 1957 by Dreger and Aiken), acknowledgement of cultural differences (China v. US), deficit theory and cognitive interference theory, who has it, and specific actions faculty can take to address it; 10 minute write-out before an exam where students just write what they know, fewer smaller exams, address negative math attitudes, normalize struggle, acknowledge the struggle of famous people, and identify diverse role models. All-in-all, a good session, but nothing really new for me.
12:40 - Using Faculty Collaboration to Design and Improve Corequisite Courses
This was a good session on a Texas' CCs design and implementation of coreq courses. The level of internal organization of the department was the most interesting part of the presentation, as they had faculty teams with faculty leads for college algebra and business math, statistics, and quantitative reasoning (non-STEM) courses. Each of the three courses had different balances of prerequisite content and course content. The college algebra course was fairly front loaded with prereq content, but more and more course content was added over time. The quantitative reasoning course had some weeks where prereq content was the majority, and other weeks where course content was the majority. The stats course had a fairly even split with an equal number of prerequisite topics each week. Below are some diagrams I sketched in my notes;
The presenters had quite a bit of data, but nothing was disaggregated by race. I asked whether they have done so, and they did not. I shared my own experience with not doing the same with an assessment project, and made it pretty clear that we can do better. I got sideways glances from the presenters for the rest of the conference, but I hope I made it clear; we must include equity data in all of our conversations.
I believe I took lunch at this point, at a bar across the street. I talked to a few people over writing questions in MyOpenMath and learned that David Lippman used PHP as the base language to write questions in. Good to know!
1:50 - Hitting the Mark: Organization for Optimal Outcomes
I walked into this session a little late, but Sarah Miller shared her organization methods for both teaching and into a little of her personal life. As someone with the following to-do list for the week, I was definitely interested.
She shared that she has a student intake form that providers her with a range of information about the student. On the back of the form she'll write notes about the student's progress, emails or messages the students sent, and other useful information. This seems close to a case management system an advisor would use for students, but would be really helpful for faculty. Using this she sees patterns, can piece together the student's life, and reach out when necessary.
She shared her methods for organizing her emails. She creates an email folder for each student! In my college's LMS this isn't necessary as you can already look at all the messages a student sent, but in Outlook this would be good only if students emailed me. Since Sarah gets so many emails she uses the 3-2-1-0 method of checking emails 3 times a day, 20 minutes a day, 1 touch. She shared her workflow to decide how to take care of it, or if its a task in itself. I thought that was an interesting way to think about some emails, a bigger project to tackle on its own.
The Thursday Keynote by Francis Su was heartwarming. He really pushed us to remember why we're doing what we're doing; to share a deeply human endeavor with our students. Below is the same (I believe) speech he gave to the MAA in 2017.
After that the exhibit hall opened, and it was a little strange how many people were waiting to get in. I guess there was swag for the first few people who entered? Not my scene. I went back to the hotel, relaxed, and met up with colleagues for dinner. But the night wasn't over, the last session was:
8:30 PM (What was I thinking?) - Procedural and Conceptual Skills in Algebra
My entire grading rubric is based on a balance of procedural skills and conceptual understanding, so I thought this was an important session to go to. I was hoping for something a bit more about how to differentiate these two areas when grading, but the session was about the presenter's work on using a number of psychometric measurements (latent class analysis, distractor analysis, and qualitative analysis of cognitive interviews) to get a better understanding of the relationship between a student's procedural skills and conceptual understanding. I've done a bit of reading on psychometrics, but haven't heard about any of these measures, another thing to read up on.
The latent class analysis piece seems the most interesting, as it would seem to partition students into groups based on their answers. This is a much more advanced version of what I do with my assessment analysis; group students by total scores and look at question averages for low, mid, and high performing student groups. Based on the analysis I may remove questions from grading if no group had a 60% average for a question, and other similar ways to address perceived imbalances of scores. A latent class analysis could give me a better way to separate these students into these groups, but it sounds like I may have to code questions and answers differently... Not sure, hence the reading.
Done with Day 1, not sure how long this review will take me. My wife went to bed 10 minutes ago, and I think I need to follow her lead. If you have any questions, comments, suggestions, outrages, accusations, or ideas, you know what to do below.
My thoughts on teaching mathematics, using technology to teach, and finding ways to become better at both, with explorations into the education research literature. All thoughts my own, and not a reflection of any employer.
Thursday, November 21, 2019
Thursday, November 14, 2019
AMATYC 2019 Day 1 - Morning Review
Day 1 of my first AMATYC conference was AMAZING! So many good sessions, amazing people, and great ideas.
8:00 AM - Math Pathways: Exploring the Present and Envisioning the Future
I attended the first few sessions (Helen Burn from Highline College, Amber Rust from Anne Arundel CC, and Michael Sullivan from Joliet Junior College) on their different Math Pathways implementations. Corequisite Support courses are an integral part in many Math Pathways implementations, and I thought I could glean some ideas or thoughts from these sessions. While the presenters were knowledgeable and thoughtful about their programs I didn't get much about corequisite support courses. Michael Sullivan did provide a great perspective on statistics education (TI calculators and Excel bad, Minitab and SPSS good) and some resources as well. But really, any session moderated by Helen Burn is one worth attending, she's great.
9:10 AM - Teach Your Students How To Be Students: Scaffold Everything!
Amy Hatfield and Jessica Lickeri from Columbus State CC shared their very structured way of instilling foundational skills (their prefered word for success skills or study skills). They focused on multiple tasks that aligned to support specific success skills. For example, student's attendance grade is measured in three ways; completing a sign-in sheet during the first five minutes, staying during class, and coming prepared to class by completing certain tasks. I'm really interested in trying a few of their activities, and look forward to seeing how I can integrate their tasks into what I do.
Their main focus was on
A few small interesting things were shared;
My comment on their session evaluation form sums up my current thoughts on how to go about helping students develop their foundational skills;
"I struggle with the tension between requiring students to do tasks that are meant to develop student's foundational skills and activities that allow students to choose how to grow their foundational skills on their own. In my department I have conversations with a colleague about his Taoist approach and my Confucian approach. His stresses providing choice and establishing what the consequences of those choices will be, while I want to lay out the order and structure of what students should do. Really its a question of interval vs. external pressures. "
10:20 AM - Teaching and Learning Practices for Equitable Math Student Success
Ralf Youtz from Portland CC (formerly of Clark College) presented a great session on developing strategies that would help create culturally relevant teaching. Ralf used think pair share to a great effect, and in talking to him later on in the day he mentioned the book Routines for Reasoning: Fostering Mathematical Practices in All Students. I've always been a little worried about using think pair share on faculty, but Ralf's prompts were great and he attributed them partially to this book.
The group around my table centered on reflection as a tool to help center student's experiences. One person (Nancy) used math journals in the past, another (Greg) used math biographies at the start of the class to get a better sense of who the students were, and I shared my post-class quizzes that ask students to reflect on specific questions. A question I'd like to add that I got from this session was "What was difficult about the material today? Why?"
Ralf also focused on collaborative learning and proactive teaching practices. He mentioned specific topics and prompts to help students work in groups as students don't know how to talk about math. Really, its a skill, and one math faculty should be more attentive to. While some may feel these proactive teaching practices are 'hand holding', I have no problem holding my student's hands to help them get to where they are going; connect students to academic resources, if it is important make it mandatory (resonant with my previous session), and early alerts to let students know when they are getting behind.
It is midnight and I've only gotten through 11:10 am of the first day. I might (unlikely) get through the rest of Day 1 tomorrow, and will hopefully post more over the next few days. Before I got here I organized a binder with tabs for each day so I can keep my notes somewhat ordered. I'm going to try summing up my sessions here with the help of those notes, so help me Leibniz.
8:00 AM - Math Pathways: Exploring the Present and Envisioning the Future
I attended the first few sessions (Helen Burn from Highline College, Amber Rust from Anne Arundel CC, and Michael Sullivan from Joliet Junior College) on their different Math Pathways implementations. Corequisite Support courses are an integral part in many Math Pathways implementations, and I thought I could glean some ideas or thoughts from these sessions. While the presenters were knowledgeable and thoughtful about their programs I didn't get much about corequisite support courses. Michael Sullivan did provide a great perspective on statistics education (TI calculators and Excel bad, Minitab and SPSS good) and some resources as well. But really, any session moderated by Helen Burn is one worth attending, she's great.
9:10 AM - Teach Your Students How To Be Students: Scaffold Everything!
Amy Hatfield and Jessica Lickeri from Columbus State CC shared their very structured way of instilling foundational skills (their prefered word for success skills or study skills). They focused on multiple tasks that aligned to support specific success skills. For example, student's attendance grade is measured in three ways; completing a sign-in sheet during the first five minutes, staying during class, and coming prepared to class by completing certain tasks. I'm really interested in trying a few of their activities, and look forward to seeing how I can integrate their tasks into what I do.
Their main focus was on
A few small interesting things were shared;
- Menti is a pretty lightweight polling tool that looks really promising.
- Instead of Vygotsky's Zone of Proximal Development being focused on mathematical content, the presenters really stressed how we should think of foundational skills in the same way. I'm convinced with that argument, but I'm still on the fence as to whether requiring students to complete activities will help scaffold these skills.
- They mentioned the use of an adaptive learning environment and were a bit cagey about sharing the name but I think it was ALEKS, possibly Knewton.
- Their Spotlight on Student Success assignments seemed really good, and I'd like to see some examples of them.
- Two websites were shared PocketPoints and Flipd.
My comment on their session evaluation form sums up my current thoughts on how to go about helping students develop their foundational skills;
"I struggle with the tension between requiring students to do tasks that are meant to develop student's foundational skills and activities that allow students to choose how to grow their foundational skills on their own. In my department I have conversations with a colleague about his Taoist approach and my Confucian approach. His stresses providing choice and establishing what the consequences of those choices will be, while I want to lay out the order and structure of what students should do. Really its a question of interval vs. external pressures. "
10:20 AM - Teaching and Learning Practices for Equitable Math Student Success
Ralf Youtz from Portland CC (formerly of Clark College) presented a great session on developing strategies that would help create culturally relevant teaching. Ralf used think pair share to a great effect, and in talking to him later on in the day he mentioned the book Routines for Reasoning: Fostering Mathematical Practices in All Students. I've always been a little worried about using think pair share on faculty, but Ralf's prompts were great and he attributed them partially to this book.
The group around my table centered on reflection as a tool to help center student's experiences. One person (Nancy) used math journals in the past, another (Greg) used math biographies at the start of the class to get a better sense of who the students were, and I shared my post-class quizzes that ask students to reflect on specific questions. A question I'd like to add that I got from this session was "What was difficult about the material today? Why?"
Ralf also focused on collaborative learning and proactive teaching practices. He mentioned specific topics and prompts to help students work in groups as students don't know how to talk about math. Really, its a skill, and one math faculty should be more attentive to. While some may feel these proactive teaching practices are 'hand holding', I have no problem holding my student's hands to help them get to where they are going; connect students to academic resources, if it is important make it mandatory (resonant with my previous session), and early alerts to let students know when they are getting behind.
It is midnight and I've only gotten through 11:10 am of the first day. I might (unlikely) get through the rest of Day 1 tomorrow, and will hopefully post more over the next few days. Before I got here I organized a binder with tabs for each day so I can keep my notes somewhat ordered. I'm going to try summing up my sessions here with the help of those notes, so help me Leibniz.
Thursday, November 7, 2019
Faculty and Data: Threading the needle
Last spring my department did an assessment project to determine if our corequisite support courses (which I am the coordinator of) are effective in helping students learn the course material. My home institution has been running corequisite support courses (Business Mathematics with Support, College Algebra with Support, and College Trigonometry with Support) for more than a year and it seemed like a good time to assess whether these courses were effective. I wrote assessment questions aligned to our course outcomes and asked for faculty feedback, which turned into really interesting and engaging conversations. I also wrote rubrics with granular criteria (53 in total) and asked for faculty feedback, which turned into even more interesting conversations. Faculty in all of the support courses and most of the standard courses included these questions (a feat by itself), and after a day of about two dozen instructors grading the 220 student artifacts we had solid data showing that there was no significant difference in student understanding of the course outcomes.
As a result of this project I was asked to became the Outcomes Assessment Liaison for the Math Department at my home institution. I've also joined the Outcomes Assessment Committee, and it feels like I'm gearing up for another phase of my work. The conversations I had with faculty showed me that I have a lot more to learn about assessment, rubrics, and measuring understanding. Granted I've made decisions about what questions to write to assess outcomes and how to grade them. However seeing how faculty make these decisions for their own courses, and then the effect these decisions have on students makes me wonder whether faculty can (or should) have conversations about the decisions they are making. Why decide on a five point scale versus a ten point? How would different faculty grade the same student work? What are the best practices of math assessments senior faculty have and junior faculty crave?
With these questions the idea of collecting data on questions, rubrics, and course success rates is fairly natural, but like venomous animals, natural doesn't mean safe. Faculty are (rightfully) concerned that this data would be used as a hammer to shape everyone into the mold of the faculty member with the highest course success rates. Note these questions are separate from the expectations of most administors; How can we measure program-level outcomes? How can we show student progress to accreditors? How can we determine the areas in a program that need support? Yes, administrator's questions are absolutely related to faculty decisions, but their focus seems to be on the system as a whole. (I could be wrong in that last point, but being in a service department I don't have students wanting to go into my discipline, and therefore I don't have a lot of control over a specific program.)
Another point to consider is the college's Guided Pathways initiative. Like other community colleges we are undergoing reforms to clarify our pathways, get students onto pathways, keep students on pathways, and to determine whether these pathways are effective. The college created four Guided Pathway Pillars for each of these tasks, and I was (am?) on the pillar tasked with keeping students on the path. In that work I looked at both automated and manual methods for determining students who are 'at-risk' of not being successful in their courses. One dataset that was always asked for was access to the live student grades in our learning management system, Canvas. The idea is that with actual grade data our student services staff could conduct interventions to help students get back on track.
I am not opposed to the idea that student grades could be used to determine how outreach should happen, but that use of grade data needs to be restricted in a very specific way. Granted, administrators can get access to course success rates but live student grades during a term feels like a different beast altogether. If a course had a 50% pass rate half-way through the term and an administrator heard about that, is it out of the realm of possibility that the administrator would consciously or unconsciously apply pressure to that faculty member to increase their pass rate? Sure, lots of hypotheticals and unknowns there, but there very well could be other impacts I'm not thinking of.
This leads me to two thoughts on how we can create an environment of trust among faculty to use data;
1. A memorandum of understanding of how course data is to be used. There needs to be a line between faculty and student services use of the data, with administration accessing the data. Yes, administration needs some data, and that would be outlined in the memo. After the current union negotiations are completed I'm hoping to get facetime with our union president to propose such a document.
2. A faculty learning community where membership means you share your course data with the other faculty members. I really like this idea as it creates and builds community, while taking ownership for what we are doing. Starting from a place of looking at equity data we could address reforms that would have the biggest impact on all students.
Have you tried any of this? What are the successful data projects you have been a part of, or heard about?
As a result of this project I was asked to became the Outcomes Assessment Liaison for the Math Department at my home institution. I've also joined the Outcomes Assessment Committee, and it feels like I'm gearing up for another phase of my work. The conversations I had with faculty showed me that I have a lot more to learn about assessment, rubrics, and measuring understanding. Granted I've made decisions about what questions to write to assess outcomes and how to grade them. However seeing how faculty make these decisions for their own courses, and then the effect these decisions have on students makes me wonder whether faculty can (or should) have conversations about the decisions they are making. Why decide on a five point scale versus a ten point? How would different faculty grade the same student work? What are the best practices of math assessments senior faculty have and junior faculty crave?
With these questions the idea of collecting data on questions, rubrics, and course success rates is fairly natural, but like venomous animals, natural doesn't mean safe. Faculty are (rightfully) concerned that this data would be used as a hammer to shape everyone into the mold of the faculty member with the highest course success rates. Note these questions are separate from the expectations of most administors; How can we measure program-level outcomes? How can we show student progress to accreditors? How can we determine the areas in a program that need support? Yes, administrator's questions are absolutely related to faculty decisions, but their focus seems to be on the system as a whole. (I could be wrong in that last point, but being in a service department I don't have students wanting to go into my discipline, and therefore I don't have a lot of control over a specific program.)
Another point to consider is the college's Guided Pathways initiative. Like other community colleges we are undergoing reforms to clarify our pathways, get students onto pathways, keep students on pathways, and to determine whether these pathways are effective. The college created four Guided Pathway Pillars for each of these tasks, and I was (am?) on the pillar tasked with keeping students on the path. In that work I looked at both automated and manual methods for determining students who are 'at-risk' of not being successful in their courses. One dataset that was always asked for was access to the live student grades in our learning management system, Canvas. The idea is that with actual grade data our student services staff could conduct interventions to help students get back on track.
I am not opposed to the idea that student grades could be used to determine how outreach should happen, but that use of grade data needs to be restricted in a very specific way. Granted, administrators can get access to course success rates but live student grades during a term feels like a different beast altogether. If a course had a 50% pass rate half-way through the term and an administrator heard about that, is it out of the realm of possibility that the administrator would consciously or unconsciously apply pressure to that faculty member to increase their pass rate? Sure, lots of hypotheticals and unknowns there, but there very well could be other impacts I'm not thinking of.
This leads me to two thoughts on how we can create an environment of trust among faculty to use data;
1. A memorandum of understanding of how course data is to be used. There needs to be a line between faculty and student services use of the data, with administration accessing the data. Yes, administration needs some data, and that would be outlined in the memo. After the current union negotiations are completed I'm hoping to get facetime with our union president to propose such a document.
2. A faculty learning community where membership means you share your course data with the other faculty members. I really like this idea as it creates and builds community, while taking ownership for what we are doing. Starting from a place of looking at equity data we could address reforms that would have the biggest impact on all students.
Have you tried any of this? What are the successful data projects you have been a part of, or heard about?
It's been HOW long? SBCTC Coreq Workshop and the Quality Improvement Process
It's been a minute and I need to get a few ideas out on 'paper' so you may get a burst of a couple posts today and/or tomorrow.
Went to the SBCTC Corequisite Workshop on Tuesday, November 5th. While there we discussed the current corequisite support courses at various Washington state community colleges. My institution (Clark College) presented data on course outcome attainment rates of our coreq students compared to their standard course colleagues. Overall the data doesn't show a significant difference, but there is room to improve. I don't feel comfortable sharing the data here as we it doesn't really satisfy the conditions for a two-tailed difference of means hypothesis test, but I will say the means are fairly close.
While there one of the presenters Joan Zoellner of the Dana Center Mathematics Pathways (a former colleague of mine) shared the article Tools for Improving Corequisite Models: A guide for College Practitioners from the Rand Corporation. The article provides a really interesting model for identifying problems and how to investigate them. My own quick summary based on the article and Joan's presentation:
Went to the SBCTC Corequisite Workshop on Tuesday, November 5th. While there we discussed the current corequisite support courses at various Washington state community colleges. My institution (Clark College) presented data on course outcome attainment rates of our coreq students compared to their standard course colleagues. Overall the data doesn't show a significant difference, but there is room to improve. I don't feel comfortable sharing the data here as we it doesn't really satisfy the conditions for a two-tailed difference of means hypothesis test, but I will say the means are fairly close.
While there one of the presenters Joan Zoellner of the Dana Center Mathematics Pathways (a former colleague of mine) shared the article Tools for Improving Corequisite Models: A guide for College Practitioners from the Rand Corporation. The article provides a really interesting model for identifying problems and how to investigate them. My own quick summary based on the article and Joan's presentation:
Identifying the Problem of Practice: Here you are trying to identify an actual problem to address. Simple enough right? Well in practice my team found it fairly difficult to do. These Problems of Practice can range from granular instructional problems (What can I do to get students to graph rational functions and identify their properties?) to systemic problems (How can we get faculty to use data to improve their instruction?)
Aim: In a traditional experiment this would be the treatment of the experimental group. For most of us (although there seems to be information in the article on how to do this in the) we are not going to separate students into a control and experimental group, as that would be treating some students differently than others.
Measures: What data will be collected to indicate there was an improvement.
Changes: Possible solutions that would address the problem of practice, based on the measures.This is a deceptively simple way of cutting through all our plans, all our potential improvements, all of our great ideas, and getting to the things we really need to focus on; the problem, how we're going to address it, and how we're going to measure success. Too often in education we do a thing, thinking it'll solve our (perceived most of the time) problem, and then never checking in to see whether it worked. We can't keep going on this tailspin, we need some direction.
Subscribe to:
Posts (Atom)
A Simple Path Forward: Trading digital clutter for physical simplicity.
I talked to my 'rabbi' the other day, a senior faculty member in the department, and we talked over his observation of my class. He ...
-
In a previous post ( Pre-Pre-(Pre?)-Planning: Fall 2021 - Standards based grading, flipped classroom, corequisite support, and maybe group ...
-
Along with everyone else freaking out about Chat GPT ( John Dickerson did a nice segment on CBS ) I thought I might test it out with a few b...
-
I am hosting a professional learning community for my state-level organization (SBCTC) and am sharing some thoughts on how I am planning my...


