S1E6. New Approaches to Assessment During the Pandemic

Featuring Dr. Erika Merschrod - Associate Professor, Chemistry, Memorial University

The Learning Technology Coach Podcast is a CITL production.

Chapters

• 1:10 Episode Introduction
• 2:43 Guest Introduction
• 4:36 How the pandemic impacted teaching style
• 6:01 Final assessments options
• 9:46 Are new approaches to teaching sustainable after the pandemic
• 10:29 Using the ungrading approach to assessment
• 12:30 Dealing with pushback when introducing new approaches to teaching
• 15:32 Working with CITL
• 18:59 It takes time to teach well
• 20:22 Student engagement during the pandemic
• 25:49 Summary - Big Learning Moment!

Transcript

Javad (00:08):

Hello everyone. My name is Javad,

Timilehin (00:10):

And my name is Timilehin.

Javad (00:11):

Welcome to the Learning Technology Coach podcast, where we talk to educators about adapting to the new normal.

Timilehin (00:21):

Here we talk to instructors about how Covid-19 affected their teaching style, the challenges they had to overcome. The technologies they used.

Javad (00:28):

Plus a whole lot more. Hi, Timilehin, how are you doing today?

Timilehin (00:33):

Well, I'm doing great, thank you. How about you?

Javad (00:36):

Just a little bit of allergy. Other than that, it's all fine. I know you're a grad student and also a teacher and teaching assistant. How do you like grading?

Timilehin (00:46):

Oh, well, that's one of the major things I do as a TA. I can't run away from it <laugh>, In fact, it's the most challenging thing <laugh>

Javad (00:52):

I Know. Oh, so do you mean that you don't like grading?

Timilehin (00:57):

Not really. It's just like a stressful exercise, especially if you have a very large class.

Javad (01:02):

I feel a little bit better because I, at first I thought that I'm the only one who doesn't like to grade stuff, but now we are in the same boat. <Laugh>,

Timilehin (01:08):

It rains everywhere.

Javad (01:10):

<Laugh>. Yeah. Did your approach to assessment change during the pandemic?

Timilehin (01:15):

I would say I never had an approach pre-covid because I started teaching and grading in 2020, so I may not have answer to that, you know?

Javad (01:24):

I see. Well, let's refer to the interview we had with Dr. Erika Merschrod, who shared her experiences regarding assessment during the pandemic with us.

Timilehin (01:32):

Yeah. So whether you are a student teaching assistant or instructor, please stay tuned to learn how this dynamic professor adapted to the new normal with her unconventional style of assessment.

Javad (01:44):

Yes. This is a topic you don't want to miss.

Timilehin (01:46):

Yeah.

Javad (02:00):

Welcome back to the studio. The topic of today is new approaches to assessment during the pandemic. Assessment is an important part of teaching and learning. At least it's one of the ways by which learner performance is measured. The pandemic changed a lot of things, including the way we teach, how we learn, and basically how we live. It compelled educator to devise more flexible, yet realistic ways of assessing new learning, new student learning. Today we are excited to be interviewing a prominent educator, Dr. Erika Merschrod, who will be sharing her experiences with us in this regard. Timilehin, would you please introduce our lovely guest to our listeners?

Timilehin (02:39):

For sure. I'd like to do that. <Laugh>. Yeah. Erika has a BA in Chemistry from Bryn Mawr College and a Ph.D. In Theoretical Chemistry from Cornell University. She is a professor at the Department of Chemistry, Memorial University of Newfoundland, where she runs the Merschrod Research Group, which focuses on cross-disciplinary research in hierarchical materials and emphasizes multiscale materials design for novel properties. Researchers in her research group actually design, prepare, characterize, and model new composite films and coatings with applications in sensors, antifouling coatings, and tissue scaffolds. Such a great profile. <Laugh>, welcome to the studio, Erica, how are you doing today?

Erika (03:28):

Oh, fine, thanks. Thank you so much for inviting me.

Timilehin (03:31):

Yeah, it's a pleasure. I mean, it's, like a big honor, you know, to have someone of this kind of profile in our building today. And just to start with, I would like to put a little bit of context into what we'll be discussing today. Would you like to tell our audience the courses you teach? Maybe they are at the undergraduate level or the graduate level? I mean, that will provide a lot of context.

Erika (03:56):

Right? Yes. At Memorial I've taught courses and I regularly teach courses from large first year courses in, you know, in intro chemistry to more specialized graduate level courses. So the whole gamut.

Timilehin (04:10):

Great. And these were courses you've taught pre pandemic or, you know, did your number of, did the number of courses you teach during the pandemic change or something like that?

Erika (04:21):

I, know, it was about the same. So during the pandemic I've had some intro courses and some smaller seminar style courses.

Timilehin (04:30):

Right.

Javad (04:31):

So how did pandemic change your teaching style?

Erika (04:35):

Oh, it changed everything. <Laugh>, I mean, the, the pandemic was has been, continues to be really tough. You know, I'm it's, it has brought a lot of gifts for me in terms of, of opening up so many doors and making things much more flexible for teaching and for assessment. But I do wanna say, okay, it was, it was rough. It was really rough. It had an impact on my health, and I know for a lot of other people as well, just dealing with the stress and the isolation. That said, yeah, it was really a gift for, for teaching, for assessment in my mind. Suddenly all the rules went out the window and we were just told, okay, get the students do that first semester. So that would've been fall of 2020. And it was like, okay, just, just get through this semester somehow.

So I was teaching one section of a multi-section intro course, you know I don't know, I had 150 students I don't remember now. And usually we coordinate our final exams. We plan midterms, we all follow the same grading scheme, and we each did our own thing that fall. And so I ended up with a very low stakes final exam, which we hadn't been allowed to do previously. Right. now we can even not have a final exam, and there isn't a problem with the registrar. So it just opened a lot of possibilities.

Javad (05:53):

Well, when we're talking about not having final exam before covid, like, did your courses have final exams?

Erika (06:01):

Well, I always offered a few options. So for the seminar style courses, it was typically a take home final which they could complete, you know, at any point during the, the final exam period. And so it's a little bit more flexible. You can ask more interesting questions, so it's is more fun to read the answers. And I've also, for a number of years now, offered an option of an oral final. I also offer this for the evaluations during the term where we just sit down and talk about different topics in the course. And you really get a very in-depth assessment of what the student knows. It allows 'em to show that they can make those connections across topics. And what I like the most is you can follow up when a student says something. I mean, sometimes you'll get an answer written on a test, and it's just so not what you are asking. Right. It's just completely unrelated. And it's really hard to figure out what was the student thinking, you know, do they actually know what I'm testing here? And they just couldn't express it or didn't understand the question. So yeah. What I love about the oral test is that you can do that follow up right there and you really get a sense for how the student is thinking and how they understand the material.

Javad (07:10):

One of the major issues that many profs were dealing with after pandemic was how to invigilate the exams. I guess in your case, because like most of your exams were oral exams, like I guess you didn't have that problem, did you?

Erika (07:25):

That's right. And another thing was that all the evaluations for that first intro course, and I've kept that ever since, are open book and untimed, and that includes the final exam. So that completely did away with any invigilation issues.

Timilehin (07:41):

Right, right. Hmm. So you said 150 students, that's such a huge class. Would you say that was one of the major factor that, you know, led you to devising new means of assessing students?

Erika (07:57):

No, it was more this problem with the students not being physically there, being in different time zones. And of course the invigilation issue and students really facing a lot of hardships so that during a particular term, there's always gonna be some students who unfortunately encounter something during the semester that doesn't allow them to continue the course, or they fall behind by a couple weeks. But during the pandemic, it seemed like that could be happening to everyone <laugh>. So I really had to come up with an evaluation scheme that was gonna be flexible enough so that I wasn't constantly having to deal with late submissions. So it really was the pandemic and this issue with invigilation and students dealing with different schedules that really drove that change.

Timilehin (08:37):

Right. So while trying these different types of assessments or, you know, styles of assessments, did you run into any challenges? I mean, these are novel styles and there are, there is possibility of, you know, running into challenges or having key lessons from trying new things,

Erika (08:56):

Right. So some of these methods I've been using before anyway, sort of, sometimes you had to couch them in different language to follow the exact rules in the, in the course catalog. But it was really the, the big change for me was that idea of an open book test and how do you make an open book test? And I thought that was gonna be a huge challenge. It wasn't so much actually it wasn't too hard to make a an open book test that was not too hard to grade, that was feasible with such a large class.

Timilehin (09:31):

Right. Right now, things are getting back to normal, gradually, you know; we are getting back to what it used to be in 2019. So do you see these new approaches, you know, sustainable beyond the pandemic?

Erika (09:46):

Yes, I've kept them actually. And it worked out well because this past semester I was doing a remote class again. So even though the work classes on campus and the students were physically here we were still remote for, for my part of the course. And yeah, I was able to keep everything and I don't see any reason to, to leave it later. It's really been a gift for me in terms of the teaching, in terms of the assessment, in terms of how I interact with students. It's so much better.

Javad (10:11):

I don't know if you're familiar with like ungrading schemes, are you?

Erika (10:15):

Yes, yes. I'm a big fan of that.

Javad (10:18):

Oh, okay. So like, have you used this approach, like in comparison with like other method, like the traditional ones? Which one do you prefer? Do you prefer like one over the other, or do you prefer like hybrid one?

Erika (10:29):

Well, I really do prefer this ungrading, and it's always been part of my philosophy in terms of what I think I'm doing as an educator and how I need to, you know, interact with society and impact society. But I hadn't really implemented my thoughts into my practice, and the pandemic really allowed me to do that and to really say, okay, if I don't believe in grades, why am I still giving them? Right? So my approach to ungrading has been twofold. One thing I've been doing for a while now, but really increased during the pandemic is this notion of contract grading, where you make it very clear if you know this, if you do this, you get this grade. If you do that, you get that grade. And so students can decide very clearly what effort they can put into each class because they don't necessarily have the time.

And I understand that for many of the students taking into account, that course isn't their top priority, and that's fine. So contract grading is sort of a step toward ungrading. I see. But then also doing open book exams and laying things out so that it is possible for students to get a hundred percent. And that's been I guess another big change for me over my teaching career. I used to think evaluations are really there to discriminate between the top students and the other students and who should really get a hundred. And, and I've come to realize that if you learn the material, if you meet the expectations, you should be able to get a hundred. So it makes grading a lot easier when you make those expectations clear. When you give the tools, the students the tools to succeed, they can do it.

Javad (11:55):

I know that you are one of the pioneers in zero text textbook cost (ZTC) programs. So combining ZTC with ungrading and having like open book course, like all together, like it's makes a course totally different from like other traditional courses that we have. And I guess like most students are mostly familiar with traditional courses. Do you get any feedbacks from your students that if they prefer the new method, the method that you are doing or the method that other profs usually do?

Erika (12:29):

Right. so I think it's really changed as I've aged. When I first arrived here, there was a lot of student pushback for anything that was seen as different. And any of the teaching innovations that I was bringing to Memorial were really sort of suppressed, <laugh>, actively discouraged. And I think that has a lot to do with gender and age. And so as I've aged I've gotten less pushback and also getting CITL involved has really helped to put that stamp of approval so that administrators aren't saying, you have to do this, you have to do that. And really allowing me to, to really bring in a new and I think better approach to, to the teaching. So from the student point of view, there used to be a lot more pushback. Again, I think that, that had to do a bit with age and where they see you as a young person and don't respect you or something.

And so that, and the, and the administration wouldn't necessarily back me up. So, so that was a real barrier for me before. But it's gotten a lot better. And honestly, from the last semester, the feedback I got from students was so exuberantly positive, you know, students emailing me unsolicited saying, thank you so much. Your flexibility made this possible. And, you know, it really the students really, really seemed to appreciate it. Yeah. There was one complaint that they just wanted to, they just wanted to have lectures, <laugh>, you know, why can't we do lectures like the other sections? But that was one person, out of all the, the ones who who contacted me

Javad (14:02):

There are always complaints.

Erika (14:03):

<Laugh>, right? Yeah. I mean, you can't, you can't give the perfect course for everyone, right? So yeah, it's it's often you know, I wish students would have a little bit more flexibility and maybe they will in future with the new senate process. This is probably not really what you want on the podcast, but anyway where senate has approved a new way to label courses as to their delivery method. And so is it going to be remote? Is it gonna be hybrid? Is it in person? Is there an online component? Is attendance required? All those sort of things. So it used to be just is attendance required. Right. And now there's other designations, so students could pick the section that works for them. So if what they want is lectures, they can pick a section that just does lectures.

Javad (14:44):

I think it's very student-oriented at the end of the day.

Erika (14:47):

Yeah. Yeah. And that's something that actually out of the first teaching and learning plan that was developed at Memorial, so this is, I don't know how many years ago, <laugh>, we had consultations across campus, and I ended up in a group called Student-Centered Learning. And having that label really helped me to kind of solidify what I was doing and, and why. And that was a really, really helpful process.

Javad (15:15):

You, you mentioned that at some point you asked for CITL helps, especially like maybe like at the beginning of the fall 2020, like when we were hit with covid. How did CITL could help you in the problems or challenges that you faced at the time?

Erika (15:32):

So part of it was technological, just that CITL offers this whole infrastructure that allows us to do things online very easily, right. By maintaining a learning management system, but also all the other little things that go with it. So at the technical level, that was huge. But I think also at the sort of moral support level, it really helped too, you know, it helped to know that there was this team of people who wanted us all to succeed and who had the knowledge to do so. And really the, the service from CITL is always fabulous, but I was really amazed in the pandemic where I'm sure the workload, I don't know, it tripled for everybody. Maybe it went up six times for CITL and, and still the, you know, people were there whenever you needed them to give you practical advice.

Timilehin (16:19):

Right. Talking about technology, was there any software or technological tool that you used in particular?

Erika (16:27):

No. I, I was able to bring in some, I guess, more video. I definitely put more video. As Javad mentioned, the zero cost textbook is part of how I teach my course. And so basically it's a textbook that's completely integrated with the with the course webpage, the learning management system. And so within that, I can embed videos. And so that was something I did. I brought in a lot more video.

Timilehin (16:56):

Okay. And then you mentioned that you taught, in fact, you still teach graduate level and undergraduate students. In your approaches, was there any disparity, you know, in the experiences that the students shared with you, maybe from their feedback or something, did you notice anything change with the level of studies, maybe graduate or undergraduate?

Erika (17:18):

I guess I noticed a bigger change in the undergraduate response. So it was really the students were, were really needing much more flexibility whereas the graduate students already had that in the summer course. So the, the change was largest for the undergraduates, and I noticed the largest sort of positive feedback from them. But the under the, the graduate course, it was already pretty flexible the way I normally teach it. So I didn't notice a huge difference.

Timilehin (17:46):

Right. And that could be because maybe undergraduate students take more courses than graduate students.

Javad (17:54):

Yes, that's so true. And I just wanted to mention that. Well as soon as pandemic started and like CITL saw the workload, that new workload that's coming from the profs they started to hire some graduate students as technology coaches just to help like you know, profs and like, even like students, like, because Covid was new for everyone, and Yeah. I, I hear you on that <laugh>. When we are gradually going back to normal way of life do you see this novel approaches as sustainable sustainable beyond remote learning?

Erika (18:29):

Oh, absolutely. Yeah. I'm really thrilled that we can keep this going. So I, I was able to do all open book tests this past semester, whereas pre-pandemic, I think that would've caused a lot of consternation, <laugh> among certain people. So,

Timilehin (18:46):

Right. So what we are saying right now is that even beyond the pandemic, then we can have all of these different approaches going on. Do you see any weakness to these approaches?

Erika (18:59):

Well, there is always the time burden. It takes time to teach <laugh>. It takes time to teach well. And the notion that, oh, we can just learn tips for how to teach a large class more efficiently. We can learn how to use our time more efficiently, but that doesn't mean we're teaching more efficiently. So I think something to recognize is that all of this does take time, and I think our workloads really need to reflect that better than they do. I'm fortunate in that I don't have a high teaching load, and so I, because I do teaching in the classroom, I shouldn't say that I have a teaching load where less of it is in the classroom and more of it is in the lab. So I'm working with undergraduate and graduate students in my research lab in that kind of a learning environment. So the, the kind of regular classroom teaching is lower. And so I was able to dedicate a fair amount of time to this because my research group was more mature and they could kind of work on their own for, for a bit. So I definitely had more time to put into this than a lot of my colleagues would.

Timilehin (20:03):

Hmm. So now we've sort of talked about assessment, we've talked about student performance and their feedback. How do, how would you describe students engagement during the pandemic as a personal experience of yours? How would you, how would you describe it?

Erika (20:21):

Well, that was really remarkable for me. So I've had two semesters now with first year students that I've never seen <laugh> in my life. And I remember more of them than I did from in-person classes because they were reaching out. And they were really sharing a lot of interesting ideas with me because of the flexibility of the course. For example, students could choose certain topics to do an extra assignment on, and if they were particularly interested in a topic, they could do something extra with that. And they came up with fabulous ideas. So in terms of the engagement, I, I felt it was much higher. I think another example of that is there was one student who had told me upfront right at the beginning, I just need to pass this course, what do I need to do? You know? And I said, you know, I'm not offended that this is not a priority for you. Here's what you need to do. And so he came to me at the end of the course and he said, you know, I ended up doing a lot more work for this course than I thought I would, because I was able to do these extra assignments, and I found them really interesting. So yeah, so that was quite remarkable.

Timilehin (21:24):

Hmm. That's awesome. That's awesome. So essentially the pandemic, in a way, helped to devise flexible ways by which profs could actually engage the students and have probably, you know, get more from the students in terms of the courses and activities. And do you have any notes or any sort of statement that you would like to give to your colleagues, you know, that are not really receptive of changes? You know, they're not, they really don't want to adapt, you know, to the new normal, and they were like, this pandemic has come, and needs to go. Do you have any note for them? You know, any, maybe just any sort of statement to give them?

Erika (22:09):

Oh, I mean, I, I don't think I would say anything really. I think we have room for a lot of different ways of teaching and for some students that might work, you know, their, their methods I would like to see more opening on the part of the university to move toward more ungrading, because many of us do grades because we have to, because we have to assign a number at the end of the, at, at the end of the term. And by moving away from grades we could really engage more people in, in you know, considering alternate forms of assessment if they didn't have to give a number.

Timilehin (22:47):

Right.

Javad (22:50):

Okay. Maybe for the last question, I'm just following by what Timiehlin said. When we come, when it comes to online assessment, like we have like different tools that we can use. One of those tools is like discussion forums for, for example, like on Brightspace. Do you in particular, do you use any type of tools, any type of like like if you want to recommend a, a really useful tool for other instructor who are interested, who are interested, like in this method, what would you recommend? Like my, my my, my question is like, let's say like when you have, when you go to the class, you have like face-to-face engagement with the students, you have office hours, and like, when we are doing online or remote studying or learning, like all those face-to-face engagements are removed. What are the alternative to these ones?

Erika (23:43):

Well, one thing that I, I used was a suite called Gather Town. And so that allows a virtual space for students to come in and do small group work or do large group work, and just the technology allows you to speak to the whole room or just speak to a table, that sort of thing. Within D2L, I use the discussion board of Fair Bit for students to share personal tidbits not personal in terms of their personal lives, but just individual ideas or topics they were most interested in. So when submitting these extra optional assignments, the way they would submit them is to post them to the discussion board. And so I think that did help to make more public their contributions, but also, you know, to let other students benefit from them. But also to give everyone a sense that there were real people around taking this course.

Javad (24:32):

And is it an open source app or Software?

Erika (24:35):

No, gatherer Town is freely available for up to 50 participants, so there are ways you can work around that. I've used it for larger classes before. It's really nice for online open book exams where students would still wanna ask questions, and so they can have that platform open, they can digitally put up their hand and you can go and talk to them and they can be like in a cone of silence, right? <Laugh> mm-hmm.. And so they, they, it seems to work better for them to, to get your attention and to be able to ask questions as though you were there physically.

Javad (25:07):

Wonderful. Well,

Timilehin (25:09):

Yeah, that's awesome. And we just want to thank you again for coming for your time, for sharing your experiences, and you know, it's really a privilege to have you here. Thanks once again,

Erika (25:19):

Thank you both for having me. This was really fun.

Javad (25:22):

It's our pleasure. Thank you.

Timilehin (25:39):

Welcome back to the studio. That was one kind of an interview. I mean, I enjoyed it. What about you, Javad?

Javad (25:45):

It was really nice. It was very engaging as well. Yeah.

Timilehin (25:48):

Right. So I mean, there are a lot of things that we've learned through this interview, and I'm sure our listeners also enjoyed it, but Javad, what would you say you've learned or what are the things that you know that are really, really meant a lot to you from the interview?

Javad (26:03):

I like it how her courses are very, very different. Like all the techniques and schemes that she's using for her courses, such as ungrading, especially ZTC, I remember like when we had Open Education Week, that's a global week in the world. Memorial University was one of the few universities in the world that took participate in this year. And Erika and I, we had a presentation about ztc zero text, zero textbook. Awesome. Yeah, so basically ZTC is a course, or it could be a program that enables all the students to have equal access to all materials without paying anything. And when you don't have to pay automatically, you want to, you know enroll for that course and you're increasing the engagement for that course. And she's doing really, really great in, in, in that regard. And also, like, remember like we had this the same topic but for different conference for CITL conference like two months ago, and we were talking about like different aspects of ZTC. It was really, really wonderful.

Timilehin (27:01):

Wow. Fantastic. I know, you know, I wasn't really, really surprised because when I was reading through her profile, I saw a lot of amazing things about her and Yeah. And what you've just said is a testament to that. For me, I think there are a couple of things. First is the feedback and second is what the CITL did. So the first thing when I'm talking about the feedback now, I meant how she actually demonstrated, you know a kind of gradiational difference in the way our students actually reached out to her. Yeah. During the pandemic. She actually recognized issues in her teaching, especially because of the change in everything and how students really felt uncomfortable about that. And now in the last fall, she said they had the best feedback ever. And, you know, recognizing the impact the CITL staff are having on students, you know, is very, very fulfilling for me, especially. And that's one of the major points for me. It was really nice having Dr. Erika Merschrod here in the building. That was a great interview. And we just want to thank you, our listeners for, you know, staying tuned throughout the interview. Thank you so much. Till we see you next time. My name is Timilehin

Javad (28:13):

And my name is Javad. See ya.