Estimated reading time for this article:
35
minutes.
Speakers:
Apurva: Hello everybody. Welcome to another Office Hours session. It is lovely to see so many of you from so many different parts of the world. For those of you I'm meeting for the first time, my name is Apurva Ashok. I am the director of open education and the assistant director at the Rebus Foundation. And I work on the Rebus Community project.
We really try to build human capacity in OER publishing and open education through professional development, the publication of free openly licensed resources and through sessions like this one, Office Hours, which we've been co-organizing with the Open Education Network for many, many years now.
Before I pass it over to Karen, I also just want to acknowledge that I'm joining you all today from the traditional territories of many nations. I'm joining you from the territories of the Mississaugas of the First Credit, the Anishnabeg, the Chippewa, the Haudenosaunee, and the Wendat peoples.
I'm very grateful to be here on this territory, I'm grateful for the privilege to be able to live and meet and learn here. And look forward to conversing with all of you and learning more about where you might be joining us from today. Karen, over to you.
Karen: Okay, thank you, Apurva. And welcome everyone, we are so glad you could join us for another session of Office Hours. I am the publishing director with the Open Education Network. My name is Karen Lauritsen. And we are a community of professionals working together in higher education to make it more open. And if this is your first time joining us for Office Hours, we are going to turn to you for questions and conversation after we briefly hear from our guests.
It's intended to be a casual conversation. As we all come together from different parts of the world, I am joining you today from San Luis Obispo, California, which is the traditional land of the Northern Chumash. And today we're going to discuss showing your work, tools for reporting impact. We are joined by Tara Lebar, who is Associate Director of Academic Affairs with the Kansas Board of Regents; Barb Thees, who is Community Manager with the Open Education Network; and Emilia Bell who is Coordinator of Evidence Based Practice with Library Services at the University of Southern Queensland.
We're so delighted to have the three of you with us today, as well as all of our participants. We invite all of you to contribute to the conversation. I am sure that many of you are also engaged in reporting the impact of your OER programs, learning to make the case and to tell stories about why it's so important to support OER development in open education. And so we invite you to chime in in the chat and then as I mentioned, we'll turn it over to you for conversation after we hear from our guests. So I think that's it, and to kick it off, I will hand things over to Tara.
Tara: Great, thank you very much, I'm excited to be here. What I've learned in my 18 months of being with the Kansas Board of Regents and working with the OER, the Kansas OER Steering Committee is that no matter how much you know about OER, you never know enough about OER, right? And you never really feel like you are an expert.
And I definitely don't feel like an expert, but I'm excited to be here to share at least some of the things that we've done with our Steering Committee as OEN members. And how we've used the dashboard to expand our faculty workshop programs, expand and use a state stipend grant with those. And then also I started to collect some of the adoption data in there.
So I'm excited to show you what that looks like as well. So I'm going to share my screen here. And let's see. A little bit of background on the Kansas Board of Regents. The State of Kansas has 32 public institutions. Seven public universities, seven technical colleges and then 19 community colleges. And we are the governing body for all of the public universities and then we coordinate with the community colleges and technical colleges across the state.
And so, it's a partnership, but it's also something that we kind of all have to walk together on because we all have different governances, until the state tells us what we have to do, and then we all have the same boss. So we do have a lot of collaboration, and one of the great things about the OER Steering Committee is we do have representation from all different types of institutions. And most of our institutions are represented on that Steering Committee.
And so, the work they've done together to advance OER initiatives across the state has been phenomenal in the last two I think it really started, they started slightly before I joined the team, maybe a year right before Covid. So in the three years that this group has been going, just the programming about OER has been really exciting statewide. So you can see here, so this is our OEN dashboard.
We joined a year ago as a consortial member. A couple of our institutions are institutional members from OEN. And we had some trainers that had worked specifically a trainer from University of Kansas that was very connected with OEN. And then, another trainer at Fort Hays State University was connected with OEN. And so we had some folks that were really connected and spoke highly of the organization.
And so we were able to do this, and we saw this membership as a way to share their resources and specifically the programming with all of the institutions in our system across the state. And so last year, the Spring of '21 we implemented five faculty workshops, and you can see that, using the OEN faculty workshop model. And then we had faculty from all across the system. These filled up super fast.
We kept them moderately small so that we could encourage discussion. They filled up very quickly. And then, we invited just like the workshop model does, we invited the participants to go into the OTL and enter an online review if they chose to. And if they did that, then they were eligible for $150 stipend from the state. So we've been doing that. This Fall we continued that workshop series by offering another six workshops, again, multiple institutions.
We used our trainers, all these workshops were virtual. And so, here we were using the OEN dashboard as it was intended, as it was presented and as a way to register for those workshops and track participation. And what's nice about the dashboard is that Barb's team they have created all the registration, the emails, the communication to go to these participants and make it very easy for us to put this together.
And especially for me, at the Board of Regents office location, one central location, it was very easy for me to just put the details of the event in the dashboard. And then, the emails were sent directly, or we sent out the registration, people signed up, faculty signed up to the one they wanted to attend. It was really slick. What it also created though, is all of these participants come with an email and a contact, which has been nice.
And so, we did 11 workshops last year over the course of 2021. And we realized we still had quite a bit of grant money left over. And so, we transitioned this model into using it for our institutions, and we opened it up to any of our 32 institutions to hold a training for their faculty as a professional development event. We said we'll provide you the trainers, if you want to offer the stipend, we have enough money to offer a stipend.
And so, you can see since January we've had several of these institutions reach out and create their own. We have another one coming up next week, Johnson County Community College. And these vary from the professional development that everyone has to take, to an optional session created just for faculty. And so really just letting the institutions, they can be virtual, they can be in person.
And with this model, the participants don't register through the dashboard, the contact just sends me who's signed up. Now that really has been the extent of how we used the dashboard up until a couple of weeks ago. Because as I was looking the dashboard was recently redone, and so I had used activity things to send out invites and reminders and track our stipends and things like that.
But because I looked over on this reporting status, they added a couple of things. Well, all of these graphs were empty. And really, the only thing I would scroll down, and the only graph I would see is this. And I'm kind of looking at this going, "Boy, it would be really, really cool to be able to see student savings, student impact, faculty using OER." So one of the surveys that was added recently from OEN to the dashboard is called an adoption survey.
And we hadn't played with that yet. And so what I did was I clicked on the adoption enrolment update, and I sent that out to all 188 faculty that we had interacted with during 2021. Now, again, these are small numbers, and this is just a start. But this is the information that I got back from those 188 participants. And so just that small slice and really, it's probably only about 50 or 60 participants that had really answered and responded.
All of a sudden, this data started coming to life, too in ways that now I'm thinking ooh, how can we do this? How can we replicate this on a larger scale? How could we get this out to all of our institutions? Because this was really exciting to see. Barb, did you want to chime in here?
Barb: I think I'll let Emilia go, this is giving me great ideas and I'll circle back on it. Thank you, Tara.
Tara: Cool, no worries, well so this is the start of what got me excited about looking at different ways that we could use that. And maybe even twisting OEN's arm a bit to see if there's a way that we could share that survey with more than just workshop participants. If there's a way that we could send that out to our whole entire system or something in that aspect. But I love the simplicity of this survey. I think it's five or six questions.
It's very easy and it talks very clearly about what class you're teaching, how many students. Talking about sections and numbers of students and really getting that concrete information from the faculty member that's answering the question. And so, like I said, this is a cool tool that I am excited to play more with. And so, with that, I will maybe toss it to Emilia. And be happy to answer questions as we go along today. And I'll stop sharing, there we go.
Emilia: Wonderful, thank you and hi everyone. So I'm going to talk a little bit on how we've highlighted the reach of USQ's open texts which are on the Pressbooks platform, using Power BI as a data visualization and dashboard tool. And this is work that's emerged from conversations between two library teams. So that's the open educational practice team and the evidence based practice team, which is where my role is.
So we had several initial discussions and ended up taking a phased approach to impact. And our first stage has involved showing the attention or the reach with web analytics data. So we've created a Power BI report that has a dedicated page for each of our open texts and it can be accessed by individual authors. It's probably worth mentioning as well that web analytics data is very easy to access, can be usefully measured, can appear very promising.
So there's a lot of appeal to it. It's been a lot more challenging, however, to actually recognize what's actually meaningful and accurate from it. So there are limitations that have required some consideration and transparency. But really, the dashboard has highlighted the attention that open texts receive and a new way just to promote that OER output as well. So we've been able to not only visualize the reach or attention, but to also provide just a single space for authors to explore and interact with data themselves.
There's a few groups accessing the Power BI report, so that includes academic authors and library staff as well. And what we're aiming for there is to actually make this as accessible as possible, so that authors can really easily navigate and explore the data independently. So I'll share an example of this work. Awesome. So as I mentioned, we have dedicated a separate page for each open text.
And this means that authors can simply select their text and find the data specific to their own text or chapter. So we'll look at academic success. And initially we can see the page views and downloads. And academic success has different authors for each chapter. So we've drilled down to this level of detail, so that the numbers on page view is actually relevant to individual authors as well.
Some of our other measures included sources of traffic, browsers and devices, and geographic reach. And after a few iterations and some feedback we chose to include these measures on the same page open texts. Just to use tabs, which are Power BI bookmarks to switch between different views. So we can switch to sources of traffic and still on the same page we've just built upward on the canvas and hidden the other visualizations.
So we've got sources of traffic, which is showing the referral path, so the website or search engine that they've used to find through. And we've included any social media sites, USQ's learning management system, which is called Study Desk as well as access by any other university or school links as well. So from many other websites. There browsers and devices was a little bit less specific for reach or impact, but it's provided opportunities for greater evidence based decision making when creating open texts.
And getting an understanding of what forms of technology are actually being used to access our open education resources. And then finally, under the last tab, we have access by geographic location, so by country. And that's really interesting just to explore between different open texts and subjects and to be highlighting reach in that respect as well. And we needed to accommodate several needs for evidence.
Some around decision making and others to actually highlight that reach and attention but we still had to be providing some of those visualizations that were considered and responding to the questions that we were asking. And really there's only so much canvas space that could be used effectively, while also facilitating authors and library staff to actually frame their own stories around reach and impact.
And within the library we can repurpose and refine any of these visualizations in other dashboards or communications, just as part of the larger narratives around openness. When we were designing the dashboard though we were also considering what measures can actually signify impact. So I think some of the language that we're using reflects our ongoing discussions around that.
So at this stage, we've addressed the reach of open text or the attention that they receive as being at best proxies for actually impact on user engagement. So we've distinguished between attention and impact. And that's where our phased approach comes in in that we're intending to then collect and incorporate the data on adoption and reuse of open texts as well as later qualitative evidence.
Just to provide opportunities to actually highlight other voices which is going to mean that we can extend beyond just quantifying reach and intention that our open texts receive to further emphasizing the value and impact of their openness as well. So thank you. And I might pass on to Barb now.
Barb: Thank you both. And I'm going to circle back kind of on similar things to what Tara was sharing with her screen. So I was a part of the team that like Tara mentioned revamped the dashboard. And I do want to give a shout out, there's a lot of OEN members in this crowd who use the dashboard on a regular basis. We have some data rockstars in here, too that we tapped in to create this version of the dashboard as well.
So I invite any of you to share or chime in if anything is really striking a chord with you. But just to expand on some of the things that Tara shared, don't worry, this is not me sharing anybody's data. But this is our test account. And as she mentioned, so the general purpose of the dashboard itself is to track the impact of open educational initiatives of our members. And so, it does have this capability, where like Tara was saying administrators themselves can add details on their programs.
Whether that be a grant program, an event you're hosting, a workshop you're hosting, and who participated in that. And then, also has this capability, as she showed, to actually create email campaigns that are then sent out directly to instructors where they can add that information themselves. So it has that flexibility for administrators to manage, but then hopefully keep the information as up to date as possible with including the instructors in the data process directly having them added to the dashboard.
Another one of the things that I think is unique about our data dashboard and one of the reasons that we did want to do a revamp is because we have a number of consortial members, as well as individual institutions who are using the dashboard, who work with one another. And there is that overlap between faculty that attend professional development OER related events, through either the consortium or through their local institution.
So something I want to show you which Tara is at the consortial level and that is the view we're seeing here is when you login to your dashboard you see the programs that you have created. But then, down below if I'm a consortium, for example, I can see the programs that my member institutions have also created in terms of their open education initiatives more locally.
So with that, I can see what they're up to. I can see which of their faculty are participating and within an individual faculty profile, I don't have an example in this screen. If someone has participated in more than one program, whether it be at the local level or the consortial level, they're marked on here and you can see that within their profile and vice versa. So if you're an institution that's part of the consortium, you can see the programs you've created up top.
And then, any programs that your local faculty have participated in at the consortial level, you can see the basic details down on the bottom of your view here. So we're hoping that that really helps in terms of transparency and facilitating communication between these different levels that are working with some of the same faculty members and instructors within the open ed space.
And then, I am going to come back to the individual profile. So in addition to you start at the program level and you can zoom in to an individual instructor. You can see the basic information for the person which in this case Mia Hamm, who knew she was involved in open educational work? You can collect basic details about a faculty member, make any notes about their engagement in your programs.
View which programs they've participated in, and then you can also see which activity requests you've sent to them. So what have you invited them to do? You can see details about the campaigns that they've been a recipient of. And then, zooming in even closer, you can see how they've engaged as a result of you reaching out to them through the data dashboard.
And that would either be by for example, if you had sent them an invite to review an open textbook in the Open Textbook Library, you can see how they've engaged in that way. Or if they have any adoptions associated with their account, you can view that information, as well as any enrolments associated with that individual right in here. So this information, like we've been saying gets populated either you can do it manually as the administrator.
Or this is something that perhaps Mia Hamm would enter directly based on a communication that she received, or what we call an activity request that she received through the dashboard. And then just finally to come back to the reporting, the data slices these visuals are downloadable. So the hope is that we polled the community, we're asking what is most helpful in terms of using your data for your advocacy efforts and your reporting efforts.
And from that, we've put together these different visuals that Tara was showing, that you can easily download and plop into a report or a slide share presentation somewhere. And then also, for our superusers, which again, we have lots of them on here. There is a way to download in spreadsheet form your data to extract it and be able to manipulate in whichever way serves your program or your reporting needs best.
So I will leave it at that, again, knowing that there's a lot of OEN members in the crowd. I did just want to drop a link. We have a documentation site that outlines all of the capabilities of the dashboard with some screenshots. So if anybody wants to dig deeper into that, that is now in the chat.
Karen: Thank you Barb, Emilia, and Tara, it's great to hear from you about how you've been leveraging different tools, both at the programmatic level and at the resource or book level. And thinking about who has access to that data. Emilia, you talked about really wanting authors to have direct access to the data related to their resource, which is interesting. And so now is the time when we transition to a conversation and talk about the tools that we've seen today as well as more generally about reporting impact.
So we'd love to hear more from all of you about what you're working on, what the challenges in reporting impact for your OER programs, what you've found to be effective, whether it's hard numbers or stories or both. And so with that in mind, I thought maybe I'd start with you, Tara, if you could tell us a little bit more about what you do at the state level to sort of share back the impact of your OER programs and what you've found legislators and other really glom onto when you're doing that reporting.
Tara: So data reporting is a newer function of our state organization so far. And I am the state liaison from our office. But unlike other states, it's a little part of my job, and so our big dream would be to have an OER staff person in academic affairs. That honestly, I'm working really hard to work myself out of that job, only because we have an amazing steering committee of professionals and they are wonderful to work with.
But wow, the things that we could do with someone who has OER background working full time in advocacy out of academic affairs would be fantastic. So like I said, my end goal is to get that position built into Kansas Board of Regents and I think that's definitely a doable request. But so what we did last year though, in an attempt to create some baseline data for our state was we looked at several different states.
And we created an annual survey, so we put that out towards the end of the academic year, last Spring. So we're getting ready to put another one out. And then we created a report that we could give to our Kansas Board of Regents just on those open education initiatives and resources. So like I said, last year was baseline data. This is going to be our first year of measuring.
But this year I can tell you, yes we had 11 faculty workshops last year. We also did our very first OER summit statewide this past February that was virtual. And that was our first foray in a year out of the OEN faculty workshop model, too. And so looking at different ways to reach faculty, and I remember we had about 350 attendees for our virtual summit, we were really thrilled about that.
And I remember just hearing over and over again, "Oh my gosh, I didn't know there was so much OER happening across the state." And that was from our OER Steering Committee members, these are the people in the trenches, and they're going, "Holy cow, I didn't know all these other people existed, too." So I've been really excited to see the expansion, the excitement, the encouragement.
And I think we've only just begun to tap some of that. But measuring it is a beginning thing, in fact I talked to my boss earlier today and he was asking because we are looking at our performance agreements. And so we're looking at is there a way that we could put OER as one of the performance measures, specifically for our community colleges. And not because the universities, but because it's just one of those things that A it provides a measure of validity to the effort.
And it gives those schools that might need an excuse to try something new that excuse. So oh, well if it's something that we can put down on our performance measures for the next upcoming year, what the heck, we've been talking about that, or we've got some folks that are already doing some of those things. And so that's one thing that we're looking at.
And I think as we look forward, we're going to be putting together a showcase where we're going to be able to show our Board of Regents and our legislators all the different things are happening with regard to OER at our individual institutions. And combining that with putting together our legislative ask and so our Steering Committee is working with our legislative liaison to make that a budgeting priority for our board in the next session.
So those are the things we're looking forward at right now. And I'm hopeful the data will help us in that as we continue to grow with that. But also, because of these graphs that I've been playing with recently, I'm tempted to see if some of those questions maybe can get in or if there's a way that we capture some of that. Because like I said, it's pretty powerful. It's hard to capture student savings and quantify that. But man, when we can, I think that starts really speaking to the power players, right, to the decision makers and the money holders.
Apurva: Thanks, Tara. And your excitement around what you can do with this data reminds me of what Emilia was saying about thinking just beyond the reach and just beyond those numbers. Emilia, I'm curious if you could take us with you to some of those discussions that your team has been having around other ways to demonstrate value and impact.
You showed us a lot of really excellent book level metrics, but I'm curious about what the ideas are in your discussions? What is brewing and where do you see yourselves going as the next step?
Emilia: Yeah, so for starters looking at extending beyond reach and attention, we were looking at the language around how we communicate the impact and considering what we were actually counting as impact. So we considered whether what we were collecting could actually show impact and reflecting on what was being counted or emphasized when we were tracking page views or downloads.
So we chose to make that distinction between impact and attention. So page views might show the attention an open text received or interest in it, but not how it was actually engaged with. And there are some presentation slides, I think it's by Luc Boruta on the impact of open that really explores this. And I'll share it in the chat in a little bit.
But it's on open access metrics, I love the ideas around how we measure impact and what's worth counting are really relevant to explore with OER, too. So across the different stages of our approach to impact, some metrics or evidence won't necessarily show that on their own. But they will show that reach or attention that an open text receives. And I think that's where we need to be actually critically reflecting on what's meaningful for us to actually be collecting.
And how we can demonstrate the meaning and value a user gains from open text. So not everything is going to be worthwhile counting or is going to highlight impact or even attention in a meaningful or relevant way. And that's also going to vary a lot between different local contexts and different stakeholders as well depending on who this is being communicated to.
Moving forward though, adoption is going to be the next part of our stage. So looking at reuse, attribution, so part of that would be internal and include the use of textbooks in courses. And so, that internal reporting would really just help to collate USQ activity around OER. And separate to that, would also be adoption of USQ texts by other institutions as well. And we can explore different approaches to how actually communicating that in evidence how we might incorporate it with what we're already collecting.
And also, just keeping to help that reflection on our current stage and revisiting the questions that we're asking around the data there. Whether they need to change, whether we need to change what we're collecting, how we're communicating it, who we're providing access to. Whether we need to be building other dashboards, or other means of communication around it as well. So yeah, that's really where we're heading with that.
Apurva: Thank you so much. And Michelle was asking I think for any of our speakers if any of you have tied this back to retention or completion? And I'll extend that question to everybody here on the call. If you had a chance to use the data you've collected around your OER initiatives to talk about retention or completion of courses that you've seen your students perform better with OER.
Emilia, it sounded like this is something that you're hoping to do with the next iteration. Barb, is this something that you can get from that adoption form information in the dashboard? Or is that perhaps a future feature?
Barb: Yes, I think that would be a future feature. Kind of like Tara said, a lot of people are just hopping into the adoptions part, so it's going to be exciting what we can do with that.
Tara: No, I was just going to say I think it's more of a function of course marking. And once you can get your course marking in your institution, then you can trace that GPA and maybe do some comparisons there or at least follow your students that way. We have a couple of our institutions that have fairly elaborate course marking systems and are able to do some tracking.
But I don't know that I've seen anything that has tracked them all the way to retention or completion that way. But I do know that course marking is pretty helpful in that. I mean, course marking is helpful for students knowing how to sign up for classes, but I also think there's the other side of the course marking piece, too where you can follow that and track that data on the back side.
Emilia: Yeah, I was just going to say very early stage for us at USQ as well in this area and probably Australia I think as well. So it's just something that we're going to continue to explore and yeah, very much a staged process.
Apurva: Thank you, I'm actually wondering if, Tara, you talked a lot about reaching out to legislature administrators and stakeholders. But this retention or completion question has me wondering whether the authors who might have access to the data at USQ, what has their response been to seeing the impact? And has that driven any desire for change at the classroom level in terms of pedagogy?
Because that's also another way I think that we can talk about the impact of OER. Right? There's that student piece, but also the faculty or teacher side.
Emilia: Yeah, absolutely. I'm not sure about actually in the courses themselves. But we've received plenty of positive feedback from several stakeholders, including authors themselves. So we took the time to actually walk through groups of authors from each open text through the dashboard and have got plans as well to record some more walk throughs around that just to make sure that everyone's familiar with how they can use it, any of the limitations associated with it as well.
And some authors have actually felt really encouraged or driven by it to want to create more open texts, so that was some really positive feedback that we received as well. And it's also prompted several changes in the design and granularity of the dashboard. So from the outset each text had its own page, but we've realized that a lot of the data around that wasn't just page views or downloads was being missed.
Because everyone would just go to their own page and forget that the rest of the dashboard existed, which is why we changed the navigation around to include that data on the same page, there. And also, filtering down to day that was another request we had, and some other changes that we made. But on the whole it's been yeah, really positive feedback that we've received from our academic authors. Yeah.
Apurva: Thank you and I see, Adrian, you seem to have sent me a message in the chat. But perhaps you would like to share it with the larger group around other ways in which USQ or others have used this data? Yeah, Adrian mentions that they've also used the data in academic promotion rounds, for learning and teaching fellowship applications and thinking about their learning and teaching practices. And Tara, you were talking about advocating for that OER staff person, so it seems like there is a way in which to make the case for it and to help with that, either career advancement or carving out of new roles as well.
Tara: And one of the things that one of our institutions did that sounds kind of counterintuitive at first, but I think it's proven really popular is with their course marking system, they indicated all of the low or no cost courses. And added a very small stipend, $10 stipend, that the student would pay in lieu of obviously a larger textbook fee. But with that, the money paid for those students in those courses a portion of that money goes back to the department for OER initiatives.
And so, there was incentive to create more for faculty to revamp their texts and their curriculums to using open education resources when appropriate. Nobody was pressured. But they had access to resources if they wanted to use time or to go learn or whatnot, they had a pocket of money that they could use for their department in the effort of OER to add more OER courses to their department.
The other part of that stipend fund went back to the university to do again, OER grant programming school wide. And so, while it was a very small cost to each student in those OER courses, it helped fund more OER initiatives and create that motivation for faculty to take that step and maybe stretch themselves or add another OER class. Because again, the more classes you had, the more money you generated for your department. So there was a little bit of a win-win there, and I think that that also helped encourage faculty.
Karen: Thank you and thank you also to Emilia who put a link in the chat for a presentation impact cannot be measured and other sad half truths about impact measurement. Not having seen this presentation, I'm just going to take a leap and say that I identify with the title. Because I do think that that is sort of a larger, if you will, philosophical struggle with what we're talking about today in terms of wanting to quantify everything.
Wanting to put a number on something, just looking for ways that we can quickly tell people why these programs matter. And sometimes that can feel reductionist or frustrating, and yet it's also understandable that we probably need more than stories. So that's the position that we find ourselves in. And so, very happy to talk about that too. Amy, I'm not sure what you were "ha-ha'ing, me too." Who presented the webinar?
Anyway, I invite anyone to chime in. Maybe just in thinking about some of the things that our guests have been reflecting on. I wonder Barb, in your role because you hear from so many new OEN members who are being onboarded and perhaps just getting their programs started. Do you hear about what they're doing? How they imagine tracking impact or pressures they might be under to do that? Or what are those early conversations and inquiries like?
Barb: That's a good question. I think it seems overwhelming. There are some people that come at it like we've done the more anecdotal things because of capacity measures that we don't have the capacity to be crunching the numbers in the way that we would want to. And so, when our conversations around the dashboard kind of come up, they see that as a boost to what they feel is a very valuable piece of this advocacy and reporting, which is the student voice, which is the more storytelling behind it.
And that kind of feeds into what Tara is saying, too I think. There is always a concern around capacity. But I think in my work and in my conversations a lot of what I say to people just starting out is that in the culture of open, relying on those and connecting with those who are further along in their journeys and learning from maybe the bumps that they've faced along the way is probably the best way to go.
So again, some of those people that I name drop are in this call right now, and I invite them to join in. But I think what I've heard also from the more experienced people is how they've gotten to their processes of the way that they use their data, the challenges the come along with it. Because it is so involved, especially when you're talking about multiple adoptions and multiple people who are parts of multiple programs.
And how you quantify the return on investment when there's that overlap between programs and people in them and what your instructors are doing. So that is always my biggest takeaway is lean on and rely on those who have come before you and worked out some of those kinks. And we're lucky to have a lot of those people in this call and in our community.
Karen: Thanks, Barb. I'm just going to allow for an awkward pause, in case any of those people care to chime in. It was worth a shot. Okay, Kaitlin in the chat, thank you for your question. Kaitlin is curious to hear your biggest hopes for this work, whether that's funding and policy change, finding non-traditional ways to measure student success and impact, perhaps increasing capacity on your local teams. Those all sound like great incomes, are any of them close to your hearts?
Emilia: I'll jump in just saying incorporating some qualitative evidence into this body of work, whether that's separate from the dashboard itself or whether we incorporate aspects into that dashboard or create more work from it. Yeah, love to see some other voices included just beyond the quantitative reach.
Tara: I think I'm most excited to see what we can do from the statewide level and if we can take it from a volunteer steering committee effort to something that is more policy either legislated or just policy written. I think we've got support, but we don't have anything written down yet. So I'm excited to see if that materializes for the state, because I think it would be really exciting.
But honestly, one of the things I love about OER the most is there's no downside, right? It's all good. It feels good, and regardless of as long as you're taking steps forward, it's exciting. And I love seeing people get connected. I love hearing the student stories, but I also love hearing the teacher stories about how they feel like they're doing something good for their students and they're more free to teach. Like I said, there's just not a downside, in my perspective. And so I just love following where the work takes us.
Karen: Thanks, Tara and Emilia. I will pause for a second on student stories. I'm curious to hear from our group, when we say student stories, if most of those focus now on hey, this was a free resource that I could access from day one. And that's made a huge difference in my educational career. Or are student stories also more sort of open pedagogy based or reflecting some of the open practices that faculty are engaging with students in creating these resources?
Are there both of those kinds of stories out there? Do you think that they hit decision makers or funders in a similar way? Or that they serve different purposes? What thoughts might this group have on that?
Kaitlin Schilling: I think they're both. I think it can be both. And in terms of that qualitative, I know the gut reaction is to say if it's qualitative or a story it can't sell to policy or legislature. But I think it's just finding the right way to tell that story and share that impact. Maybe it's not a number but breaking down that data and the number you do have of maybe showing equity or how accessible things have become. There are always ways to weave in selling points throughout stories. Hopefully that made sense.
Apurva: I think what you're saying also, Kaitlin, is that big reminder of this is not work in a vacuum, but this is work for people, right? Whether it's the story of the students being impacted as you were gesturing towards, Karen. Or I know, Tara, you talked about doing a showcase to the Board of Regents and legislature. Could it also just be a faculty or librarian or educator showcase of the various champions who've attended all of these workshops?
Who've seen all of this tremendous growth, who've changed their practice, who are bringing all of these different ways of hopefully innovative more student centric, empathy focused, trauma informed ways of teaching in the classroom. I think there is definitely a lot of different types of stories to share. And I wonder if focusing on the people and encouraging some conversation around that is on your radar, any of the three of you?
And I don't know, Barb, if you've heard from other OEN members whether that's something they have done or are planning to do?
Tara: Well, I think for me, the hook or the connection with those policymakers and stakeholders is everybody knows about college textbooks. Right? And buying the textbooks, everybody's got a story from when they did it. And they have their own child story, blah, blah, blah. That's the place we connect. But I think to Kaitlin's point, that's not the only story that's being told.
And you hear the stories, and it's not just from students that are economically disadvantaged, either. You hear this from every income level, from every college student across the board. "Well, I decided I didn't want to buy all the books for this class." Right? But what I love about it though is sometimes when you go and you ask students on campus, "What do you think about OER classes?"
And they go, "What are you talking about?" "Oh you know." And you explain it. "Oh yeah." Sometimes those are their favorite classes. Sometimes those are the most engaging classes and the classes that they'll go, "That teacher was the best teacher I had. That was the best class I had." And you hear that just as much as you hear, "And I didn't have to pay for the book." Right?
And that's what I love about this is it goes hand in hand. So yeah, we start there, but when you unlock the teaching and you get connected, faculty can unlock their teaching and get creative with their teaching, connect with students in different ways. Students aren't stressed out and they can connect from day one. It's just more than saving money on textbooks. So yeah, and I agree.
To your point, our showcase is going to highlight all the OER initiatives happening on our institutions, not just that student voice. But all the initiatives that have been happening and we had a student keynote speaker for our summit, and he literally stole the show. You know what I mean? You can't go wrong with that student voice ever.
Apurva: That's definitely something we're always mindful of because in these conversations, which is faculty or librarian focused, we don't have a student necessarily in this room. So I'm always mindful of that. Amy, why don't you do next? And Emilia, I saw you unmute as well, so I'll pass it over to you after.
Amy Hofer: Yeah, I'm curious if anyone else who is in the MHEC presentation this morning that Katie Zaback gave. So she was presenting a new document that MHEC, which is the Midwestern Regional Compact created about trying to find standardized ways to measure student savings, impact, and cost benefit analysis. I loved that she took that approach, because there's cost to any implementation of course materials whether open or not.
Anyway, it was a really tough crowd in the chat, and a lot of back and forthing about which dollar amount is correct. And should it be 68? Should it be 160? And I didn't want to jump in on the dollar amount, because I feel like the way that this conversation has been emphasizing there are so many impacts that go beyond whether it was $68 or $78 right? And also, just the fact that I hear from students that a $10 book can be too expensive.
A $10 book can be barrier to success, truly. So trying to figure out the real dollar amount can feel sort of quibbly at that point. Anyway, I just wanted to say that that report is out, I'll dig around for that link. I think it's still open in my browser, but I'm hoping that there will be more discussion. Because it feels like a really major entry into the conversation to talk about cost benefit analysis.
And also, just the way that this conversation has unfolded going to the more qualitative pieces that we can share that talk about importance, I'm just really appreciating that approach.
Apurva: Thank you, Amy. I think I found the report, you can tell me after taking a look if that's the one you were referring to. Emilia, over to you.
Emilia: Yes, just on the topic of conversations and stories, I think what everyone's been speaking to is why some of our initial conversations before you started looking at the data were actually so important. And while we did end up drilling down to more data driven questions, we actually started with that bigger picture with values driven questions around looking at what openness means as a value to USQ.
And to anyone actually involved with that process including students as well. And yeah, we did that before we started refining questions around the data itself. And it was really great to actually have just to be starting from a values-based kind of approach to how we were communicating evidence to what we were collecting. And to have those initial conversations around it, I think it really defined I guess the trajectory that we were taking and what we might collect in the future as well.
Karen: Yes, Emilia. I can really appreciate how taking that values-based approach makes it feel meaningful for everybody at that human level, that Amy, Apurva and maybe of us have been talking about today. And I see too that Adrian had a related comment in the chat and so thank you for sharing that, Adrian. We are starting to run out of time. And so if there are any pressing final thoughts or questions that you would like to share, there are some great resources shared in the last couple of minutes.
I'm really curious about the MHEC report, and I know that we have an Office Hours quick form that we love to share at the end of each session to invite you to inform what we talk about next. We look at them every month during our planning sessions, so Apurva will drop that in the chat. There it is, we invite you to nominate a speaker, yourself, a topic, whatever strikes your fancy, and we'll try and flesh it out and bring it to you in the future.
For now, please join us in thanking our guests, Emilia, Barb, and Tara for joining us today to have this conversation about impact, what that means to us in our work. And it's always nice to see all of you and get together and just have a place where we can share ideas and talk freely about what we're trying to accomplish together with students and for students. So with that, I'll turn things back over to Apurva for a final farewell.
Apurva: I will just say thank you everybody for all of your contributions, your observations, your willingness to share your experiences. And for being able to come across many different time zones today as well. I always learn a lot at these sessions, and I find that especially this year, 2022 Office Hours has left me with a lot of reflecting in sort of ways that we can continue to do more and have greater impact.
I think that's what we're all trying to do. So I will just say thank you again. And maybe pause for a minute while Amy tries to put that link into the chat, which I think has been accomplished, wonderful. Well, thank you all, Emilia, Tara, Barb, appreciate all of your expertise and look forward to seeing you all next month for another Office Hours. Bye everybody, take care.
END OF VIDEO
Share this post:
Watch the video recording of this Office Hours session, or keep reading for a full transcript. For those interested in reading the conversation that took place among participants and the resources shared, the chat transcript is also available below.
Note: If your comments appear in the transcripts and you would like your name or other identifying information removed, please contact Tonia.
Note: If your comments appear in the transcripts and you would like your name or other identifying information removed, please contact Tonia.
Audio Transcript
Speakers:
- Apurva Ashok (Director of Open Education, The Rebus Foundation)
- Karen Lauritsen (Publishing Director, Open Education Network)
- Emilia Bell (Coordinator, Evidence Based Practice, Library Services, University of Southern Queensland)
- Tara Lebar (Associate Director, Academic Affairs, Kansas Board of Regents)
- Barb Thees (Community Manager, Open Education Network)
Apurva: Hello everybody. Welcome to another Office Hours session. It is lovely to see so many of you from so many different parts of the world. For those of you I'm meeting for the first time, my name is Apurva Ashok. I am the director of open education and the assistant director at the Rebus Foundation. And I work on the Rebus Community project.
We really try to build human capacity in OER publishing and open education through professional development, the publication of free openly licensed resources and through sessions like this one, Office Hours, which we've been co-organizing with the Open Education Network for many, many years now.
Before I pass it over to Karen, I also just want to acknowledge that I'm joining you all today from the traditional territories of many nations. I'm joining you from the territories of the Mississaugas of the First Credit, the Anishnabeg, the Chippewa, the Haudenosaunee, and the Wendat peoples.
I'm very grateful to be here on this territory, I'm grateful for the privilege to be able to live and meet and learn here. And look forward to conversing with all of you and learning more about where you might be joining us from today. Karen, over to you.
Karen: Okay, thank you, Apurva. And welcome everyone, we are so glad you could join us for another session of Office Hours. I am the publishing director with the Open Education Network. My name is Karen Lauritsen. And we are a community of professionals working together in higher education to make it more open. And if this is your first time joining us for Office Hours, we are going to turn to you for questions and conversation after we briefly hear from our guests.
It's intended to be a casual conversation. As we all come together from different parts of the world, I am joining you today from San Luis Obispo, California, which is the traditional land of the Northern Chumash. And today we're going to discuss showing your work, tools for reporting impact. We are joined by Tara Lebar, who is Associate Director of Academic Affairs with the Kansas Board of Regents; Barb Thees, who is Community Manager with the Open Education Network; and Emilia Bell who is Coordinator of Evidence Based Practice with Library Services at the University of Southern Queensland.
We're so delighted to have the three of you with us today, as well as all of our participants. We invite all of you to contribute to the conversation. I am sure that many of you are also engaged in reporting the impact of your OER programs, learning to make the case and to tell stories about why it's so important to support OER development in open education. And so we invite you to chime in in the chat and then as I mentioned, we'll turn it over to you for conversation after we hear from our guests. So I think that's it, and to kick it off, I will hand things over to Tara.
Tara: Great, thank you very much, I'm excited to be here. What I've learned in my 18 months of being with the Kansas Board of Regents and working with the OER, the Kansas OER Steering Committee is that no matter how much you know about OER, you never know enough about OER, right? And you never really feel like you are an expert.
And I definitely don't feel like an expert, but I'm excited to be here to share at least some of the things that we've done with our Steering Committee as OEN members. And how we've used the dashboard to expand our faculty workshop programs, expand and use a state stipend grant with those. And then also I started to collect some of the adoption data in there.
So I'm excited to show you what that looks like as well. So I'm going to share my screen here. And let's see. A little bit of background on the Kansas Board of Regents. The State of Kansas has 32 public institutions. Seven public universities, seven technical colleges and then 19 community colleges. And we are the governing body for all of the public universities and then we coordinate with the community colleges and technical colleges across the state.
And so, it's a partnership, but it's also something that we kind of all have to walk together on because we all have different governances, until the state tells us what we have to do, and then we all have the same boss. So we do have a lot of collaboration, and one of the great things about the OER Steering Committee is we do have representation from all different types of institutions. And most of our institutions are represented on that Steering Committee.
And so, the work they've done together to advance OER initiatives across the state has been phenomenal in the last two I think it really started, they started slightly before I joined the team, maybe a year right before Covid. So in the three years that this group has been going, just the programming about OER has been really exciting statewide. So you can see here, so this is our OEN dashboard.
We joined a year ago as a consortial member. A couple of our institutions are institutional members from OEN. And we had some trainers that had worked specifically a trainer from University of Kansas that was very connected with OEN. And then, another trainer at Fort Hays State University was connected with OEN. And so we had some folks that were really connected and spoke highly of the organization.
And so we were able to do this, and we saw this membership as a way to share their resources and specifically the programming with all of the institutions in our system across the state. And so last year, the Spring of '21 we implemented five faculty workshops, and you can see that, using the OEN faculty workshop model. And then we had faculty from all across the system. These filled up super fast.
We kept them moderately small so that we could encourage discussion. They filled up very quickly. And then, we invited just like the workshop model does, we invited the participants to go into the OTL and enter an online review if they chose to. And if they did that, then they were eligible for $150 stipend from the state. So we've been doing that. This Fall we continued that workshop series by offering another six workshops, again, multiple institutions.
We used our trainers, all these workshops were virtual. And so, here we were using the OEN dashboard as it was intended, as it was presented and as a way to register for those workshops and track participation. And what's nice about the dashboard is that Barb's team they have created all the registration, the emails, the communication to go to these participants and make it very easy for us to put this together.
And especially for me, at the Board of Regents office location, one central location, it was very easy for me to just put the details of the event in the dashboard. And then, the emails were sent directly, or we sent out the registration, people signed up, faculty signed up to the one they wanted to attend. It was really slick. What it also created though, is all of these participants come with an email and a contact, which has been nice.
And so, we did 11 workshops last year over the course of 2021. And we realized we still had quite a bit of grant money left over. And so, we transitioned this model into using it for our institutions, and we opened it up to any of our 32 institutions to hold a training for their faculty as a professional development event. We said we'll provide you the trainers, if you want to offer the stipend, we have enough money to offer a stipend.
And so, you can see since January we've had several of these institutions reach out and create their own. We have another one coming up next week, Johnson County Community College. And these vary from the professional development that everyone has to take, to an optional session created just for faculty. And so really just letting the institutions, they can be virtual, they can be in person.
And with this model, the participants don't register through the dashboard, the contact just sends me who's signed up. Now that really has been the extent of how we used the dashboard up until a couple of weeks ago. Because as I was looking the dashboard was recently redone, and so I had used activity things to send out invites and reminders and track our stipends and things like that.
But because I looked over on this reporting status, they added a couple of things. Well, all of these graphs were empty. And really, the only thing I would scroll down, and the only graph I would see is this. And I'm kind of looking at this going, "Boy, it would be really, really cool to be able to see student savings, student impact, faculty using OER." So one of the surveys that was added recently from OEN to the dashboard is called an adoption survey.
And we hadn't played with that yet. And so what I did was I clicked on the adoption enrolment update, and I sent that out to all 188 faculty that we had interacted with during 2021. Now, again, these are small numbers, and this is just a start. But this is the information that I got back from those 188 participants. And so just that small slice and really, it's probably only about 50 or 60 participants that had really answered and responded.
All of a sudden, this data started coming to life, too in ways that now I'm thinking ooh, how can we do this? How can we replicate this on a larger scale? How could we get this out to all of our institutions? Because this was really exciting to see. Barb, did you want to chime in here?
Barb: I think I'll let Emilia go, this is giving me great ideas and I'll circle back on it. Thank you, Tara.
Tara: Cool, no worries, well so this is the start of what got me excited about looking at different ways that we could use that. And maybe even twisting OEN's arm a bit to see if there's a way that we could share that survey with more than just workshop participants. If there's a way that we could send that out to our whole entire system or something in that aspect. But I love the simplicity of this survey. I think it's five or six questions.
It's very easy and it talks very clearly about what class you're teaching, how many students. Talking about sections and numbers of students and really getting that concrete information from the faculty member that's answering the question. And so, like I said, this is a cool tool that I am excited to play more with. And so, with that, I will maybe toss it to Emilia. And be happy to answer questions as we go along today. And I'll stop sharing, there we go.
Emilia: Wonderful, thank you and hi everyone. So I'm going to talk a little bit on how we've highlighted the reach of USQ's open texts which are on the Pressbooks platform, using Power BI as a data visualization and dashboard tool. And this is work that's emerged from conversations between two library teams. So that's the open educational practice team and the evidence based practice team, which is where my role is.
So we had several initial discussions and ended up taking a phased approach to impact. And our first stage has involved showing the attention or the reach with web analytics data. So we've created a Power BI report that has a dedicated page for each of our open texts and it can be accessed by individual authors. It's probably worth mentioning as well that web analytics data is very easy to access, can be usefully measured, can appear very promising.
So there's a lot of appeal to it. It's been a lot more challenging, however, to actually recognize what's actually meaningful and accurate from it. So there are limitations that have required some consideration and transparency. But really, the dashboard has highlighted the attention that open texts receive and a new way just to promote that OER output as well. So we've been able to not only visualize the reach or attention, but to also provide just a single space for authors to explore and interact with data themselves.
There's a few groups accessing the Power BI report, so that includes academic authors and library staff as well. And what we're aiming for there is to actually make this as accessible as possible, so that authors can really easily navigate and explore the data independently. So I'll share an example of this work. Awesome. So as I mentioned, we have dedicated a separate page for each open text.
And this means that authors can simply select their text and find the data specific to their own text or chapter. So we'll look at academic success. And initially we can see the page views and downloads. And academic success has different authors for each chapter. So we've drilled down to this level of detail, so that the numbers on page view is actually relevant to individual authors as well.
Some of our other measures included sources of traffic, browsers and devices, and geographic reach. And after a few iterations and some feedback we chose to include these measures on the same page open texts. Just to use tabs, which are Power BI bookmarks to switch between different views. So we can switch to sources of traffic and still on the same page we've just built upward on the canvas and hidden the other visualizations.
So we've got sources of traffic, which is showing the referral path, so the website or search engine that they've used to find through. And we've included any social media sites, USQ's learning management system, which is called Study Desk as well as access by any other university or school links as well. So from many other websites. There browsers and devices was a little bit less specific for reach or impact, but it's provided opportunities for greater evidence based decision making when creating open texts.
And getting an understanding of what forms of technology are actually being used to access our open education resources. And then finally, under the last tab, we have access by geographic location, so by country. And that's really interesting just to explore between different open texts and subjects and to be highlighting reach in that respect as well. And we needed to accommodate several needs for evidence.
Some around decision making and others to actually highlight that reach and attention but we still had to be providing some of those visualizations that were considered and responding to the questions that we were asking. And really there's only so much canvas space that could be used effectively, while also facilitating authors and library staff to actually frame their own stories around reach and impact.
And within the library we can repurpose and refine any of these visualizations in other dashboards or communications, just as part of the larger narratives around openness. When we were designing the dashboard though we were also considering what measures can actually signify impact. So I think some of the language that we're using reflects our ongoing discussions around that.
So at this stage, we've addressed the reach of open text or the attention that they receive as being at best proxies for actually impact on user engagement. So we've distinguished between attention and impact. And that's where our phased approach comes in in that we're intending to then collect and incorporate the data on adoption and reuse of open texts as well as later qualitative evidence.
Just to provide opportunities to actually highlight other voices which is going to mean that we can extend beyond just quantifying reach and intention that our open texts receive to further emphasizing the value and impact of their openness as well. So thank you. And I might pass on to Barb now.
Barb: Thank you both. And I'm going to circle back kind of on similar things to what Tara was sharing with her screen. So I was a part of the team that like Tara mentioned revamped the dashboard. And I do want to give a shout out, there's a lot of OEN members in this crowd who use the dashboard on a regular basis. We have some data rockstars in here, too that we tapped in to create this version of the dashboard as well.
So I invite any of you to share or chime in if anything is really striking a chord with you. But just to expand on some of the things that Tara shared, don't worry, this is not me sharing anybody's data. But this is our test account. And as she mentioned, so the general purpose of the dashboard itself is to track the impact of open educational initiatives of our members. And so, it does have this capability, where like Tara was saying administrators themselves can add details on their programs.
Whether that be a grant program, an event you're hosting, a workshop you're hosting, and who participated in that. And then, also has this capability, as she showed, to actually create email campaigns that are then sent out directly to instructors where they can add that information themselves. So it has that flexibility for administrators to manage, but then hopefully keep the information as up to date as possible with including the instructors in the data process directly having them added to the dashboard.
Another one of the things that I think is unique about our data dashboard and one of the reasons that we did want to do a revamp is because we have a number of consortial members, as well as individual institutions who are using the dashboard, who work with one another. And there is that overlap between faculty that attend professional development OER related events, through either the consortium or through their local institution.
So something I want to show you which Tara is at the consortial level and that is the view we're seeing here is when you login to your dashboard you see the programs that you have created. But then, down below if I'm a consortium, for example, I can see the programs that my member institutions have also created in terms of their open education initiatives more locally.
So with that, I can see what they're up to. I can see which of their faculty are participating and within an individual faculty profile, I don't have an example in this screen. If someone has participated in more than one program, whether it be at the local level or the consortial level, they're marked on here and you can see that within their profile and vice versa. So if you're an institution that's part of the consortium, you can see the programs you've created up top.
And then, any programs that your local faculty have participated in at the consortial level, you can see the basic details down on the bottom of your view here. So we're hoping that that really helps in terms of transparency and facilitating communication between these different levels that are working with some of the same faculty members and instructors within the open ed space.
And then, I am going to come back to the individual profile. So in addition to you start at the program level and you can zoom in to an individual instructor. You can see the basic information for the person which in this case Mia Hamm, who knew she was involved in open educational work? You can collect basic details about a faculty member, make any notes about their engagement in your programs.
View which programs they've participated in, and then you can also see which activity requests you've sent to them. So what have you invited them to do? You can see details about the campaigns that they've been a recipient of. And then, zooming in even closer, you can see how they've engaged as a result of you reaching out to them through the data dashboard.
And that would either be by for example, if you had sent them an invite to review an open textbook in the Open Textbook Library, you can see how they've engaged in that way. Or if they have any adoptions associated with their account, you can view that information, as well as any enrolments associated with that individual right in here. So this information, like we've been saying gets populated either you can do it manually as the administrator.
Or this is something that perhaps Mia Hamm would enter directly based on a communication that she received, or what we call an activity request that she received through the dashboard. And then just finally to come back to the reporting, the data slices these visuals are downloadable. So the hope is that we polled the community, we're asking what is most helpful in terms of using your data for your advocacy efforts and your reporting efforts.
And from that, we've put together these different visuals that Tara was showing, that you can easily download and plop into a report or a slide share presentation somewhere. And then also, for our superusers, which again, we have lots of them on here. There is a way to download in spreadsheet form your data to extract it and be able to manipulate in whichever way serves your program or your reporting needs best.
So I will leave it at that, again, knowing that there's a lot of OEN members in the crowd. I did just want to drop a link. We have a documentation site that outlines all of the capabilities of the dashboard with some screenshots. So if anybody wants to dig deeper into that, that is now in the chat.
Karen: Thank you Barb, Emilia, and Tara, it's great to hear from you about how you've been leveraging different tools, both at the programmatic level and at the resource or book level. And thinking about who has access to that data. Emilia, you talked about really wanting authors to have direct access to the data related to their resource, which is interesting. And so now is the time when we transition to a conversation and talk about the tools that we've seen today as well as more generally about reporting impact.
So we'd love to hear more from all of you about what you're working on, what the challenges in reporting impact for your OER programs, what you've found to be effective, whether it's hard numbers or stories or both. And so with that in mind, I thought maybe I'd start with you, Tara, if you could tell us a little bit more about what you do at the state level to sort of share back the impact of your OER programs and what you've found legislators and other really glom onto when you're doing that reporting.
Tara: So data reporting is a newer function of our state organization so far. And I am the state liaison from our office. But unlike other states, it's a little part of my job, and so our big dream would be to have an OER staff person in academic affairs. That honestly, I'm working really hard to work myself out of that job, only because we have an amazing steering committee of professionals and they are wonderful to work with.
But wow, the things that we could do with someone who has OER background working full time in advocacy out of academic affairs would be fantastic. So like I said, my end goal is to get that position built into Kansas Board of Regents and I think that's definitely a doable request. But so what we did last year though, in an attempt to create some baseline data for our state was we looked at several different states.
And we created an annual survey, so we put that out towards the end of the academic year, last Spring. So we're getting ready to put another one out. And then we created a report that we could give to our Kansas Board of Regents just on those open education initiatives and resources. So like I said, last year was baseline data. This is going to be our first year of measuring.
But this year I can tell you, yes we had 11 faculty workshops last year. We also did our very first OER summit statewide this past February that was virtual. And that was our first foray in a year out of the OEN faculty workshop model, too. And so looking at different ways to reach faculty, and I remember we had about 350 attendees for our virtual summit, we were really thrilled about that.
And I remember just hearing over and over again, "Oh my gosh, I didn't know there was so much OER happening across the state." And that was from our OER Steering Committee members, these are the people in the trenches, and they're going, "Holy cow, I didn't know all these other people existed, too." So I've been really excited to see the expansion, the excitement, the encouragement.
And I think we've only just begun to tap some of that. But measuring it is a beginning thing, in fact I talked to my boss earlier today and he was asking because we are looking at our performance agreements. And so we're looking at is there a way that we could put OER as one of the performance measures, specifically for our community colleges. And not because the universities, but because it's just one of those things that A it provides a measure of validity to the effort.
And it gives those schools that might need an excuse to try something new that excuse. So oh, well if it's something that we can put down on our performance measures for the next upcoming year, what the heck, we've been talking about that, or we've got some folks that are already doing some of those things. And so that's one thing that we're looking at.
And I think as we look forward, we're going to be putting together a showcase where we're going to be able to show our Board of Regents and our legislators all the different things are happening with regard to OER at our individual institutions. And combining that with putting together our legislative ask and so our Steering Committee is working with our legislative liaison to make that a budgeting priority for our board in the next session.
So those are the things we're looking forward at right now. And I'm hopeful the data will help us in that as we continue to grow with that. But also, because of these graphs that I've been playing with recently, I'm tempted to see if some of those questions maybe can get in or if there's a way that we capture some of that. Because like I said, it's pretty powerful. It's hard to capture student savings and quantify that. But man, when we can, I think that starts really speaking to the power players, right, to the decision makers and the money holders.
Apurva: Thanks, Tara. And your excitement around what you can do with this data reminds me of what Emilia was saying about thinking just beyond the reach and just beyond those numbers. Emilia, I'm curious if you could take us with you to some of those discussions that your team has been having around other ways to demonstrate value and impact.
You showed us a lot of really excellent book level metrics, but I'm curious about what the ideas are in your discussions? What is brewing and where do you see yourselves going as the next step?
Emilia: Yeah, so for starters looking at extending beyond reach and attention, we were looking at the language around how we communicate the impact and considering what we were actually counting as impact. So we considered whether what we were collecting could actually show impact and reflecting on what was being counted or emphasized when we were tracking page views or downloads.
So we chose to make that distinction between impact and attention. So page views might show the attention an open text received or interest in it, but not how it was actually engaged with. And there are some presentation slides, I think it's by Luc Boruta on the impact of open that really explores this. And I'll share it in the chat in a little bit.
But it's on open access metrics, I love the ideas around how we measure impact and what's worth counting are really relevant to explore with OER, too. So across the different stages of our approach to impact, some metrics or evidence won't necessarily show that on their own. But they will show that reach or attention that an open text receives. And I think that's where we need to be actually critically reflecting on what's meaningful for us to actually be collecting.
And how we can demonstrate the meaning and value a user gains from open text. So not everything is going to be worthwhile counting or is going to highlight impact or even attention in a meaningful or relevant way. And that's also going to vary a lot between different local contexts and different stakeholders as well depending on who this is being communicated to.
Moving forward though, adoption is going to be the next part of our stage. So looking at reuse, attribution, so part of that would be internal and include the use of textbooks in courses. And so, that internal reporting would really just help to collate USQ activity around OER. And separate to that, would also be adoption of USQ texts by other institutions as well. And we can explore different approaches to how actually communicating that in evidence how we might incorporate it with what we're already collecting.
And also, just keeping to help that reflection on our current stage and revisiting the questions that we're asking around the data there. Whether they need to change, whether we need to change what we're collecting, how we're communicating it, who we're providing access to. Whether we need to be building other dashboards, or other means of communication around it as well. So yeah, that's really where we're heading with that.
Apurva: Thank you so much. And Michelle was asking I think for any of our speakers if any of you have tied this back to retention or completion? And I'll extend that question to everybody here on the call. If you had a chance to use the data you've collected around your OER initiatives to talk about retention or completion of courses that you've seen your students perform better with OER.
Emilia, it sounded like this is something that you're hoping to do with the next iteration. Barb, is this something that you can get from that adoption form information in the dashboard? Or is that perhaps a future feature?
Barb: Yes, I think that would be a future feature. Kind of like Tara said, a lot of people are just hopping into the adoptions part, so it's going to be exciting what we can do with that.
Tara: No, I was just going to say I think it's more of a function of course marking. And once you can get your course marking in your institution, then you can trace that GPA and maybe do some comparisons there or at least follow your students that way. We have a couple of our institutions that have fairly elaborate course marking systems and are able to do some tracking.
But I don't know that I've seen anything that has tracked them all the way to retention or completion that way. But I do know that course marking is pretty helpful in that. I mean, course marking is helpful for students knowing how to sign up for classes, but I also think there's the other side of the course marking piece, too where you can follow that and track that data on the back side.
Emilia: Yeah, I was just going to say very early stage for us at USQ as well in this area and probably Australia I think as well. So it's just something that we're going to continue to explore and yeah, very much a staged process.
Apurva: Thank you, I'm actually wondering if, Tara, you talked a lot about reaching out to legislature administrators and stakeholders. But this retention or completion question has me wondering whether the authors who might have access to the data at USQ, what has their response been to seeing the impact? And has that driven any desire for change at the classroom level in terms of pedagogy?
Because that's also another way I think that we can talk about the impact of OER. Right? There's that student piece, but also the faculty or teacher side.
Emilia: Yeah, absolutely. I'm not sure about actually in the courses themselves. But we've received plenty of positive feedback from several stakeholders, including authors themselves. So we took the time to actually walk through groups of authors from each open text through the dashboard and have got plans as well to record some more walk throughs around that just to make sure that everyone's familiar with how they can use it, any of the limitations associated with it as well.
And some authors have actually felt really encouraged or driven by it to want to create more open texts, so that was some really positive feedback that we received as well. And it's also prompted several changes in the design and granularity of the dashboard. So from the outset each text had its own page, but we've realized that a lot of the data around that wasn't just page views or downloads was being missed.
Because everyone would just go to their own page and forget that the rest of the dashboard existed, which is why we changed the navigation around to include that data on the same page, there. And also, filtering down to day that was another request we had, and some other changes that we made. But on the whole it's been yeah, really positive feedback that we've received from our academic authors. Yeah.
Apurva: Thank you and I see, Adrian, you seem to have sent me a message in the chat. But perhaps you would like to share it with the larger group around other ways in which USQ or others have used this data? Yeah, Adrian mentions that they've also used the data in academic promotion rounds, for learning and teaching fellowship applications and thinking about their learning and teaching practices. And Tara, you were talking about advocating for that OER staff person, so it seems like there is a way in which to make the case for it and to help with that, either career advancement or carving out of new roles as well.
Tara: And one of the things that one of our institutions did that sounds kind of counterintuitive at first, but I think it's proven really popular is with their course marking system, they indicated all of the low or no cost courses. And added a very small stipend, $10 stipend, that the student would pay in lieu of obviously a larger textbook fee. But with that, the money paid for those students in those courses a portion of that money goes back to the department for OER initiatives.
And so, there was incentive to create more for faculty to revamp their texts and their curriculums to using open education resources when appropriate. Nobody was pressured. But they had access to resources if they wanted to use time or to go learn or whatnot, they had a pocket of money that they could use for their department in the effort of OER to add more OER courses to their department.
The other part of that stipend fund went back to the university to do again, OER grant programming school wide. And so, while it was a very small cost to each student in those OER courses, it helped fund more OER initiatives and create that motivation for faculty to take that step and maybe stretch themselves or add another OER class. Because again, the more classes you had, the more money you generated for your department. So there was a little bit of a win-win there, and I think that that also helped encourage faculty.
Karen: Thank you and thank you also to Emilia who put a link in the chat for a presentation impact cannot be measured and other sad half truths about impact measurement. Not having seen this presentation, I'm just going to take a leap and say that I identify with the title. Because I do think that that is sort of a larger, if you will, philosophical struggle with what we're talking about today in terms of wanting to quantify everything.
Wanting to put a number on something, just looking for ways that we can quickly tell people why these programs matter. And sometimes that can feel reductionist or frustrating, and yet it's also understandable that we probably need more than stories. So that's the position that we find ourselves in. And so, very happy to talk about that too. Amy, I'm not sure what you were "ha-ha'ing, me too." Who presented the webinar?
Anyway, I invite anyone to chime in. Maybe just in thinking about some of the things that our guests have been reflecting on. I wonder Barb, in your role because you hear from so many new OEN members who are being onboarded and perhaps just getting their programs started. Do you hear about what they're doing? How they imagine tracking impact or pressures they might be under to do that? Or what are those early conversations and inquiries like?
Barb: That's a good question. I think it seems overwhelming. There are some people that come at it like we've done the more anecdotal things because of capacity measures that we don't have the capacity to be crunching the numbers in the way that we would want to. And so, when our conversations around the dashboard kind of come up, they see that as a boost to what they feel is a very valuable piece of this advocacy and reporting, which is the student voice, which is the more storytelling behind it.
And that kind of feeds into what Tara is saying, too I think. There is always a concern around capacity. But I think in my work and in my conversations a lot of what I say to people just starting out is that in the culture of open, relying on those and connecting with those who are further along in their journeys and learning from maybe the bumps that they've faced along the way is probably the best way to go.
So again, some of those people that I name drop are in this call right now, and I invite them to join in. But I think what I've heard also from the more experienced people is how they've gotten to their processes of the way that they use their data, the challenges the come along with it. Because it is so involved, especially when you're talking about multiple adoptions and multiple people who are parts of multiple programs.
And how you quantify the return on investment when there's that overlap between programs and people in them and what your instructors are doing. So that is always my biggest takeaway is lean on and rely on those who have come before you and worked out some of those kinks. And we're lucky to have a lot of those people in this call and in our community.
Karen: Thanks, Barb. I'm just going to allow for an awkward pause, in case any of those people care to chime in. It was worth a shot. Okay, Kaitlin in the chat, thank you for your question. Kaitlin is curious to hear your biggest hopes for this work, whether that's funding and policy change, finding non-traditional ways to measure student success and impact, perhaps increasing capacity on your local teams. Those all sound like great incomes, are any of them close to your hearts?
Emilia: I'll jump in just saying incorporating some qualitative evidence into this body of work, whether that's separate from the dashboard itself or whether we incorporate aspects into that dashboard or create more work from it. Yeah, love to see some other voices included just beyond the quantitative reach.
Tara: I think I'm most excited to see what we can do from the statewide level and if we can take it from a volunteer steering committee effort to something that is more policy either legislated or just policy written. I think we've got support, but we don't have anything written down yet. So I'm excited to see if that materializes for the state, because I think it would be really exciting.
But honestly, one of the things I love about OER the most is there's no downside, right? It's all good. It feels good, and regardless of as long as you're taking steps forward, it's exciting. And I love seeing people get connected. I love hearing the student stories, but I also love hearing the teacher stories about how they feel like they're doing something good for their students and they're more free to teach. Like I said, there's just not a downside, in my perspective. And so I just love following where the work takes us.
Karen: Thanks, Tara and Emilia. I will pause for a second on student stories. I'm curious to hear from our group, when we say student stories, if most of those focus now on hey, this was a free resource that I could access from day one. And that's made a huge difference in my educational career. Or are student stories also more sort of open pedagogy based or reflecting some of the open practices that faculty are engaging with students in creating these resources?
Are there both of those kinds of stories out there? Do you think that they hit decision makers or funders in a similar way? Or that they serve different purposes? What thoughts might this group have on that?
Kaitlin Schilling: I think they're both. I think it can be both. And in terms of that qualitative, I know the gut reaction is to say if it's qualitative or a story it can't sell to policy or legislature. But I think it's just finding the right way to tell that story and share that impact. Maybe it's not a number but breaking down that data and the number you do have of maybe showing equity or how accessible things have become. There are always ways to weave in selling points throughout stories. Hopefully that made sense.
Apurva: I think what you're saying also, Kaitlin, is that big reminder of this is not work in a vacuum, but this is work for people, right? Whether it's the story of the students being impacted as you were gesturing towards, Karen. Or I know, Tara, you talked about doing a showcase to the Board of Regents and legislature. Could it also just be a faculty or librarian or educator showcase of the various champions who've attended all of these workshops?
Who've seen all of this tremendous growth, who've changed their practice, who are bringing all of these different ways of hopefully innovative more student centric, empathy focused, trauma informed ways of teaching in the classroom. I think there is definitely a lot of different types of stories to share. And I wonder if focusing on the people and encouraging some conversation around that is on your radar, any of the three of you?
And I don't know, Barb, if you've heard from other OEN members whether that's something they have done or are planning to do?
Tara: Well, I think for me, the hook or the connection with those policymakers and stakeholders is everybody knows about college textbooks. Right? And buying the textbooks, everybody's got a story from when they did it. And they have their own child story, blah, blah, blah. That's the place we connect. But I think to Kaitlin's point, that's not the only story that's being told.
And you hear the stories, and it's not just from students that are economically disadvantaged, either. You hear this from every income level, from every college student across the board. "Well, I decided I didn't want to buy all the books for this class." Right? But what I love about it though is sometimes when you go and you ask students on campus, "What do you think about OER classes?"
And they go, "What are you talking about?" "Oh you know." And you explain it. "Oh yeah." Sometimes those are their favorite classes. Sometimes those are the most engaging classes and the classes that they'll go, "That teacher was the best teacher I had. That was the best class I had." And you hear that just as much as you hear, "And I didn't have to pay for the book." Right?
And that's what I love about this is it goes hand in hand. So yeah, we start there, but when you unlock the teaching and you get connected, faculty can unlock their teaching and get creative with their teaching, connect with students in different ways. Students aren't stressed out and they can connect from day one. It's just more than saving money on textbooks. So yeah, and I agree.
To your point, our showcase is going to highlight all the OER initiatives happening on our institutions, not just that student voice. But all the initiatives that have been happening and we had a student keynote speaker for our summit, and he literally stole the show. You know what I mean? You can't go wrong with that student voice ever.
Apurva: That's definitely something we're always mindful of because in these conversations, which is faculty or librarian focused, we don't have a student necessarily in this room. So I'm always mindful of that. Amy, why don't you do next? And Emilia, I saw you unmute as well, so I'll pass it over to you after.
Amy Hofer: Yeah, I'm curious if anyone else who is in the MHEC presentation this morning that Katie Zaback gave. So she was presenting a new document that MHEC, which is the Midwestern Regional Compact created about trying to find standardized ways to measure student savings, impact, and cost benefit analysis. I loved that she took that approach, because there's cost to any implementation of course materials whether open or not.
Anyway, it was a really tough crowd in the chat, and a lot of back and forthing about which dollar amount is correct. And should it be 68? Should it be 160? And I didn't want to jump in on the dollar amount, because I feel like the way that this conversation has been emphasizing there are so many impacts that go beyond whether it was $68 or $78 right? And also, just the fact that I hear from students that a $10 book can be too expensive.
A $10 book can be barrier to success, truly. So trying to figure out the real dollar amount can feel sort of quibbly at that point. Anyway, I just wanted to say that that report is out, I'll dig around for that link. I think it's still open in my browser, but I'm hoping that there will be more discussion. Because it feels like a really major entry into the conversation to talk about cost benefit analysis.
And also, just the way that this conversation has unfolded going to the more qualitative pieces that we can share that talk about importance, I'm just really appreciating that approach.
Apurva: Thank you, Amy. I think I found the report, you can tell me after taking a look if that's the one you were referring to. Emilia, over to you.
Emilia: Yes, just on the topic of conversations and stories, I think what everyone's been speaking to is why some of our initial conversations before you started looking at the data were actually so important. And while we did end up drilling down to more data driven questions, we actually started with that bigger picture with values driven questions around looking at what openness means as a value to USQ.
And to anyone actually involved with that process including students as well. And yeah, we did that before we started refining questions around the data itself. And it was really great to actually have just to be starting from a values-based kind of approach to how we were communicating evidence to what we were collecting. And to have those initial conversations around it, I think it really defined I guess the trajectory that we were taking and what we might collect in the future as well.
Karen: Yes, Emilia. I can really appreciate how taking that values-based approach makes it feel meaningful for everybody at that human level, that Amy, Apurva and maybe of us have been talking about today. And I see too that Adrian had a related comment in the chat and so thank you for sharing that, Adrian. We are starting to run out of time. And so if there are any pressing final thoughts or questions that you would like to share, there are some great resources shared in the last couple of minutes.
I'm really curious about the MHEC report, and I know that we have an Office Hours quick form that we love to share at the end of each session to invite you to inform what we talk about next. We look at them every month during our planning sessions, so Apurva will drop that in the chat. There it is, we invite you to nominate a speaker, yourself, a topic, whatever strikes your fancy, and we'll try and flesh it out and bring it to you in the future.
For now, please join us in thanking our guests, Emilia, Barb, and Tara for joining us today to have this conversation about impact, what that means to us in our work. And it's always nice to see all of you and get together and just have a place where we can share ideas and talk freely about what we're trying to accomplish together with students and for students. So with that, I'll turn things back over to Apurva for a final farewell.
Apurva: I will just say thank you everybody for all of your contributions, your observations, your willingness to share your experiences. And for being able to come across many different time zones today as well. I always learn a lot at these sessions, and I find that especially this year, 2022 Office Hours has left me with a lot of reflecting in sort of ways that we can continue to do more and have greater impact.
I think that's what we're all trying to do. So I will just say thank you again. And maybe pause for a minute while Amy tries to put that link into the chat, which I think has been accomplished, wonderful. Well, thank you all, Emilia, Tara, Barb, appreciate all of your expertise and look forward to seeing you all next month for another Office Hours. Bye everybody, take care.
END OF VIDEO
Chat Transcript
00:13:42 Iwona (she/ona/wiya/elle): Thank you Tara!
00:14:44 Kaitlin Schilling (she/they): Grateful to be joining you all today from a blizzard-y win-nipi (colonially known as Winnipeg, Manitoba), the shared traditional territory of the Anishinaabeg, Ininiwak, and Dakota Oyate, and the homeland of the Red River Métis. Happy to meet you all and see many familiar faces!
00:15:17 Adrian Stagg: Joining from Jarowair and Giabal lands in Australia.
00:17:05 Apurva Ashok: Constant learning!
00:20:43 Barbara R Thees: Info on the OEN Workshop Strategy that Tara is referring to: https://docs.google.com/document/d/1Cycd8tAs0r-MnBnIjh5MtNePStPzQdQhPPjOKzCuYn4/edit#
00:21:04 Apurva Ashok: Thanks, Barb!
00:24:44 Adrian Stagg: That's really impressive, Tara!
00:24:51 Amy Hofer (she/her): I agree, really nice work!
00:24:52 Apurva Ashok: Truly - congratulations!
00:25:36 Kaitlin Schilling (she/they): These are great visuals, Tara!
00:28:26 Amy Hofer (she/her): I missed which software Emilia is using for this dashboard?
00:28:40 Apurva Ashok: Power BI
00:28:45 Amy Hofer (she/her): Oh - she just said that. Thanks!
00:37:06 Barbara R Thees: https://sites.google.com/umn.edu/oen-data-dashboard-documentati/home
00:37:13 Kaitlin Schilling (she/they): Thank you!
00:38:35 Apurva Ashok: Feel free to post your questions in the chat, or raise your hand to unmute and ask our guests yourself.
00:42:17 Clare Sobotka TBCC (she/her): I have to take off, but thank you all!
00:42:29 Apurva Ashok: Thanks, Clare!
00:45:21 Michele Behr she/her: Wondering if anyone has tied any of this back to retention or completion
00:46:16 Sybil from NDSCS (she/her): @Michele - I’ve seen that connection made in a webinar… can’t recall which one though.
00:49:52 Sybil from NDSCS (she/her): I found my tweet about it: https://twitter.com/ihaveabug/status/1453378541614211077?s=20&t=Fa-dP130nuWm2dukjs4Wpw
00:50:29 Adrian Stagg: USQ authors have also used the data in academic promotion rounds, for L&T Fellowship applications, and thinking about their L&T practices.
00:51:10 Emilia Bell: Presentation slides on measuring the impcat of open: https://figshare.com/articles/presentation/Impact_cannot_be_measured_and_other_sad_half-truths_about_impact_measurement/16866355/1
00:51:22 Apurva Ashok: @Sybil - thank you!
00:52:53 Sybil from NDSCS (she/her): I wish I could remember who presented that webinar…
00:53:09 Amy Hofer (she/her): haha! me too!
00:53:52 Amy Hofer (she/her): the presentation title.
00:54:34 Karen Lauritsen: ?
00:55:13 Kaitlin Schilling (she/they): Curious to hear your biggest hopes for this work, whether that's funding & policy change, finding non-traditional ways to measure student success and impact, increasing capacity, etc.
00:57:25 Kaitlin Schilling (she/they): Love that!
00:59:20 Sybil from NDSCS (she/her): OMG I found the slides! https://docs.google.com/presentation/d/18WriIRlXZo262LisudZzOe3h7CMMZI9oEs-NQl7NQNg/edit?usp=sharing
00:59:45 Sybil from NDSCS (she/her): Slide 80
01:02:42 Adrian Stagg: We had some stories of students in our first course that offered open assessment. A number of students had previously failed the course, yet achieved very well in the open assessment offer. The lecturer captured some statements about how the students felt more engaged, saw the purpose of the assessment, and were motivated to 'do well'.
01:05:03 Apurva Ashok: Creating Clarity to Drive More
Consistency in Understanding
the Benefits and Costs of OER report from MHEC: https://www.mhec.org/sites/default/files/resources/2022MHECOER-Toward-Convergence.pdf?utm_source=msdynmktg&utm_medium=email&utm_campaign=22towardconvergence#msdynttrid=VmvtnKKQChDysdy-aIj7yfOOyxcm8b2-mfmsls0m8hQ
01:05:24 Sybil from NDSCS (she/her): To speak to Tara’s comments about students enjoying OER courses more, I think when teachers have materials they LIKE or CREATED, they care more. I never thought I’d “teach to a book” but now I teach to MY OER and I love it.
01:05:30 Adrian Stagg: From the lecturers' perspective, we've had staff who have really leveraged their open work for other opportunities (research, grants, speaker invitations, etc). We're planning for our next Open Showcase to have those staff speak so they can hopefully connect with other aspects of academic workload and career progression.
01:05:53 Amy Hofer (she/her): yes that's it!
01:07:20 Apurva Ashok: @Adrian - that is really exciting. @Sybil - thank you for highlighting that shift in perspective you’ve seen!
01:07:37 Apurva Ashok: * Any suggestions for topics or speakers are welcome: https://docs.google.com/forms/d/e/1FAIpQLScaGr1NCvVnk1C6uKiwkfYWvJcK0QDfwJIZJJV-ckmGK19Wpg/viewform
01:08:00 Tara Lebar: Amy- We've struggled around the measuring student savings and data points and work together as a committee to try and set some state definitions just for consistency sake around OER reporting. IT wasn't easy.
01:08:03 Amy Hofer (she/her): thanks everyone!
01:08:08 Barbara R Thees: Thanks for having us!
01:08:09 Michele Behr she/her: Many thanks. Very helpful
01:08:16 Adrian Stagg: Thank you everyone! These sessions are always so valuable!
01:08:36 Kaitlin Schilling (she/they): Thank you everyone! :)
01:08:39 Amy Hofer (she/her): @Tara we avoid consistency in Oregon! One sec while I look for taht link...
01:08:58 Amy Hofer (she/her): https://openoregon.org/local-data/
01:09:09 Cheryl (Cuillier) Casey | she/her: Thank you all
01:09:18 Karen Lauritsen: Take care.
01:09:19 Tara Lebar: Amy- Thanks for sharing!
00:14:44 Kaitlin Schilling (she/they): Grateful to be joining you all today from a blizzard-y win-nipi (colonially known as Winnipeg, Manitoba), the shared traditional territory of the Anishinaabeg, Ininiwak, and Dakota Oyate, and the homeland of the Red River Métis. Happy to meet you all and see many familiar faces!
00:15:17 Adrian Stagg: Joining from Jarowair and Giabal lands in Australia.
00:17:05 Apurva Ashok: Constant learning!
00:20:43 Barbara R Thees: Info on the OEN Workshop Strategy that Tara is referring to: https://docs.google.com/document/d/1Cycd8tAs0r-MnBnIjh5MtNePStPzQdQhPPjOKzCuYn4/edit#
00:21:04 Apurva Ashok: Thanks, Barb!
00:24:44 Adrian Stagg: That's really impressive, Tara!
00:24:51 Amy Hofer (she/her): I agree, really nice work!
00:24:52 Apurva Ashok: Truly - congratulations!
00:25:36 Kaitlin Schilling (she/they): These are great visuals, Tara!
00:28:26 Amy Hofer (she/her): I missed which software Emilia is using for this dashboard?
00:28:40 Apurva Ashok: Power BI
00:28:45 Amy Hofer (she/her): Oh - she just said that. Thanks!
00:37:06 Barbara R Thees: https://sites.google.com/umn.edu/oen-data-dashboard-documentati/home
00:37:13 Kaitlin Schilling (she/they): Thank you!
00:38:35 Apurva Ashok: Feel free to post your questions in the chat, or raise your hand to unmute and ask our guests yourself.
00:42:17 Clare Sobotka TBCC (she/her): I have to take off, but thank you all!
00:42:29 Apurva Ashok: Thanks, Clare!
00:45:21 Michele Behr she/her: Wondering if anyone has tied any of this back to retention or completion
00:46:16 Sybil from NDSCS (she/her): @Michele - I’ve seen that connection made in a webinar… can’t recall which one though.
00:49:52 Sybil from NDSCS (she/her): I found my tweet about it: https://twitter.com/ihaveabug/status/1453378541614211077?s=20&t=Fa-dP130nuWm2dukjs4Wpw
00:50:29 Adrian Stagg: USQ authors have also used the data in academic promotion rounds, for L&T Fellowship applications, and thinking about their L&T practices.
00:51:10 Emilia Bell: Presentation slides on measuring the impcat of open: https://figshare.com/articles/presentation/Impact_cannot_be_measured_and_other_sad_half-truths_about_impact_measurement/16866355/1
00:51:22 Apurva Ashok: @Sybil - thank you!
00:52:53 Sybil from NDSCS (she/her): I wish I could remember who presented that webinar…
00:53:09 Amy Hofer (she/her): haha! me too!
00:53:52 Amy Hofer (she/her): the presentation title.
00:54:34 Karen Lauritsen: ?
00:55:13 Kaitlin Schilling (she/they): Curious to hear your biggest hopes for this work, whether that's funding & policy change, finding non-traditional ways to measure student success and impact, increasing capacity, etc.
00:57:25 Kaitlin Schilling (she/they): Love that!
00:59:20 Sybil from NDSCS (she/her): OMG I found the slides! https://docs.google.com/presentation/d/18WriIRlXZo262LisudZzOe3h7CMMZI9oEs-NQl7NQNg/edit?usp=sharing
00:59:45 Sybil from NDSCS (she/her): Slide 80
01:02:42 Adrian Stagg: We had some stories of students in our first course that offered open assessment. A number of students had previously failed the course, yet achieved very well in the open assessment offer. The lecturer captured some statements about how the students felt more engaged, saw the purpose of the assessment, and were motivated to 'do well'.
01:05:03 Apurva Ashok: Creating Clarity to Drive More
Consistency in Understanding
the Benefits and Costs of OER report from MHEC: https://www.mhec.org/sites/default/files/resources/2022MHECOER-Toward-Convergence.pdf?utm_source=msdynmktg&utm_medium=email&utm_campaign=22towardconvergence#msdynttrid=VmvtnKKQChDysdy-aIj7yfOOyxcm8b2-mfmsls0m8hQ
01:05:24 Sybil from NDSCS (she/her): To speak to Tara’s comments about students enjoying OER courses more, I think when teachers have materials they LIKE or CREATED, they care more. I never thought I’d “teach to a book” but now I teach to MY OER and I love it.
01:05:30 Adrian Stagg: From the lecturers' perspective, we've had staff who have really leveraged their open work for other opportunities (research, grants, speaker invitations, etc). We're planning for our next Open Showcase to have those staff speak so they can hopefully connect with other aspects of academic workload and career progression.
01:05:53 Amy Hofer (she/her): yes that's it!
01:07:20 Apurva Ashok: @Adrian - that is really exciting. @Sybil - thank you for highlighting that shift in perspective you’ve seen!
01:07:37 Apurva Ashok: * Any suggestions for topics or speakers are welcome: https://docs.google.com/forms/d/e/1FAIpQLScaGr1NCvVnk1C6uKiwkfYWvJcK0QDfwJIZJJV-ckmGK19Wpg/viewform
01:08:00 Tara Lebar: Amy- We've struggled around the measuring student savings and data points and work together as a committee to try and set some state definitions just for consistency sake around OER reporting. IT wasn't easy.
01:08:03 Amy Hofer (she/her): thanks everyone!
01:08:08 Barbara R Thees: Thanks for having us!
01:08:09 Michele Behr she/her: Many thanks. Very helpful
01:08:16 Adrian Stagg: Thank you everyone! These sessions are always so valuable!
01:08:36 Kaitlin Schilling (she/they): Thank you everyone! :)
01:08:39 Amy Hofer (she/her): @Tara we avoid consistency in Oregon! One sec while I look for taht link...
01:08:58 Amy Hofer (she/her): https://openoregon.org/local-data/
01:09:09 Cheryl (Cuillier) Casey | she/her: Thank you all
01:09:18 Karen Lauritsen: Take care.
01:09:19 Tara Lebar: Amy- Thanks for sharing!
Share this post: