2021 Final Report Summary
Affordable Learning Georgia Transformation Grants
Highlights
Transformation Grants projects ending in Spring, Summer, or Fall 2021 took place entirely during the second year of the COVID-19 pandemic. While it may be difficult to determine the effects of a Transformation Grant project in 2021, we can infer the following from a summary of all 2021 Transformation Grant Final Reports:
- A total of 57 reporting teams saved 9,300+ students over $1.2 million just within the time of the project, and mostly within only the final semester of instruction.
- The COVID-19 pandemic caused a set of complications with the implementation and measurement of grant projects, including the continuation of drastic changes in instruction and learning after 2020’s emergency move to online instruction.
- The pandemic continued to increase difficulty on grant teams in gathering qualitative and quantitative data from students, exacerbated by multiple reports of lower student enrollment.
- Despite these complications, Transformation Grant projects were not likely to result in negative changes to student opinions, outcomes, or retention during the pandemic:
- Teams reported positive (84.85%) or neutral (15.15%) overall student opinions of the new materials.
- Teams reported largely positive (60.61%) or neutral (33.33%) comparative student learning outcomes compared to previous semesters, control groups, and/or instructor and departmental averages with a commercial textbook.
- Teams reported largely positive (48.48%) or neutral (30.3%) comparative course-level retention and completion rates compared to previous semesters, control groups, and/or instructor and departmental averages with a commercial textbook, but multiple teams remarked that the COVID-19 pandemic’s effects confounded these measures.
- Students who gave positive reviews of the new materials in courses often commented on cost, convenience, and relevance (to the course and to their overall career goals/lives) as positive aspects of the new materials.
- While many 2020 teams noticed that the pandemic affected instructors’ teaching conditions and collaboration, teams in 2021 noticed more frequently that the pandemic was adversely affecting students’ personal and academic lives during the course of their projects.
Background
Affordable Learning Georgia’s (ALG) Transformation Grants are intended to pilot different approaches in University System of Georgia (USG) courses in reducing the cost of course materials to students. Approaches include the adoption, adaptation, and creation of open educational resources (OER), the adoption of all-rights-reserved no-cost or low-cost materials, and the adoption of materials available through GALILEO and USG libraries at no additional cost to students. The grants support release time or salary/overload, extra staff assistance (instructional designers, librarians, etc.), materials, and professional development needed for faculty to transform their use of learning materials.
This report summarizes the findings of all grantees in grant rounds ending in calendar year 2021. Three rounds (spanning rounds 16 through 18) resulted in 33 Transformation Grants projects from teams of USG faculty and staff at USG institutions. This is a significant decrease from calendar year 2020, where over 50 Transformation Grants projects ended, but an increase from calendar year 2019, where less than 30 Transformation Grants projects ended.
All grant projects, along with a collection of their proposals, syllabi, and final reports, are included in the lists of Rounds 16-18 grantees:
The summary below addresses the perceptions and efficacy of OER and no-cost/low-cost implementations to students in course sections affected by ALG Transformation Grant projects. Continuous Improvement Grants are not included in this report, as the goal of these grants is the creation and revision of open educational resources—those results are visible in OpenALG and GALILEO Open Learning Materials repositories as well as the individual final reports for Continuous Improvement Grants.
Transformation Grants project teams submitted final reports at the end of their final semester, during which all implemented materials were taught within the course. Final reports included quotes from students and professors, data on student performance, drop/fail/withdrawal rates, and measures of student perceptions of course materials. Including savings estimates, this data meets all four requirements of the Open Education Group’s COUP Framework, measuring cost, outcomes, usage, and perceptions in each implementation.
Savings
During the course of their initial implementation, the projects in these three grant rounds affected an enrolled over 9,300 students, and projects saved those students an estimated $1.2 million in student textbook costs in relation to their commercial equivalent—the purchase of new commercial materials as previously required within the course.
With all teams indicating that these materials or other affordable materials will be used in future semesters, a high sustainability of these student savings is anticipated over at least the next academic year.
Grantee Experiences
Final reports continue to support the idea that grants for open, no-cost, and low-cost materials adoption activities are valuable tools in building affordable and sustainable learning materials practices among faculty with positive benefits for students.
“Student's overall grades improved, there was a decrease in Ds, Fs, and Ws, adjunct faculty felt more supported, and students were satisfied with the course.”
– Dr. Jessica Traylor, Gordon State College
Project Leads indicated in reports that the activity of redesigning a course with affordable resources enhanced their teaching and learning experiences, including instances of pedagogy that only open resources could enable (Open Pedagogy).
"In regard to our instruction, these new ways of teaching truly pushed us to be better instructors. The conditions allowed us to think outside of the box on how to deliver content in a concise and clear way."
– Dr. John Meyers, Clayton State University
COVID-19 Pandemic Complications
While the experience of participating in a Transformation Grant project was overall positive to team participants, 2021 was also the second year of the COVID-19 pandemic, with drastic shifts in the way teaching and learning happened throughout the year.
"While preparing this report, one must understand the teachers’ mentality and students’ acceptance to an eBook which was offered to them when pandemic was at its peak and everyone is fatigued and at end of their “normal” zone of teaching and learning.”
– Dr. Antara Dutta, Georgia State University
26 out of 33 teams reported encountering substantial barriers to a successful project that were caused by the long-term effects of the COVID-19 pandemic. These barriers included lower enrollment, issues with research, adverse effects on students, and changing conditions for instructors.
Students Affected
While teams noticed adverse effects on student success in 2020, 2021’s reports had a marked increase of teams noticing adverse effects on students’ academic lives.
"During the Covid-19 pandemic, student burnout was at an observably lower threshold and resilience seemed decreased. In addition, the lack of as many face-to-face sections means fewer opportunities for organic intervention to encourage improvement, despite our best efforts."
– Dr. Lisa Gezon, University of West Georgia
Not only did teams notice that adverse effects on students in the classroom occurred, they also noticed that the effects of the pandemic on students’ personal lives were affecting students’ academic success:
"Yes, [we] had students who withdrew because they themselves had complications with COVID, some who lost family members who died. Some [students] feared coming to class, so, it added many layers of stress for the students [and] myself."
– Dr. Dennis Miller, Clayton State University
Instruction Affected
Reports about teaching conditions being affected adversely during the COVID-19 pandemic continued in 2021, from the more obvious issues in shifting to online instruction to lesser-known issues such as sudden changes in class sizes or section enrollment.
"Some of our work was impacted by the pandemic in unavoidable ways. Our efforts to transition online and then teach using new technology and platforms in the fall kept us busier than we anticipated."
– Dr. Houbin Fang, Columbus State University
Collaborations Affected
While the emergency online shift to online communications happened in 2020, many teams still found themselves grappling with online collaboration as a replacement to in-person meetings, hindering plans to collaborate on grant projects.
"The main challenges that the team faced were due to the Covid pandemic. Since face-to-face meetings couldn’t occur, the team had to meet online and this reduced the ability to work collaboratively on the project.”
– Dr. Scott Sykes, University of West Georgia
Research Issues
COVID-19 and the emergency shift to online instruction caused the findings of many project teams to be less attributable to the introduction of affordable materials; determining whether or not the Transformation Grant affected student learning or retention was at times impossible, while opportunities to share research dwindled due to travel and gathering restrictions.
"The obvious co-factor that could influence outcomes is the COVID-19 pandemic. Due to changes in course set up and scheduling challenges, not all of the team members had a chance to teach Chem 1151 during the 2020-2021 academic year."
– Dr. John Meyers, Clayton State University
Lower Course Enrollment
Multiple teams in 2021 experienced lower course enrollments than usual, resulting in smaller sections for a pilot implementation and sometimes a delay in the project due to a smaller amount of course sections than expected as a result of lower enrollment.
“COVID-19 and lower enrollments may have influenced the outcomes of the project. This explains why the actual total number of students impacted by the transformation [in our final semester] is lower than the projected number of students impacted by the transformation [per semester].”
– Dr. Anastasia Angelopoulou, Columbus State University
Student Satisfaction
Teams reported that students were highly satisfied with the new materials. Student quotes highlighted satisfaction with the low or zero cost, the convenience of having and using open resources, and the relevance of the materials to what they were expected to learn.
28 teams (85%) reported overall positive student perceptions, and five teams (15%) reported neutral perceptions. Zero teams reported negative perceptions. In 2020, 100% of teams reported positive perceptions, but 2021’s findings are still an overall good outcome for no- or low-cost materials.
“I enjoyed how we didn’t have to pay for the textbook and it was easy to find more detailed information if I was confused on something.”
– Student of Dr. Christine Whitlock, Georgia Southern University
Student Learning Outcomes
Comparing student performance in a course before and after an Affordable Materials Grant always includes taking multiple confounding variables into account, but the complications and barriers encountered during the COVID-19 pandemic continued to make student learning outcomes even more difficult to attribute to a grant project alone. However, we can determine from the 2021 Final Reports that the implementation of affordable materials did not adversely affect learning outcomes during the course of the project.
20 teams (61%) reported positive significant changes to student achievement in comparison to control groups, previous semesters, faculty averages, and/or departmental averages, while 11 teams (33%) saw no significant changes to outcomes with the new affordable materials. Only two teams (6%) saw negative changes. This is a combined 94% of teams who experienced either a significantly positive or neutral effect on learning outcomes while using free and/or affordable materials, which is very similar to last year’s reports.
“Transformative impacts on students were that students were very positive about the no-cost learning materials. They felt the materials for the course were current, relevant and at the level that a textbook would achieve.”
– Dr. Rebecca Rutherfoord, Kennesaw State University
Course-Level Retention Rates
Comparisons of course-level retention rates during the COVID-19 pandemic continued to be almost impossible to attribute to a grant project alone. However, we can determine from the 2021 Final Reports that the implementation of affordable materials largely did not adversely affect retention rates in the shift to online instruction during COVID-19.
16 teams (48.5%) reported positive significant changes, 20 teams (30%) reported no changes, and 7 teams (21%) reported negative changes, which is similar to last year’s reports.
"One consideration is the impact of Covid during the grant period may have also impacted outcomes, with higher than usual W, WF, F grades overall, but this does not appear significantly present in this data.”
– Dr. Daniel Farr, Kennesaw State University
Lessons Learned
Each final report contains a section within the narrative document for lessons learned. From these 2021 reports, Affordable Learning Georgia has reached the following overall conclusions when implementing open, no-cost, or low-cost materials to replace a commercial textbook in a course.
As with previous years, OER implementation takes substantial planning in order to achieve goals within the deadline: Many teams with completed projects in 2020 reported time management and planning as a barrier to project completion, especially due to the pandemic. Some of these issues were also attributed to a lack of foreknowledge of the time it would take to complete major tasks and reach milestones. In response to this trend of time management being reported as a barrier to completion, Affordable Learning Georgia revised the application process in Spring 2020 to include more detailed time management within action plans, containing achievable milestones and an estimate of the time needed to complete major tasks.
2021 is the first year where this proposal change was in effect for final reports. Time constraints were still reported in 2021, but in a different way: mostly that adverse conditions, such as the COVID-19 pandemic and increased amounts of organizational change and turnover, affected the detailed plans created within the application.
"I would say that I greatly underestimated the time commitment necessary to complete this project, but the reality is that my workload doubled when the other faculty member who applied for the grant with me, a Limited Term lecturer, abruptly walked away from the project in Fall of 2020.”
– Dr. Kaleigh Kendrick, Kennesaw State University
Plan on ongoing development after the project: This year, a majority of the lessons learned that were shared in final reports were split on themes between time constraints and the need to make further revisions to the project after the implementation semester.
If possible, ALG recommends new projects to plan for a pilot implementation semester before the final semester. Having a full semester of testing the materials can lead to a more successful full implementation in the final semester after revisions take place. Otherwise, these types of revisions can be built into a proposal’s Sustainability Plan.
"One of the things we could change if we had to do it over again, would be to develop an assignment around the concept of building your own study guide."
– Dr. Lisa Gezon, University of West Georgia
When evaluating the quality of materials or planning on the creation or adaptation of new materials, focus on the relevance of these materials to your students. Faculty assessments of quality often diverge on what quality means, often because of differences in subject matter or pedagogical techniques. Quality aspects to faculty include comprehensiveness, design issues, technical issues, accuracy issues, writing styles, and opinions about using print versus digital formats. In 2021, when students commented on the quality of a resource, they always focused on relevance, whether saying a resource was streamlined, to the point, practical, concise, and/or related directly to assignments and exams.
Future projects should consider the relation of potential course materials to students’ overall experience with the course; for example, the connection between a textbook and the corresponding tests and quizzes, the connection between materials and the typical career path within a course’s corresponding major, or the connection between materials and the lived experiences of the students taking the course.
"To students in this class, I would highly recommend looking at the text, because it directly applies to the class objectives."
– Student of Dr. Skanda Vivek, Georgia Gwinnett College
"Not a lot of wasted pages like books in my other classes."
– Student of Dr. Dennis Miller, Clayton State University
“The books have provided practical and concise descriptions of how and why we research.”
– Student of Dr. Elizabeth Pope, University of West Georgia
"It streamlined the important topics covered in this course, and it provided supplemental instruction that helped me to understand the material presented."
– Student of Dr. Antara Dutta, Georgia State University
Conclusions
The COVID-19 pandemic affected higher education as a whole in 2020, and grant teams continued to report substantial barriers due to the pandemic in 2021. Shifts in instruction, learning, collaboration, and the ability to conduct research all had an impact on the success of Transformation Grant projects during the pandemic. These shifts continued to make determining the effects of implementing affordable materials more difficult.
Despite these complications and barriers, just like with 2020, it is clear from the 2021 Final Reports that the implementation of affordable materials did not adversely affect student perceptions, performance, or course-level retention while improving affordability. Grant teams funded in these three rounds still largely saw overall positive or neutral changes to student performance and to DFW course-level retention rates.
Students who gave positive reviews of the new materials in courses often commented on the relevance of the materials (to the course and to their overall career goals/lives) as a quality indicator, and future grant teams should consider relevance when planning the adoption, adaptation, or creation of materials. Despite the adverse effects of the pandemic, faculty still remarked that they enjoyed examining their pedagogical methods and the relevance of their materials to their students and desired learning outcomes.
Note on Analysis Challenges
Variation will occur with the number of semesters of data used and whether averages calculated were department-wide or instructor-specific. Confounding factors when measuring learning outcomes and retention efficacy can include differences in student composition between semesters, enrollment shifts, and organizational complications due to institutional consolidations.
Three uniform questions about student perceptions, student learning outcomes, and drop/fail/withdraw rates are given to all project leads within the final report as supplementary to their summaries of all research conducted. The interpretation of qualitative and quantitative findings, evaluations of statistical significance, and analysis of confounding factors are the responsibility of the faculty teams conducting the research.