The Relationship Between Quality Assurance & Student Grades in Online Course Design
Explore the impact of quality assurance in online course design on student grades. The study delves into limitations, controlled variables, and components required for similar research initiatives on campuses.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
DOES QUALITY IN ONLINE COURSE DESIGN MATTER? The Relationship Between Quality Assurance & Student Grades Alissa Sells - SBCTC Educational Technology & Open Education Darby Kaikkonen - SBCTC Research 10.29.19
Session Objectives o List 3 limitations of this study. o Name 2 confounding/explanatory variables that were controlled for in this study. o Identify components required to conduct similar quality assurance research on your campus.
SBCTC QM Implementation Course Reviews Official QM course reviews are not currently funded. Training APPQMR, IYOC and other selected QM training for faculty, and staff. Subscription System level subscription funded annually for all 34 CTCs in Washington.
Project Components 1 2 3 Research Question Research Partner Data 6 5 4 Participant Consent Library Access Permission 7 8 9 Editors Time Patience
1 Research Questions Mean student course GPA before & after APPQMR workshop completion? What is the relationship between Mean student course GPA before & after official QM course certification? Mean student course GPA before APPQMR workshop completion & after official QM course certification?
2 Research Partner SBCTC Educational Technology SBCTC Research Quality Partnership
3 Data (2008 2018) 138 officially certified QM courses Excludes 33 faculty without verifiable APPQMR training records, no training, or informal QM training MyQM SBCTC Data Warehouse Excludes 52 recertifications, non HE rubric reviews, and additional certified courses from the same faculty Excel Excludes +/-7 faculty who did not have course data for a full year before and after training and before and after course certification 46
4 Permission o No permission to conduct research was required. o System data collected quarterly for policy research analysis, accountability reporting, and research and evaluation of educational programs
5 Consent o Consent was not required. o Faculty were informed of their possible inclusion in the study by email.
6 Library Access CSU Student Quality Assurance Impact Research (SQuAIR) Hollowell, Brooks, and Anderson (2017) Ralston-Berg (2011) Harkness (2014) Baranovic et al (2017) Studying the impact of QA professional development and course certification on student grades, course completion rates, student satisfaction, and achievement gaps. Found a 66% percent decrease in failing grades (F), a 20% increase in passing grades (A-D), and a 23% decrease in course withdraws in newly designed online courses in relation to faculty QM training. Found a significant correlation between informal QM course review scores and student final exam performance and final grades. Found a steady increase in A- grades in courses meeting QM standards with the total GPA for all online courses increasing 1.25% from 2014 to 2017. Found no significant differences in student satisfaction, grades, or retention between before and after QM certified online courses.
7 Editors o Feedback provided by my supervisor, & my research partner. o Feedback provided by Kay Shattuck & Barbra Burch
8 Time & 9 Patience o This has been a 2 year process and we re not quite finished. o Still to do final content draft, journal formatting, 2 page brief, journal submission.
Assumptions o We had the data we needed. o Faculty would be willing to have their course included in the study. o Faculty made changes to their course after completing QM s APPQMR workshop and continued to make changes to their course until official course certification was achieved.
Limitations Small sample size. SBCTC system data may not be generalizable to other community and technical college students. No faculty demographics were collected. No record of specific changes made to courses. Does not measure every possible factor that might influence grades. Before and after data is compiled from different student sets.
Methodology - MEASUREMENT Compared the average before and average after course GPAs for each of the three measurement periods. After Before 1 yr. following the training or course certification date. 1 yr. prior to the training or course certification date.
Methodology - TESTING ANOVA Used to test for statistically significant differences in each of the before and after groups. Analysis of Variance Secondary Testing Used to assess if the observed change in student course GPA from before to after might be due to factors related to characteristics of the student population Equity Gap Analysis Explanatory/ Confounding Variables Age, sex, disability status, low income status, full or part-time status, number of college credits earned at the time of the course, kind of student, and historically underrepresented students of color.
Results Before & After Training Before Training & After Certification Before & After Certification ANOVA: Only the before APPQMR training and after official course certification showed a statistically significant increase. (.24%) GAP ANALYSIS: Revealed possible areas for additional study related to specific student populations.
Areas for further study o What is the effect of faculty QM training and the QM course certification process on targeted demographic groups such as students of color, students with disabilities, or by age or gender?
Other things to look at o Changes in course withdraw rates. o Changes in course completion rates. o Increases in passing grades.
Summary o The quality assurance process of training combined with course certification appears to contribute to increases in student grades more so than just training or course certification alone. o There is room for further study on the effectiveness of the Quality Matters program in general and in relation to specific student populations. 20
Potential Impact o To date our system has focused primarily on the training aspects of QM implementation and our recent findings support the importance of expanding that process to include course certification. o QM subscribers focused solely on the training aspects of quality assurance may need to re-evaluate and consider how to incorporate the course review process into their quality assurance program.
Lets Review o Who can list 3 limitations of this study? o Who can name 2 confounding/explanatory variables that were controlled for in this study? o Who can identify 3 of the 9 components required to conduct similar quality assurance research on your campus?
Contact Us Alissa Sells Program Administrator System Lead QMC Darby Kaikkonen Policy Research Director asells@sbctc.edu 425.239.0456 @WAeLearning dkaikkonen@sbctc.edu 360.704.1019
Thank you! Unless otherwise specified, the content in this presentation is licensed under a Creative Commons Attribution 4.0 International License.
