Remote proctoring of exams is one of the most divisive issues in higher education. Critiques of remote proctoring abound, and there are a variety of perspectives particularly in relation to the advantages and disadvantages of this type of assessment, and opportunities for cheating. However, these perspectives are largely based on rhetoric with limited empirical data to support or refute the value of remote proctoring. This study used mixed methods to investigate the experiences and perceptions of cheating in open-book online proctored exams. An online questionnaire and interviews were conducted with students and academics. Data analysis revealed that the experience of online proctored exams was generally positive although there were mixed preferences for online versus on-campus exams. A variety of advantages and disadvantages of online proctored exams were also identified. Whilst both students and academics reported that they believed students would cheat, actual instances of cheating (as reported by students or academics) were minimal. This may have been because of the use of open-book exams. Based on these findings we comment on the reality versus the rhetoric relating to online proctored exams and suggest a range of ways forward for universities.

Online exam remote proctored exam student experience cheating academic integrity

Previous article

View the issue table of contents

Remote proctoring of exams is one of the most divisive issues in higher education. In this type of assessment, students undertake an online examination on a computer of their choosing, at a location of their choosing, while they are monitored by a third party (Raman et al., Citation 2021). Critiques of remote proctoring abound, from a variety of perspectives. Remote proctoring is said to be an invasion of student privacy (Langenfeld, Citation2020); racist or ableist (Logan, Citation2020); a ‘cop shit’ approach to assessment (Moro, Citation2020); or rooted in a culture of distrust (Logan, Citation2020). The problem is that some critiques are posed by researchers or commentators who have little first-hand experience of remote proctoring, and little to empirical data in support of their arguments. The voices of the students and educators who are experiencing remote proctoring are somewhat absent in the literature. This paper seeks to understand experiences of remote proctoring for educators and students, to help disentangle the rhetoric from what actually happens in practice. In doing so, we hope to provide a more nuanced and grounded representation of some of the pros and cons of remote proctored exams.

Whilst universities may have utilized proctored online exams prior to COVID-19, the use of this method of assessment became much more common during the pandemic, particularly in contexts where there were widespread, extended lockdowns. Post-COVID-19, it appears that many universities have made the decision to retain remote proctoring (Jha, Citation 2022), although debate and discussion regarding the efficacy and ethics of this approach abound. Within the research literature online proctored exams are a relatively new phenomenon emerging in the early 2000s (Kitahara & Westfall, Citation2007). Since then, technology has rapidly expanded (Flaherty, Citation2020). Undertaking exams in an online environment is a very different experience for students when compared with sitting in-person, paper-based exams. Whilst online proctored exams might be considered advantageous by some students, others may experience difficulty finding a safe, quiet space to undertake an exam, or may not have the equipment or internet access that makes sitting the exam easy (Coghlan et al., Citation 2021). There may also be concerns regarding privacy (Selwyn et al., Citation2021) and academic integrity (Dawson, Citation2020) in these environments, given that supervision is typically undertaken via webcam using either live proctoring or a video recording of the student (Coghlan et al., Citation2021).

Kharbat and Daabes (Citation2021) have published one of the few post-COVID studies exploring students’ attitudes and concerns towards proctoring. Using surveys (n = 106), exam results (n = 106), and focus groups they found that students were typically not in favor of proctoring and had privacy concerns. Lilley et al. (Citation2016) conducted a pilot study with 17 students enrolled in online courses. They found that participants had concerns about data security, privacy, and the intrusiveness of live invigilation prior to undertaking the assessment but that this concern eased after having completed the assessment. James (Citation2016) gave students in a first-year psychology unit the choice of sitting an online invigilated exam, with 6.3% (n = 29) of the cohort accepting the opportunity. Participants were challenged by the idea of sitting an online exam, with technical difficulties and timely support identified as two of the key challenges. Karim et al. (Citation2014) randomly assigned 295 participants to either proctored or unproctored online tests and found that proctoring was associated with increased negative reactions. Participants reported concerns about invasion of privacy and feeling self-conscious about the information collected via proctoring. Bedford et al. (Citation2011) surveyed 31 students and 20 academics and found that overall, they held positive perceptions of online proctoring and that assessment results were not negatively impacted by proctoring. Selwyn et al. (Citation2021) undertook interviews with students who had sat online proctored exams, activists who led campaigns opposing online proctored exams, and academic and technical staff. Whilst online proctoring was seen to address a number of risks for universities, concern was raised regarding the surrender of control to commercial proctoring companies, the hidden workload for academics, and the degree to which students were able to exercise informed consent to online proctoring.

In an ethical analysis of online proctoring, Coghlan et al. (Citation2021) proposed that whilst online proctoring is not a ‘completely evil technology’ (p. 1583) its use should be thoughtfully considered by universities. Their philosophical analysis suggests a number of questions that universities should consider prior to using online proctoring such as: whether there are alternative forms of assessment that are acceptable and whether academic integrity would be impacted to an unacceptable level if proctoring was not in place.

In regard to student performance in online proctored exams, Alessio et al. (Citation2017) compared the academic performance of 147 students enrolled in a medical terminology course. Student results were compared for proctored and unproctored online assessments. They found that students sitting proctored assessments scored an average of 17 points lower on assessments and used significantly less time in completing assessments. However, Hylton et al. (Citation 2016) found no statistically significant difference in scores when comparing the performance of 186 students sitting the same exam who were either proctored or unproctored.

The primary claimed affordance provided by remote proctoring – above those provided by online exams in general – is that they can detect or deter cheating (D’Souza & Siegfeldt, Citation2017). However, literature relating to cheating in online proctored exams is sparse. Karim et al. (Citation2014) indirectly determined levels of cheating by comparing student results for online proctored and unproctored tests. They suggested that proctoring did decrease cheating although the effect size was small. Participants undertaking examinations in Bedford et al.’s (Citation2011) study perceived that students would be less likely to cheat when online proctoring was in use. Harmon and Lambrinos (Citation2008) compared proctored and unproctored exams in two economics courses and reported that cheating was taking place in unproctored exams. In Citation2014, Fask and colleagues similarly compared student performance in traditional proctored versus unproctored online exams for 44 students studying statistics. They found that the disadvantages of sitting online exams somewhat offset the opportunities to cheat afforded by an unproctored environment. Janke et al. (Citation2021) investigated cheating in a cohort of more than 1600 German students who sat closed-book online proctored exams. The students reported that they cheated more in these exams than in on-campus exams suggesting that academic integrity was reduced. Whilst these studies provide some empirical data, there is a need for further research exploring the rhetoric versus reality of online proctored exams.

This paper presents findings from one Australian university’s experience in using online open-book proctored exams. More specifically, the paper explores (i) student and academic preferences for online or located exams; (ii) the perceived advantages and disadvantages and the overall experience of online proctored exams for students and academics; and (iii) student and academic perceptions of cheating and academic integrity in online proctored exams. The paper concludes by discussing the key implications of the study and identifying areas for further consideration. These evidence-based insights may help to unravel the reality versus rhetoric in this space and will have implications for other Australian higher education institutions considering the use of online proctored exams as part of assessment offerings.

Research approach

This research project adopted a mixed-methods approach using online questionnaires and in-depth interviews. Mixed-methods research involves the collection of both qualitative and quantitative data within a single study. In combining qualitative and quantitative research approaches, a mixed-methods study provides strengths that offset the limitations of each research approach (Johnson & Onwuegbuzie, Citation2004). In this study, the collection of quantitative data via questionnaire occurred first. This data reflected the most common responses and experiences of the larger group of participants. Following this, qualitative data via interviews were collected allowing for a deeper exploration of the perceptions and beliefs of academics and students in relation to experiences and academic integrity in online proctored exams (Guest et al., Citation2012). Ethical approval for the study was granted by the Deakin University Human Ethics Committee (HAE-20-143).

In the second half of 2020, students who were taking part in online open-book proctored exams at an Australian university as part of the response to COVID-19 lockdowns were invited to participate in the study. The online proctored exams involved students undertaking the examination on a computer of their choosing, at a location of their choosing, while they were monitored/proctored by a third party. The specific type of remote proctoring that was used was provided by an external proctoring company using recorded proctoring via student web-cam. Artificial intelligence was used by the company to detect potential incidences of cheating. At this time the use of online proctored exams was relatively new to the university with only small-scale pilots previously undertaken. Students were undertaking qualifications in the disciplines of business, law, science, engineering, built environment, and health. Exams were on various topics relating to their disciplines. To assist with anonymity, in the hope of getting honest answers relating to cheating, students were not asked within the survey or interview to identify their discipline. This limits the conclusions relating to disciplinary differences that can be drawn from this study. Academics who had management and teaching responsibility in units with an online proctored exam in Trimester 2, 2020 were also recruited.

Five hundred and eighty-six students were invited to participate in the study. Four hundred and eighty-one students in this group consented to their answers to the questionnaire being used. Twenty-three students consented to participate in interviews. Twenty-three academics were also invited to participate. Thirteen of these staff consented to their answers to the questionnaire being used in this study; five academics participated in interviews. Demographic data was not collected from participants to encourage participation, given that the questionnaire and interviews included questions about cheating in exams.

Following the delivery of online proctored exams, both participant groups were emailed by an Administrator who was independent of the research project, to request participation in the study. A plain language statement and a link to the questionnaire were included. Consent was indicated by ticking a box within the questionnaire to provide consent for responses to be used for the purposes of research. At the end of the questionnaire participants were asked to indicate whether they were willing to participate in an in-depth interview. Academics managing the units were also asked to post an invitation to participate in the study on their online unit sites.

Data were collected using online questionnaires and in-depth interviews; students and academics were invited to participate in both data collections. The online questionnaire was developed by a team of academic and professional staff who were trialing the exams and had been modified following use in two previous years. The questionnaire was administered via Qualtrics. For students, the questionnaire covered various aspects of taking an online proctored exam such as setting up and using the online exam technology, internet connectivity, online proctoring experience, security and cheating, and preferences for taking exams in the future. For academics, the questionnaire covered perceptions on the appropriateness of online exams as an assessment approach as well as the student experience, and preferences for online or on-campus exams in the future. Both questionnaires included Likert scales, percentage scales, yes/no and open response questions. Interviews were conducted by two of the researchers via Zoom and were recorded and transcribed verbatim. The interview protocol was designed to elicit participants’ experiences and perceptions of online proctored exams with particular attention to security and cheating (see Supplementary Information). Students who participated were provided with an incentive in the form of a gift voucher.

To enhance rigour and trustworthiness, multiple data sources (questionnaires and interviews) were used, students and academics participated and all three researchers individually reviewed the data. The quantitative findings and the qualitative data were considered together when establishing study findings (Onwuegbuzie & Leech, Citation2006). The qualitative data were considered the primary data set, the quantitative data were used to provide context, and refinement where appropriate, to the story of the qualitative data. Quantitative data were taken to be ordinal and were analyzed using descriptive statistics in Qualtrics calculating percentages, means, and standard deviations for responses to questionnaire questions. Qualitative data were analysed using inductive thematic analysis. Each researcher initially familiarised themselves with the data and established themes, generating initial codes (Braun & Clarke, Citation2006). The researchers then came together on three occasions to compare their findings and to name and define themes; differences in opinion were discussed until agreement was reached (Braun & Clarke, Citation2006). When conducting thematic analysis, we sought to establish semantic rather than latent themes, as we were more interested in what our participants explicitly said and wrote rather than in identifying unstated or underlying meanings (Braun & Clarke, Citation2006).

The analysis of the combined qualitative and quantitative data (see Table 1 for quantitative data summary) led to the articulation of two core inter-connected themes and a range of sub-themes. Theme one was the online exam experience, with sub-themes of preference for the type of exam, benefits, and challenges of online exams, and change in perception. Theme two was cheating, with sub-themes of: predictions of cheating, impact of peers cheating, accidental or deliberate cheating and open book exams and cheating. Together the themes and sub-themes reveal the tensions between reality and rhetoric of online proctored exams as experienced by staff and students at one Australian university. A description of each theme and sub-theme follows, together with illustrative quotations that were derived from research interviews. Where quotations are shown, the corresponding participant number is provided at the end of each statement.

Table 1. Quantitative results for relevant questions – students and academics.

Download CSVDisplay Table

Online exam experience

Preference for type of exam

While the academics who participated in the online questionnaire had a strong preference for online proctored exams instead of a paper-based exam (69%), students were more divided on their preferences, with 37% preferring a paper exam, 39% preferring online exam, and 24% neutral on exam format. The following student quotation from the in-depth interviews illustrates why some students preferred the online format: ‘With everything else I have to fit in, I’d prefer to just be home, lock myself in, do the exams and then go right back into it [everyday life]. Logistically it’s much, much easier to do’ (Student 16).

Reasons for preferring the paper based or exam hall approach included taking the exam more seriously, and confidence that any cheating issues would be dealt with on the day as revealed in the following student quotation:

I would say I’m old-school so I would prefer a physical exam. Because it’s like written down on paper so you know what you have written, instead of like digitized you’re not sure whether you handed it in or like the info that you copied was all that you wanted. (Student 20)

Regardless of preference for online or paper-based, most students and academics interviewed spoke of the benefits of online proctored exams including decreased stress and distraction, not having to travel anywhere, being able to use own computer, having own choice of location, and being able to fit exams around other commitments. The following quotation supports this point: ‘You didn’t have the stress of, “am I going to be late, am I going to get there on time, am I going to find a car park”, kind of thing. So that was convenient (Student 11).

Students also spoke of the challenges faced with online exams, including work and childcare being harder to navigate: ‘Because I could take the supervised exam at my home location, it became a little bit more challenging to try and navigate with things like work or childcare’ (Student 19).