In November 2022, we worked with the Digital Learning Manager for HSS to recruit eight (8) students to work as Student Digital Champions (SDCs) to get their input into some of our learning technologies.
We specifically looked at the following two topics as part of a pilot project:
1. Q-Review captions and transcripts2. Technology Enhanced Learning (TEL) Student Inductions
Students were selected from a variety of schools within the faculty, as well as from interdisciplinary programmes to ensure that the student voice was broadly representative.
The Student Digital Champions worked for a total of 71 hours, an average of 9 total hours per student, over a 6-month period ending in May 2023.
We learned so much from the students, and plan to recruit more students in the future to get their input into our TEL resources.
Mini Project 1: Q-Review Captions
Mini Project 2: TEL Student Induction
Q-Review is the University’s lecture capture system. Upon recordings being published, automatic speech recognition (ASR) generated transcripts and captions are made available to the students, typically 90-93% accurate. These transcripts and captions are not reviewed or edited before being released and data is not available to us to determine their use. Captions can be edited in the Q-Review system and roles could be configured so that users only made changes to the content being reviewed (with no access to any other content or data).
15 courses were selected for a pilot, across a variety of schools and of differing subject matter, with the aim to:
Modules where students might benefit from transcripts/captions being made verbatim were chosen – based on the number of students enrolled on the module and those modules where students enrolled were also registered with DDS.
Upon the completion of the captioning process and collation of student feedback, the SDCs contributed to an evaluation session in which we reviewed the data, talked through the project methodology and the students provided feedback and shared their experiences on the project as a whole.
Of the HSS students surveyed, 80% stated that they utilise Q-Review recordings occasionally or frequently and of those that did, 80% stated that they utilise the captions.
Only 10% of respondents stated that they use the captions or transcripts for accessibility purposes, with the majority using captions for note taking (67%) or further understanding material / terminology (63%).
81% of respondents commented that they found ASR captions either somewhat or very helpful, 71% said that they found the transcript/caption accuracy as somewhat or very accurate.
The Q-Review captions’ accuracy was identified as being an issue, with the following factors listed as being challenges:
1. Subject matter - significant differences in the volume of edits required, depending on language used in taught content (e.g. lecture discussing folklore was audible, but context required).2. Teachers accents - the captions pick up incorrect words. 3. Terminology related to the topic / subject is not understood by the captioning tool (and keywords not made available on the slides). 4. Audio not captured sufficiently through the microphone array.5. Low volume levels, which was often a result of the following factors:
Updating of the captions in the Q-Review system typically took between 1.5-4x the duration of the recording. (e.g. a 1hr recording took between 90-240 mins to make verbatim), with approximately 10,000 words of content per hour of teaching for lectures.
The SDCs reported that they were able to make the edits easily, but that familiarising themselves with the terminology or having to re-listen to parts where audio was not clear, resulted in edits taking longer.
SDCs reported that when editing captions for subjects that they were studying, they found the exercise assisted them with their understanding of the content, however without having attended the lectures they might not have been able to caption the material as completely.
Captions are now a part of a student's life - in other areas outside of University as well, including: TikTok, YouTube and other video platforms, with c. 80% using captions by default.
Many students turn captions on by default on Q-Review recordings, even if they don't have any specific accessibility needs.
Students have come to rely on captions, and this is especially useful when revising for exams or completing assignments - as they can search the captions for keywords and jump to that section of the recording.
1. Caption edits to be carried out to make verbatim for modules where there are students with acute accessibility needs2. Glossary of terms should be provided with lecture slides/notes to aid student familiarity of terms and comprehension of captions3. Lapel mic / neck mic use to ensure best quality of audio and captions is made available to learners4. Reminder to students who are attending the lecture in-person to refrain from chatting for lecture recording quality purposes
Currently, the TEL team deliver approximately 40-50 induction sessions for new students during Welcome Week in early September and January each year. These student induction sessions are attended by over 1,500 new students to QM.
We discuss key TEL technology that students will use during their studies at QM, some of which include:
These inductions are 45-60 minutes in a lab on campus. There is also an online QMplus for students course area, where students can revisit what was discussed at the induction and go through additional information.
The induction sessions have been developed and delivered by members of the TEL team in the same general format for over 10 years.
Student Digital Champions were tasked with:
The students put together a survey and received 160 reponses.
Of the 160 respondents, 59% indicated that they attended a TEL student induction, with the remaining 41% indicating that they either did not attend, or do not remember whether or not they attended (25% and 16% respectively).
When asked whether the induction sessions were useful or not 82% of students replied that yes, the induction talks were useful.
Student induction usefulness
The aspects highlighted as most helpful included the following:
Everything was explored by using a step by step process. While they were showing us the steps, we got a chance to do it at the same time which was practical.
Time to try it out for ourselves with staff available to support.
Visual examples and good explanations.
The recording feature as well as the transcript option seemed most useful to me.
How helpful both instructors were, they reassured me because I was very confused about how it works.
Simplicity of the induction talk to not overwhelm the audience.
As for the reasons why students did not find the induction talks useful, there were the following comments:
It was a long session for a straightforward process.
Pretty self-explanatory on the website itself.
It felt rushed due to the limited time slot the session ran, and I had to learn to navigate through different sections of QMplus as I was doing the course.
I do not take in information well that way.
Students were asked to indicate if they had any difficulties with the learning technologies at QM.
The results were as shown:
Positively, the most popular answer was no difficulties but as seen in the above chart, the aspect that most struggled with was accessing grades and feedback via QMPlus, closely followed by navigating QMplus.
Other learning technologies that students cited as having difficulties with included:
1. MySIS2. The WiFi3. Module pages being difficult to navigate4. Printing services5. Q-Review - incorrect transcription6. Attendance records for each course7. Reading material difficult to access in VLE books through QMplus8. Library website is confusing9. Difficulties over which username / password to use for different areas10. Accessing MySIS account11. Issues trying to download alternative formats of PDFs / documents uploaded to QMplus
Where students have gone to find help:
Where students seek help on learning technologies
Students were asked to make suggestions to help to improve student induction to QMplus and other learning technologies. The following feedback was added in the open comments:
Due to the success of the project and the feedback received from the student digital champions about the TEL student inductions, the TEL Team have now secured funding to recruit 3 students to be a part of a summer project to revamp the TEL student inductions. We will recruit a further 5 students, making a total of 8, to co-deliver the TEL student inductions to new students in September 2023.
This will be the first time that students have been involved with co-creating the student induction content for TEL applications, and the first time that they will be tasked with delivering the TEL student inductions. New QM students will be able to hear first-hand from more senior students about our technologies along with some tips such as ‘what they wish they had known when they were new QM students’.
Three students have been recruited and will begin work on updating TEL student induction in June 2023.
------------------------
This report was written by:
For media information, contact: