Insights into Durham students’ digital experience

Students with digital devicesAhead of the release of the 2018 Durham Student Digital Experience Tracker survey results, Malcolm and Candace were invited to speak about how we used the Tracker at Jisc’s Connect More event in Newcastle on 10 July 2018.

Our presentation was part of a session entitled How are students’ expectations and experiences of their digital environment changing? and we highlighted how different data analysis techniques revealed some key findings from the Tracker.

Participation

Open to first- and second-year undergraduates and taught Master’s students, the survey received 877 responses. This was a representative number for this group (with a confidence level of 95% and a confidence interval of 3.19), and the data reveal that this was largely due to two emails that went out while the survey was open.

Graph showing responses peaked when emails were sent

Frequencies

SPSS was used to analyse the quantitative data. Even simple frequency tables were useful in gauging student attitude toward digital. For example, here is how students responded when asked ‘How much would you like digital technologies to be used on your course?’

44% said more, 53% said same, 3% said less

Custom questions

The Tracker allowed each institution to create its own questions as well, and we asked students about online assessment. This revealed that the majority of students liked almost all types of online assessment, although some types are rarely used at Durham.

Of students who submitted the following online: Essay: 80% liked, 6% didn't; Presentation 25% liked, 3% didn't; Portfolio: 10% liked, 3% didn't; Audio: 5% liked, 3% didn't; Video: 6% liked, 2% didn't; Quiz: 17% liked, 3% didn't; Other: 11% liked, 2% didn't

This question also provided an unanticipated insight into the types of assessment that students expect: many students who had not submitted an assessment online considered it inappropriate to their subject.

Results in this order: No but would like to, No and wouldn't like to, No not appropriate to my subject. Essay: 4%, 2%, 8%; Presentation: 20%, 16%, 8%; Portfolio: 18%, 8%, 60%; Audio: 13%, 16%, 63%; Video: 15%, 16%, 62%; Quiz: 14%, 10%, 56%; Other: 10%, 7%, 70%

Benchmarking

The Tracker survey allowed us to benchmark student responses against last year’s Durham survey and against this year’s responses from all participating universities and from Russell Group institutions. Significance tests revealed where Durham was ahead or behind for every multiple-choice and Likert question in the Tracker. For example, when asked whether they had access to reliable wifi whenever they needed it, Durham students were significantly less positive than last year, but more positive than students at other UK universities.

90% at DU in 2017, 86% at DU in 2018, 83% at Russell Group in 2018, 82% in all UK HEIs in 2018

Correlations for composites

SPSS made it easy to get overall impressions of student responses. For example, if multiple questions that measured a similar attitude were all strongly correlated (p < .01), they could be combined into a single composite to represent that attitude generally.

Composite answers to 'When digital technologies are used on my course...' questions were 42% positive, 51% neutral, 7% negative

Free-text analysis

We used Nvivo to analyse the free-text responses, coding over 2,500 discrete comments into cascading categories. This was extremely useful in drawing out individual student experience narratives and in quantifying trends.

consider embedding digital tools more in lecture and seminar content PGT in Social Science & Health; Embrace digital learning as much as you already do First-year in Arts & Humanities; often time is spent inefficiently using digital teaching First-year in Science

 

The most popular free-response topics could then be mapped back onto student responses in SPSS, allowing for correlations between multiple-choice responses and free-text responses to be discovered. A striking example is the strong correlation between attitude toward the use of digital technologies on courses (as shown above) and free-text mentions of in-class polling as a useful digital activity. Those who did not mention in-class polling as useful…

Of those who didn't mention in-class polling, attitude was: 40% positive, 52% neutral, 7% negative

…had a significantly less positive attitude toward technology use in courses than those who did.

Of those who did mention polling, 62% positive, 37% neutral, 1% negative

Dissemination

Look out for the full Durham report, which will be available on this site soon! Stakeholders across the University will also be approached to discuss deeper analysis that would be particularly useful to them.

Jisc’s analysis of the data from all participating institutions was published today on the Digital Experience Insights site.