Learner Diversity and Learning R

 

Learner Diversity and Learning Relationships: Factors Affecting the Quality of the Formative Feedback Process

 

Debra Boyask

Faculty of Environment & Technology

University of the West of England

Debra.Boyask@uwe.ac.uk

+44 (0)117 3283211

 

 

Abstract

 

In evaluating progress on change in a faculty-based assessment strategy, students and staff were consulted about what characterised effectiveness in formative feedback processes. In addition to the more mechanical qualities that are often cited as desirable, such as promptness, legibility and clarity, a strong emphasis on two issues was clear. The first was the diversity of students‘ aspirations and confidence, and the ways in which these differences could affect the kind of feedback students desired. The second issue was the bidirectional effects of effective formative feedback processes and strong learning and teaching relationships upon one another. Possible approaches to adapting feedback practice with regard to these emergent issues are discussed.

 

Keywords: assessment, formative feedback, feedback dialogue, learning relationships, social capital, learner aspiration

 

 

Introduction

 

In 2000 the Faculty of the Built Environment at the University of the West of England revised its approach to assessment to emphasise the importance of its formative function. The new strategy was based on principles of designing assessment into the curriculum in ways that ensured students’ efforts were most conducive to learning (Gibbs,1999). Across all programmes there was a clearer division between summative and formative assessment tasks. The summative work was minimised, with emphasis placed on providing early feedback-yielding formative work to better foster the development of knowledge and skills needed for summative tasks.

The importance of the provision of high quality formative feedback in enhancing student learning has been well-documented (Brockbank & McGill, 1998; Gibbs, Simpson & Macdonald, 2003; Nicol & Macfarlane-Dick, 2004; Swing, 2004; Yorke, 2003). Effective feedback processes can enable students to gain a variety of perspectives on what they are learning. They can provide the scaffolding needed to develop a deeper understanding of new or difficult concepts with which they are engaging. Participation in a feedback discourse can yield not only information to enable learners to better evaluate their own academic performance, but it encourages them to develop self-critical habits and scholarly independence.

 

We believed our revisions had been effective in increasing the quantity of formative feedback that students experience, but we needed to improve our understanding of its quality. Quality of formative feedback had been a common theme for staff development workshops in the Faculty, with an emphasis on doing a better job of giving feedback that students could make most effective use of (Brockbank & McGill, 1998). Despite this, we were aware that our students still considered there was room for improvement in this area. In reviewing the Faculty’s results for the 2005 National Student Survey, formative feedback was identified as the lowest scoring attribute. According to HEFCE’s report on the Survey, this was consistent with national data.  

 

While the NSS data suggests that students continue to be dissatisfied with feedback, the survey items associated with feedback cannot provide a great deal of depth as to students' perceptions of quality. The three relevant items cover promptness, degree of detail and the ability for feedback to provide clarity, however the items might suggest not only a limited range of criteria, but also a rather limited view of what counts as feedback.

 

Nicol's Principles of Good Assessment and Feedback Practice (Nicol, 2007) provides a fuller framework for consideration of quality of feedback. They indicate that feedback practices should: clarify learning objectives; encourage a focus on challenging tasks; and facilitate motivation, independence, self evaluation, learner choice and learning dialogues and communities. These goals indicate a more holistic relationship between feedback and learning.

 

When considering recent work on formative feedback, we became aware that the language we had used to talk about feedback was still reflecting the view that it was something teachers give to students. There is an increasing awareness in the HE community that effective formative feedback as part of a sound and holistic approach to learning is a two way process, and not simply something provided by staff. Nicol and MacFarlane-Dick (2006) suggested that in some respects formative assessment and feedback have lagged behind general shifts in thinking about learning and teaching. Discourse around accepted good practice in HE encompasses learner-centeredness, self-direction and active learning in which learners construct their own understanding. When it comes to feedback, we often fall back on a simple transmission model, talking in terms of feedback provision.

 

In carrying out our investigation, we hoped to increase our understanding of students’ experiences and perspectives on the nature and value of formative feedback processes and how they engaged with them, as well as simply finding out about the kinds of formative activities happening in the Faculty. In this report, the focus is on factors affecting the quality of formative feedback processes mainly from the students’ point of view, although staff perspectives have also been considered.

 

 

Project objectives

 

The objective of this project was to gather information directly from students and staff across the Faculty via group interviews to determine:

 

  • the range of feedback activities that were happening in the Faculty
  • how staff and students viewed their experiences of processes of formative feedback

and

  • what they considered to be the characteristics of effective formative feedback

 

  

Methods

 

Student interviews

 

Eleven of the Faculty's undergraduate programmes were selected, with the intention of ensuring a good coverage of the variety of programme types provided. From each programme, focus groups of between six and eight students were selected to discuss their thoughts about and experiences of formative feedback. Most groups were second year students (where this was possible), with one first year and two third year groups. Most participants were fulltime attending students; two groups were either part time attending students, or enrolled on a course that partially involved attending and was partially studied at a distance.  Sampling was not random; participants were asked to volunteer.

 

Focus group discussions took between 30 and 50 minutes, and were semi-structured, facilitated around a set of broad questions designed to determine:

 

  • the range of formative approaches students thought they had been involved in - (What different kinds of formative feedback have you been the recipient of?)
  • how helpful and usable they thought various feedback processes they had experienced were – (What kinds of feedback have you found helpful? What kinds aren't as helpful? How do you use feedback?)

and

  •  on what basis they evaluated the quality of feedback – (What are the characteristics of good feedback?)

 

The interviewer for all student groups was the Faculty’s educational developer, and no participants were in classes taught or assessed by the interviewer. Students were assured that data from discrete groups would only be accessible to the interviewer, and only the collated results from all groups would be reported.

 

Staff interviews

 

Teaching staff from across the Faculty were asked to participate in a group discussion during a professional development day. All who attended were included. There were four groups, each comprising between ten and fifteen participants.

 

As for the student groups interviews, the discussions took between 30 and 50 minutes, and the question set was designed to determine:

 

  • the range of formative processes staff had been introducing to modules – (What opportunities for formative feedback do you structure into your module/s? How do you encourage students to elicit the feedback they need? How do you encourage students to make effective use of formative feedback?)
  • staff views about what made feedback processes effective – (What, in your opinion, are the characteristics of effective and useful feedback?)

and

  • whether staff had noted an impact of the policy change -  (What impact, if any, have you observed from any particular feedback strategy you have introduced?)

 

The four staff group discussions were conducted simultaneously, and each group was facilitated by a peer of the participants who volunteered for the task. All facilitators were briefed by the Faculty’s educational developer prior to the discussion session.

 

 

Results from student interviews

 

Student discussions were spread across a term, and a grounded theoretical approach was taking to analysing data sets to draw out themes. Data sets from each session were coded according to the following emergent themes found to relate to students’ perceptions of quality: timing; consistency of feedback (across modules); clarity and ease of interpretation; modes of feedback; general vs. individual feedback; diversity of learners’ self perception and aspirations, and accessibility of and relationships between students and staff. The data from all student sessions was then grouped according to theme, and similar items identified in at least three of the eleven data sets were recorded.

 

Timing:

·        Sometimes feedback comes too late to be useful.

·        Getting feedback can rely on students' time management skills.

·        Some students feel lots of formative tasks are all due at the same time, making it difficult to complete formative work for every module.

·        Part-time and block course students have specific difficulties relating to the timing of feedback.

·        Timing can also create high pressure points for accessing staff for support (office hours are in more demand at specific points in the term).

 

Consistency across modules:

·        Most students felt that some modules did a really good job of providing feedback, whereas others provided far fewer opportunities.

 

Clarity/ease of interpreting feedback:

·        The usefulness of the feedback depends on how well the student can interpret it, or even read it (some tutors' handwriting is difficult to read).

·        There is a variety of discourses in marking and commentation. Markers use different protocols and notations of their own design.

·        Very general comments, such as 'good', 'fine', 'okay', 'think about this' or a line through something are puzzling. Students want to know why something is good or fine. They want to know if 'good' means merely adequate, or whether it means more.

·        Some students would prefer an indicative mark that indicates how well they are doing. Some staff are not comfortable with supplying indicative marks.

 

Modes of feedback:

·        Having an opportunity for a dialogue is much better than just being handed feedback; students need opportunities to get clarification.

·        Some students (especially part time) find it helpful to be able to get feedback via email. Having to receive it in person can cause problematic time delays.

 

General versus individual feedback:

·        Some students feel that general feedback has its place, but sometimes the very general points (not specific to the task) are raised over and over.

·        Some feel general feedback is aimed at weaker students; others feel there is little that is relevant to them in general feedback sessions.

·        Specific, individual feedback is considered a lot more valuable by most students.

 

Diversity of learners’ self-perception and aspirations:

·        Students have different needs in terms of feedback. Some are merely aiming to pass a module, while others want to excel. Some students need their confidence built up, while others feel feedback ought to be more critical. 

·        Some feel there might be a mismatch between comments and marks. They think perhaps teachers are afraid of being too critical, when really they would prefer more bluntness.

 

Accessibility to and relationships with staff:

·        There is an awareness that different staff members respond differently to different kinds of approaches. Students with initiative are good at identifying how best to approach individual staff members, but others can be mystified by the difference.

·        Some students are more confident than others about taking the initiative in eliciting feedback. For those who are not as confident, having a sense that their tutors are interested in them as individuals and in their learning is important.

·        Some staff make an effort to get to know students, by learning their names, talking to them during group work and even in the corridor. Students are more comfortable asking these lecturers for help and feedback.

 

 

Results from staff interviews

 

Staff discussions happened after the student discussions took place, and results were recorded in a more structured way by the four facilitators. Staff were asked to describe the ways in which they were attempting to incorporate formative feedback processes into their modules. Some approaches were more prescriptive in terms of building feedback opportunities into the course in a formal way. Others simply provided the invitation or suggested ways for students to seek feedback when required. Some approaches indicated a more active role for the teacher in terms of direct interaction with students, or at least with their work. Other approaches were more based around the provision of resources such as models and self-testing tools.

 

A summary of the approaches identified

 

Pre-submission hand-in:

  • of drafts, plans/outlines/synopses, proposals, literature reviews (for dissertation)
  • early stage submission of part of a longer project
  • early submission with an opportunity to improve and resubmit

 

Consultations:

  • small group or individual  tutorials prior to submission
  • invitations to seek one to one support/dialogue
  • group presentations on the broad area within which an essay topic was based
  • question and answer sessions in lectures/tutorials or via email

 

Resources for self-evaluation:

  • models and examples provided on module websites with instructions and criteria for self evaluation
  • diagnostic quiz - pre and post tests
  • online learning topics with activities to complete and submit with model answers to compare against

 

Activities in class that yield feedback:

  • group discussion work,
  • role-plays
  • weekly tutorial questions/model questions similar to coursework
  • group discussion around essay plans and drafts
  • lab based activities
  • peer assessment of drafts

 

The variety of formative feedback approaches reflects a range of impacting factors, such as type of module content, class size, level and organisation, as well as the particular views of teaching staff.

 

In all four groups, issues of accessibility and approachability of staff arose. There were different views about how this should be managed, with some staff feeling the need to be more active in seeking ways in which they could best become accessible to students while others focused on ensuring, simply, that information was available about how to access them.

 

Beyond access, some staff had things to say about ways in which they attempted to persuade or encourage students to take advantage of opportunities for feedback. Again, there was some variation in as much how active staff felt they should be, with some talking about 'offering' or 'inviting' students to have consultations or comments on work, while others considered there was a need to pursue students more actively, using a variety of means to persuade them to take feedback opportunities and help them develop more confidence and independence as learners.

 

 

Discussion

 

Analysis of data on student and staff experiences of feedback processes yielded some predictable themes. Not all issues raised by students, however, had been prioritised as highly by staff in their discussions. A number of the suggested quality indicators were ones with which we were already very familiar, and had been addressing through staff development, such as promptness, legibility and clarity in communicating feedback.  However, the considerations of diversity of learners’ self-perceptions and aspirations and the importance of learning and teaching relationships were areas in which we felt we could have a particular and further impact.

 

Timeliness

 

We knew that in order for it to be useful, feedback has to happen at a point when it can be best made use of (Brockbank & McGill, 1998). By ensuring there was time for formative feedback to happen well before the summative assessment processes, the Faculty had established a sound basis to enable this to happen. However, students raised some additional factors that needed consideration to further improve the timeliness of feedback processes in some cases.

 

Timely formative feedback processes rely, to some extent, on students' time management skills, and perhaps there is more we can do to support the development of these. Some student respondents suggested that they could be encouraged to evaluate their own time management skills, and get some support and planning tools within the context of their programmes if this was required. This is something we have been able to raise with programme staff as an area to consider for development.

 

Clarity and ease of interpretation

 

The usefulness of feedback depends on how well it can be interpreted. Students thought that written feedback was not always completely legible, and efforts should be taken to ensure students have the opportunity to check if there is something they can't understand.

 

Different tutors develop their own discourses and conventions in marking and commentating work. It's important to be aware that any shorthand notation teaching staff might develop may need to be demystified for students.

 

Comments that are very general, such as 'good', 'fine', 'okay' or 'no', might not be easy for students to interpret. They need to know precisely what is good or not good, and in what way. They also want to know if good merely means adequate, or if it is indicative of a higher standard. Some students expressed a strong preference for indicative marks that would give a quantifiable sense of how well they were doing, but some staff were uneasy about this apparent focus on quantitative measurement.  However, comments from staff did acknowledged the importance of giving students indications of performance against assessment criteria and other benchmarks or models.

 

Students need to have a clear understanding of module learning objectives and the criteria against which they are to be assessed (Brown & Knight, 2004). It was important to our student respondents that they knew what was expected of them, and what kinds of standards they need to be working to in order to achieve the results (marks) they aspired to. As well as encouraging staff to think carefully about the issues around clarity, building a dialogic process into programme strategies for formative feedback can facilitate opportunities for clarification.

 

For students to see clear links between general module learning objectives, specific criteria for formative work and the final assessment criteria, we have encouraged staff to make the process of how tutors work through evaluating work and allocating marks more transparent, in an endeavour to further demystify the process of assessment for them. Specifying and encouraging students to engage with assessment criteria is a part of this, but dialogic feedback processes also have an important role in helping students develop their understanding of what comprises quality in academic work (Nicol & Macfarlane-Dick, 2004).

 

General versus individual feedback

 

While they acknowledged that generic feedback to the group can have a useful place, the feedback that is most valued and likely to be used by students is personalised and specific to the individual. General feedback may often be reiterated across different modules, and some students felt that the generic kinds of issues covered are more relevant to weaker students; at times they might feel impatient about the amount of time used on aspects of knowledge and skills they believe they have already mastered.

 

The use of generic feedback is often justified by staff by the need to make best use of limited time, but we have suggested that more consideration is given to the process used for generic feedback to ensure it takes up a minimum of the whole group's time. Using online communication tools along with a data bank of common pointers for types of assessment activities for generic feedback could save time in class.

 

Diversity of needs

 

Aside from the obvious issues of differences in knowledge and skills amongst a group of students, their needs for feedback can vary from one another with regard to differences in confidence, self-perception and aspiration. Those with high levels of confidence in their ability to learn can find it easier to make use of critical comments, and in fact some of the more confident students felt frustrated by what they saw as a lack of rigour, thinking that their tutors were afraid of being too critical. On the other hand students who doubt their own abilities may be easily discouraged and sometimes felt tutors had been excessively tough on them

 

Students’ different aspirations mean some want to excel while others merely aim to pass. An individual student’s aspirations might vary for different modules of learning within the same programme.  We might wish that students were more intrinsically motivated, and of course we can endeavour to attract their interest in the curriculum for its own sake, but many students are very functionalist in their approach to their education.

 

Some students are more confident than others – they feel that they are able students and they can feel frustrated when they believe that their teachers are trying to ‘sugar the pill’ too much. They want their tutor to be more critical and not pull any punches. However, others feel they are easily discouraged; they are unsure of themselves and a firm critique might make them even less confident.

 

Students were very aware of a diversity of aspirations and confidence levels amongst their peers. They considered that these differences indicate differences in needs for encouragement and critique. They also knew that some students were better than others at soliciting feedback from staff.

 

While engagement at an individual level is not always possible in larger classes, there are ways we can enable students to provide more information about their aspirations and feedback needs so these can be addressed in ways that are more helpful to them. In a smaller group, this might involve asking them to tell us about what they'd like from their feedback and what they're hoping to achieve. In larger groups where it's not easy to have individual discussions, it could be useful to provide a format for students to indicate the mark they are aiming for and the level of critique they believe they would find useful. By providing structure around feedback dialogues, less is left to students’ feedback elicitation skills.

 

As well as providing more individualised information, such talking about feedback in this way can raise students' awareness about the feedback process, its purpose and how best to make use of it. Sometimes when students say they have had no or little feedback, they are not always aware of all the things that might feasibly count as feedback.  

 

Encouraging reflective self-evaluation through feedback dialogue can be useful in supporting students' development towards becoming increasingly independent as learners. By involving students in the decision making about forms of feedback, we can help students develop self efficacy in seeking feedback as well as in their self-evaluative skills, an important function of effective feedback (Nicol & Macfarlane-Dick, 2004).

 

Learning and teaching relationships

 

Students seemed to be most receptive to formative feedback when they felt that the person feeding back to them was someone who knew them and who they believed had an interest in their progress. Both students and teaching staff related ways in which staff endeavour to engage students in effective learning and teaching relationships. Students differed in their ability to engage in learning and teaching relationships, which suggests that the quality of the feedback process for an individual is likely to be at least partly dependent on factors such as social confidence and skills and perhaps social capital.

 

Students valued efforts made by staff to provide opportunities for consultation through office hours, or through individual email contact, however not all students had the confidence to take advantage of these opportunities. Some were also confused by the fact that the ways in which different staff were more easily contactable varied, while others could easily appreciate these differences and managed to gain access by working out individual staff members' preferred approach.

 

Teaching staff and students talked about strategies teachers use to develop relationships within which effective feedback processes can happen: they learn names, use names during group work times or even outside of class, show interest in students as individuals. We knew that these things were important generally, but what we’ve learnt is how important they are to students in facilitating feedback dialogue processes.

 

All of these measures contributed to students having a sense that they know their tutors and that their tutors know them and this made it more possible and comfortable for less confident students to feel they belonged to their programme, and to approach a staff member for assistance if it was needed.

 

While it seems that effective relationships are beneficial to effectiveness of feedback processes, there is evidence of a bidirectional relationship between these two factors; there are ways in which feedback has been used to facilitate more effective relationships. Some staff take a light-handed approach to engaging students in feedback dialogues, inviting them to ask for support. Others are more directing, structuring opportunities for dialogue between students and staff very specifically into modules.

 

The issue of relationships has emerged as a significant strand for this project, and sharing the findings with staff has resulted in valuable discussions around strategising for using feedback processes to develop stronger learning/teaching relationships in a more intentional way. In the context of increasing numbers and class sizes, we need to think creatively about ways of getting to know students. Many staff mentioned the value of field trips, visits and other activities outside of regular classes. Spending time in shared areas available for students work informally (such as computer rooms, design studios) was another approach that was found to be helpful.

 

The individual nature of learning relationships means that the quality of formative feedback processes students experience can depend to some extent on their own ability to forge relationships with their teachers. Students are aware that teachers respond differently to different kinds of approaches to soliciting feedback. Some students are more confident than others about initiating interactions with staff, and some are better than others about working out the best way to communicate with staff members. It is likely that maturity, social skills and social capital influence students’ ability to engage with feedback processes, and this is something we want to investigate further.

  

 

Conclusion

 

The project provided opportunities for staff to discuss the issues around formative feedback and possibilities for improvement of its quality, and share effective practice with peers. While some of the issues were predictable ones, the project raised our awareness of the importance of learning and teaching relationships for effective formative feedback, and also the ways in which feedback processes might be used to strengthen such relationships. We intend to carry out further work examining the nature of learning and teaching relationships and their impact on feedback experiences. We were also induced to plan to elicit more information about students’ specific feedback needs in terms of their aspirations and what they found most helpful.  

 

 

References

 

Brockbank, A. & McGill, I. (1998). Facilitating Reflective Learning in Higher Education. SRHE & Open University Press, Buckingham

 

Brown, S. & Knight, P. (2004). Assessing Learners in Higher Education. Oxon, RoutledgeFalmer    

 

Gibbs, G., Simpson, C. & Macdonald, R. (2003). Improving student learning through changing  assessment -- a conceptual and practical framework, at   EARLI Conference, Padova, 2003

 

Gibbs, G. (1999). ‘Using assessment strategically to change the way students learn’, In S. Brown & A. Glasner, (Eds.), Assessment Matters in Higher Education. SRHE/Open University Press.

 

The National Student Survey 2005. (2006). Available at: http://www.hefce.ac.uk/Pubs/rdreports/2006/rd22_06/ [Accessed 7th July 2008]

 

Nicol, D. (2007). Principles of good assessment and feedback: Theory and practice. From the REAP International Online Conference on Assessment Design for Learner Responsibility, 29th-31st    May, 2007. Available at http://ewds.strath.ac.uk/REAP07, [Accessed 1st July 2008]

 

Nicol, D. & Macfarlane-Dick, D. (2004). Rethinking Formative Assessment in HE: a theoretical model and    seven principles of good feedback practice. Conference presentation at Enhancing the Student Experience in Scottish Higher Education: Workshop 7, Glasgow, June 4, 2004.

 

Nicol, D. & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, Vol 31 (2), 199-218

 

Swing, R. (2004). Understanding the "Economies of Feedback": Balancing Supply and Demand, Keynote to 'Improving feedback to students' workshop held on 4 June 2004 at the St Andrew's Building,  University of Glasgow. Available at:

            http://www.scottishenhancement.ac.uk/uploads%5Cdocuments%5CSwingpaper-revised.pdf  [Accessed 6th October 2008]

 

Yorke, M. (2003). Formative assessment in higher education: Moves towards theory and the  enhancement of pedagogic practice. Higher Education 45, 477--501.

Refbacks

  • There are currently no refbacks.