This White Paper will review the use of Professional Discussion (PD) as an assessment method and will include an insight into the findings of published academic research, in particular a study by Ahmed, Pollitt & Rose (2009).

What is Professional Discussion?

This use of PDs has become increasingly significant in End-Point Assessment (EPA) either as a discrete component with dedicated criteria and graded in its own right, or as with other vocational and technical qualifications, often used to fill any gaps in Knowledge, Skills and Behaviours (KSBs) from other components and assessment methods, for example, practical demonstrations or the portfolio of evidence.

There is a clear benefit to apprentices in being able to respond to the demands of this assessment method and being fully prepared. Stakes have increased to an even higher level, as in many newer versions of apprenticeship standards, assessment methods such as PD operate on a discrete basis.

The Institute for Apprenticeships and Technical Education (IfATE) defines professional discussion on their website as “a two-way discussion between an independent assessor and an apprentice to assess the apprentice’s in-depth understanding of their work”. In terms of the actual depth of understanding required to be reached during a PD assessment, the table below (adapted from the RQF framework) offers some example indicative responses and expectations across the different levels as a potential guide (though of course grading descriptors published in assessment plans must be followed):

What is Validity?

Validity is a much-researched concept, it comes in many forms and can be evidenced in many different ways and following the advice to leave the academic conversation behind from Newton & Shaw (2013), as fascinating as that is, might be the best option here. Instead, they suggest taking validity to be an “all-encompassing concept of quality, to be judged in relation to measurement aims, decision-making aims, and broader policy aims, respectively”.

So what does this mean in reality? PAL takes very seriously our responsibility to protect the validity of all our assessments, including PD. Although it is widely accepted that there is no perfect way to assess, we ensure our independent assessors are well trained and supported and possess the necessary levels of experience and competence to help ensure all assessments are unaffected as far as possible by irrelevant factors.

Six Golden Rules

Apprentices can also play their part in ensuring that assessments run as they should. We would like to suggest the following Golden Rules for demonstrating their skills, knowledge, and behaviours:

1. Apprentices should practice and take part in mock professional discussions as much as possible.
2. Apprentices should record mock professional discussions and watch and listen back to review their performance.
3. Apprentices should follow all available best practice advice on presenting ideas, giving evidence, and communicating clearly.
4. Apprentices should remember that the professional discussion is a two-way conversation and to lead and contribute as much as possible.
5. Apprentices should pick out their best evidence and focus on “I” examples rather than talking about collective or team achievements.
6. Apprentices should use their notes and a well-structured portfolio, where required, to give confidence and a strong grounding for their answers.

Threats to Validity

There are a number of threats to validity with oral assessment laid out by Ahmed, Pollitt & Rose (2009).

• The first issue facing apprentices in a PD assessment is that they may need to focus cognitively on coping with stress rather than answering questions and fully taking part in an in-depth two-way discussion.
• Secondly, the Independent Assessor (IA) asking the questions may guide or prompt more than is allowed or necessary.
• Thirdly, IAs may vary their script from apprentice to apprentice, asking some for extra information but not others, and therefore introducing the risk of variability.
• Finally, bias, as ever, must be considered and this can transpire in a number of different ways. To this end, End-Point Assessment Organisations (EPAOs) and IAs must be self-aware and consider and mitigate the potential for bias, unconscious or otherwise, across gender and age and so forth.

To mitigate these threats PAL’s independent assessors use all their experience and incorporate the guidance and training that we provide. These mitigations start from the planning meeting and continue until all assessment methods have been delivered and beyond. PAL’s assessments are rigorously monitored and standardised and we have a clear process should any enquiries about results or appeals need to be made by customers.

Recognising Diversity

With these above factors, we think it is important to acknowledge that all apprentices are different and will need different kinds of interaction during the two-way sessions.

• Our IAs will do everything in their power to put our apprentices at their ease, by introducing themselves and providing a full briefing on the assessment process. This support they provide is enabled by the knowledge of the sector and apprentice demographics that is possessed by our IAs.
• PAL IAs will work to build a rapport and professional relationship from the outset, which they do by reminding apprentices during the upcoming PD to showcase their accomplishments and also focus on how the apprentice has contributed individually to the organisation rather than talking about any wider team activities.
• Our IAs are clear and transparent on assessment timings, they always listen carefully to apprentices and show an interest in the answers given.

Cognitive Demand

Cognitive demand is an important element of any assessment and we always need to be aware of the factors that might impact on the performance of an apprentice outside the assessable areas of their knowledge and ability. To take a step back, cognition has been defined as the mental processes that underpin thinking and knowing and includes a number of types of processes for example attention, learning, memory, and use of language (Comer, Gould, & Furnham, 2013). In order to help us understand the demands placed on cognitive processes faced by apprentices, Ahmed, Pollitt & Rose (2009) provide a number of factors that need to be considered with oral assessment and therefore professional discussions.

The Positives in Connection to Cognitive Demand

On the positive side, there is no need for an apprentice to translate thoughts into the written word, output is direct and instantaneous. Also, attention is directed by the IA to some extent, so apprentices do not have to control their own levels of attention as with written assessments, so more brain processing can be used to focus on providing evidence of how the KSBs being assessed have been met. This could be because less conscious effort overall may be needed than with written assessments; this could be important because Kahneman (1973) suggested that the attention capacity of a person is limited and they can only give attention to multiple tasks if total capacity is not exceeded.

How to Counter the Negatives in Connection to Cognitive Demand

On the negative side of cognitive demand, nerves may affect performance in PD more than other methods of assessment and regularly taking part in mock assessments and practising their PD technique will likely be key in helping apprentices cope better. Our providers are very much aware of this and will use best practice methods of formative assessment and utilise the giving of feedback to develop their apprentices further. Time management also appears to be a factor, an apprentice may have less control than with written assessments with less opportunity to go back and change answers possibly and less opportunity to leave answers blank and return to answer after further thought.

A big part of PD is bringing practical experience together with a verbal explanation, and in doing this we understand that apprentices being nervous is natural. Apprentices need to remember that this is not an interrogation but instead, a great opportunity to present evidence of competency and how the KSBs have been met.

The best way to reduce an apprentice’s anxiety or shyness in respect of assessment is to provide straightforward information regarding the nature and purpose of the assessment, alongside the provision of helpful resources and guides that providers, employers and apprentices can access to ensure they are familiar with the different assessment methods and are prepared as much as possible. PAL‘s approach to account management, stakeholder communication, development of resources, and robust and inclusive assessment planning gives us confidence that these mechanisms go a long way to making the professional discussion a positive rather than harrowing experience for the apprentice.

Paul Kelly is PAL’s Qualifications Director

References

Ahmed, A., Pollitt, A. & Rose, L., 2009. Should Oral Assessment be Used More Often? University of Cambridge Local Examinations Syndicate.

Comer, R. J., Gould, E., & Furnham, A. (2013). Psychology. Wiley.

Institute for Apprenticeships and Technical Education. 2021. Assessment methods and grading descriptors. [online] Available at: <https://www.instituteforapprenticeships.org/developing-new-apprenticeships/resources/assessment-methods-and-grading-descriptors> [Accessed 23 July 2021].

Kahneman D. (1973). Attention and Effort. Englewood Cliffs, NJ: Prentice-Hal

Newton, P. E., & Shaw, S. D. (2013). Standards for talking and thinking about validity. Psychological Methods, 18(3), 301–3.