‘GenAIrating’ Critical Clinical Thinking: A new approach to learning using large language models

This project is funded by the President and Principal’s Fund for Educational Excellence.

Aims

To help ensure that clinical students are equipped with the critical clinical thinking skills they need to succeed in their future careers, we proposed addressing three linked issues that require innovation:

• The emergence of large language models (LLMs), with the opportunities and challenges they bring to learning and assessment.

  Large language models are a powerful tool that can be used to improve critical thinking skills. However, there is also a concern that LLMs could be used to cheat. For example, students could use LLMs to generate essays or answer questions on exams.

  We are aiming to address the issue of cheating by using LLMs in a responsible way. We are developing tools that will help students to use LLMs to improve their critical thinking skills, but we are also putting in place safeguards to prevent students from using LLMs to cheat.

• The need for improved critical thinking training, opportunities, and resources for clinical students.

  Critical thinking is essential for all medical professionals, not least for sport and exercise medicine (SEM) students. Students must be able to critically evaluate scientific evidence, make sound clinical decisions, practice evidence-based medicine and communicate effectively with patients and other healthcare professionals.

  • We need more specialised training in critical thinking: SEM students have limited opportunities to practice and engage in critical thinking outside of direct interaction in face-to-face sessions.
  • There is a call for more emphasis on critical thinking in the SEM curriculum that could be addressed by incorporating more critical thinking exercises, using novel approaches such as LLMs.
  • Developing more focused resources to support critical thinking development would give students the tools to learn about critical thinking and practice their critical thinking skills.
• The emergence of improved pedagogical practices of co-creation and simulated ‘learning by doing’. It is important to engage SEM students in learning experiences that are both challenging and relevant. This can be done through ‘learning by doing’ and co-creation. Although already major strengths of the current SEM curriculum delivery, there is the opportunity to update these in the light of the challenges of new technology such as generative AI.
  • Learning by doing: giving students more opportunities to apply their critical thinking skills to real-world problems, such as through case studies and simulations.
  • Co-creation: giving students opportunities to work with others to solve problems. This can be done through group projects, peer learning, and community-based learning. Integrating in reflective feedback would enhance this even further.

Opportunity

Objective Structured Clinical Examinations (OSCEs) are a core assessment strategy in all medical domains. The opportunity to demonstrate competency in communication and clinical reasoning is core to these. But outside of the supervised setting it can be very tough to practice these skills. A tool that gives students the ability to practice these interactions, where they are challenged to think critically, systematically, and communicate openly with empathy, would be indispensable. Being able to then reflect on this process, exploring the journey the interaction would be priceless.

We explored the use of Google Bard, using the following prompt:

It was not a very well thought out prompt, but we just went with it. We ended up chatting with a young individual who had recently been having pain in their right knee. Without any guidance as to the injury or mechanisms of injury, what came was a “consultation” that reflected what we see regularly in the clinical setting. The “patient” (they even called themselves Bard) was like any real-world patient. Bard demonstrated apprehension and frustration, even catastrophising at some points. They challenged us to be empathetic whilst focusing on the clarity of our message.

What also came out of it was a transcript of this consultation. Imagine a student being able to read back through this, discuss it with other students, even show it to their tutor, to explore self-reflection and critical appraisal of how they performed. This subsequently developed into the current project ‘GenAIrating’ Critical Clinical Thinking: A new approach to learning using large language models'. Alongside a student steering group, we are aiming to change how we develop critical clinical thinking skills.

Project deliverables

We are currently working towards producing four main outputs:

1. An enhanced strand of critical thinking in the SEM curriculum. This will incorporate more critical thinking exercises and resources, alongside addressing the issue of cheating with LLMs.
2. A new set of tools for using LLMs to improve critical thinking skills. These tools would help students to use LLMs to generate arguments, evaluate evidence, and make decisions.
3. Research article on the effectiveness of using LLMs to improve critical thinking skills. This study would help to determine whether LLMs are an effective way to improve critical thinking skills, and it would also identify any potential risks associated with using LLMs for this purpose.
4. Student led research project investigating the effectiveness of using LLMs to support patients. Patients regularly undertake online searches to find out more about their symptoms, both before and after consultations with clinicians. This student research project will compare and explore clinician and LLM responses to patient queries. In doing so, we will explore the potential for incorporating into critical thinking development.

We are currently focusing on clinical settings seen within the Sport and Exercise Medicine domains – simply because this is the domain, we work in. The potential for extrapolating what we develop to other domains is easy to see.
 
Any setting where a student - in their future profession - would interact with or consult another human being, whilst thinking critically and systematically, would be able to be recreated, providing potentially endless opportunities to practice and develop critical thinking and reflection.