The Role of Simulation in Training Endovascular Interventions

Psychomotor skills for tasks with an element of risk, such as those used to ride a bicycle, are best acquired in a low risk or risk free environment which allows the ascent of the learning curve to be conducted in safety. In this way, the essential skills to remain upright while pedalling and steering can be acquired through deliberate practice, becoming automated to allow attention to be directed to more complex tasks, such as the complexities of riding in traffic. Equally, if we have not cycled for a few years, we might wish to reacquaint ourselves with the essential fine motor skills required in a controlled environment. Yet although we know such skills exist, we would find it difficult to explain exactly what they might comprise!

Today, medical procedural skills are still usually learnt by practicing on patients in an apprenticeship. This also is the case for interventional radiology (IR), where it is just as important to avoid isolating the acquisition of motor skills from key knowledge and rules, in particular those that relate to the clinical condition of the patient. Therefore training is carried out in a broad, carefully structured curriculum, which also develops the behaviour and attitudes that are essential to professionalism.1 Basic procedural skills (cognitive, psychomotor) are the essential building blocks for more complex tasks. These ‘core’ elements include clinical, anatomical, imaging, therapeutic, as well as other cognitive, and the fine motor, skills to use touch and imaging to guide the manipulation of needles, wires and catheters. Hence these fundamental skills must be automated to avoid exceeding the learner's attention capacity when moving to more complex tasks in patients.2 Once acquired, these technical skills are maintained by regular practice, or may need to be refreshed by re-training.

The training and maintenance of core skills underpins IR procedural practice, yet a dearth of invasive diagnostic work in the wake of exponential developments in non-invasive imaging, has reduced the available opportunities for basic training. European, and other, working time regulations,3, 4 together with a schedule for modernizing medical careers, have further condensed the time available to train. In addition, it is a paradox that while we should ‘first do no harm’, an essential component of patient-centred learning is feedback on errors made. At the same time, apprenticeship training prolongs procedures and occupies expert mentors, resulting in more expensive patient management.5, 6 Complex procedures such as carotid stenting bring further problems: how is the trainee to learn without exposing the patient to risk? Further, how is the trainee to be assessed, to provide feedback to motivate learning and evidence for certification? As yet there is little in the way of objective assessment methods in routine use for IR skills. Therefore a pressing need for a reappraisal of our methods of training and assessing the high stakes skills of IR exists, with an emphasis on defining minimum standards for success, and implementing an alternative to patient-centred learning.

Possible alternatives include various simulations such as models, animals and computer based methods. Simple deformable models of anatomy have been used to train and assess in surgery,7 and these also can be punctured by needles under ultrasound guidance, or act as a conduit to train catheter and guide wire skills.8 Such models often are expensive, are ultimately destroyed by multiple needle punctures, and the materials used cannot be easily altered to change anatomy and ‘pathology’. Training also can use animals, which provide real world physiology and ‘feel’, although it is difficult to reproduce human pathology states in animals, and their anatomy is somewhat dissimilar to that of humans.9, 10 Animals also are expensive to maintain, and their use has raised political and ethical issues. Technology based simulation, on the other hand, introduces the potential to attain high levels of fidelity (accuracy) through use of an interface with a computer generated virtual environment.

The question of whether medical simulation facilitates learning was the subject of a systematic review, which showed some evidence of benefit when learners could perform repetitive practice using simulations and when educational feedback was provided.11 There also was a need for tasks to range in difficulty, and to be integrated with the curriculum. When criteria of functionality such as these are met, and in particular if the simulation, its content and fidelity, are appropriate, medical simulators would seem more likely to provide training and assessment of procedural skills, as well as rare complications, contrast reactions, and other medical emergencies. Training could thus become learner-centred, performed at the learner's pace, remotely from patients, with a new opportunity to learn safely from mistakes.12 Hence a pressing need to implement an alternative to patient centred learning, to define minimum standards for success, and to train and acquire new skills, has driven an explosion of interest in medical simulation. As the great promise of ‘virtual reality’ is embraced by specialities and industry, it is important to examine the evidence, to define its validity and its developing role.

In terms of the effectiveness or validity of a simulation to train a procedure, only that part of the simulation being used for the training needs to be validated. Face validity exists where the simulation appears to a trainee to resemble the real world task. Content validity is determined by subject experts who attest as to whether the simulation accurately replicates the procedure or process it claims to model. For assessment purposes, content validity should also confirm whether the metrics used are relevant to correct performance of the target procedure: construct validity determines whether the simulator can actually use these metrics. Concurrent validity compares the assessment with a gold standard, such as with the performance of experts. Concurrent validity reflects performance at the time of testing, whereas predictive validity predicts future competence in patients as confirmed in a subsequent clinical study. Ultimately there should be proof that the skills acquired by simulator training transfer to procedures performed in patients (transfer of training), and are then maintained over time.

An evaluation of experts and novices using endovascular simulation as a training aid for carotid artery stenting reported a significant improvement in performance of both novices and experts after a period of training for 30–60min, the novices rapidly achieving scores close to those of the experts.13 The time to complete the procedure was the only significant discriminator between novices and experts: while a surrogate measure of technical ability, time alone is clearly an imprecise discriminator of quality. A further study of endovascular simulator training demonstrated significant improvement in an observer's subjective scoring of simulated catheter manipulation, as well as in objective metrics of time to completion, fluoroscopy time and contrast dose.14 Observer based scores included correctness of procedure sequencing, though without weighting there is a lack of discrimination between trivial and critical procedural tasks, introducing uncertainty over claims of construct validity.

Operator skills common to both iliac and renal simulations have been identified in the Mentice VIST-VR, with a proposal of the existence of separate core skills for renal cannulation in the simulation.15 As yet, the relevance of such observations in a simulator to real world interventional performance, remains unclear. Studies of transfer of training skills are of fundamental importance in establishing this relevance of simulator training. One such study randomised subjects to either no training or simulator training with mentorship, and demonstrated skills transfer to procedures in patients in the latter group.16 An absence of equivalent mentorship in the non-simulator trained group does not provide a control for the powerful training, particularly for cognitive skills, provided by mentorship, and possibly accounting for at least some of the observed effect.

The CIRSE and SIR simulation task forces have pointed out the limited evidence which supports use of contemporary simulations for the acquisition of fine motor skills, and that experience on a simulator cannot yet be regarded as equivalent to training involving performance of actual endovascular procedures in patients.17 They do however confirm the suitability of using these devices within a defined, mentored curriculum to train relevant cognitive and knowledge elements, and aspects of procedural experience such as learning the correct sequence of procedural steps and selection of appropriate tools. Indeed, many medical errors result from incorrect procedural sequencing and such training seems likely to be beneficial prior to performing procedures on patients.

Hence the successful demonstration of clinical benefit using computer based simulator models to train skills in laparoscopy, colonoscopy and anaesthetics18, 19, 20 has yet to be convincingly reproduced for endovascular simulations, where interactions with tissues are occurring via long and flexible instruments. More faithful reproduction of the subtle, real world visual and tactile, cues and actions of the operator may be required to reliably train lower level motor skills that are important to success and patient safety in IR. Indeed, in a range of applications, workers are exploring the physical properties of tissues in order to better define and quantify instrument-tissue interactions and hence the requirements of the human-computer interface.21, 22, 23, 24, 25, 26, 27, 28, 29 There can be little doubt that in the longer term, computer based simulation has the potential to reproduce relevant aspects of the real world task, to train motor skills that today still require the fidelity of a ‘real patient’ environment. To attain this potential, however, also requires attention to the human factors involved in procedural medicine.

Without a clear understanding of the skills used by an expert cyclist to stay upright it would indeed be difficult to embark on the development of a simulator to train and assess such skills. In order for medical simulators to realize their full potential, their content and metrics need to be drawn from a breakdown of actual procedures performed in the overarching curriculum,30, 31 and simulators need to be able to use the chosen metrics. While there is a developing impetus to use simulator models to train interventional skills,32, 33 documentation of simulator development is often incomplete, lacking an identifiable source of the content and metrics used, and with no reference to specific training curricula. Hence, when considering whether a simulator is suitable for training and assessment in a curriculum it is important to understand how and by whom the test items were developed, and whether the metrics used are appropriate to the desired training objective.

Transparency of simulator development therefore should be an essential requirement, particularly if there are aspirations to use this technology for credentialing. Training organizations should set a minimum level of documentation of a simulation's development process and validation. Specific curricular training objectives that are already met by current simulations should be identified, and subject experts should review curricula to identify those tasks that require development of new simulations. This might include access to fundamental skills training in patients becoming unavailable or limited, or because the training task is high risk, or because training about infrequent or dangerous adverse events is desired.34 These tasks should then be analysed to identify the skills used by experts, as well as relevant metrics and an estimation of the fidelity required.

Extensive collaborative efforts are required to address the fidelity and human factors issues required for training low level motor skills, revisiting simulator specification, and aligning task definitions and metrics to target curricula. This brings a great opportunity for specialities with similar or parallel training objectives to work, together and with industry, to identify congruent areas for simulation within their curricula. Hence the work of simulator manufacturers could become more uniform and relevant across specialty borders. The best use of such simulations will be to provide training and assessment of proficiency within the wider curricula of the certifying organisations, including the core skills, knowledge and attitudes that are essential to realising the ultimate benefit: that of patient safety.