Home > Projects > PatientSim


PatientSim [1, 2] is our pilot project where we are developing a hybrid surgical simulator for cement augmentation techniques (see Figure 1).


Vertebral compression fractures (VCF’s) are mainly due to osteoporosis affecting primarily the vertebral bodies of the thoracolumbar region. The incidence rate in Europe was 1.4 million in 2002 and 700.000 in the U.S. in 1997 respectively [3, 4]. Three compression fracture types with different degrees of severity can be classified. In nearly all types of VCF’s, the endplates (cranial and caudal) of the vertebral body are more or less disrupted [5]. Therefore, a suitable treatment method (conservative or surgical) should be selected only after a detailed assessment of the fracture [6, 7].

Figure 1: PatientSim – a hybrid surgical simulator for cement augmentation techniques.

In case of a surgical intervention, cement augmentation techniques like vertebroplasty or kyphoplasty are very common. Both interventions are characterized through the application of artificial bone cement (mostly PMMA – Polymethylmethacrylate) into the affected vertebra(e). In case of kyphoplasty, an inflatable balloon is inserted prior restoring the initial height of the vertebral body. Thus, small cavities develop why the applied cement is more viscous and injected with less pressure compared to vertebroplasty. This decreases the risk of cement leakage. There are different ways of learning these procedures.

  • First, a mentor teaches novice surgeons directly on patients leading to poor patient safety.
  • Second, novice surgeons attend an intervention-specific course, where they can perform on human specimen under expert control. Although these courses are close to reality, some disadvantages should be mentioned. Besides high costs, the availability and especially ethical aspects limit the usage of human specimen. Furthermore, fluoroscopy is used for instrument guidance leading to radiation exposure and limited training periods.
  • Third, and addressed by this project, the use of a surgical simulator simulator allows to overcome the latter mentioned drawbacks.

Augmented Reality Simulator

Due to the advantages in comparison to VR simulators or box trainers, an augmented reality simulator is going to be developed. The whole simulation environment is designed in a three-layered structure (see Research, Figure 1). The patient phantom with artificial, physical vertebrae and soft tissue is the key element of the hardware layer. The surgeon is able to work on these structures using real surgical instruments supported by a detailed visualization. As a consequence the image guidance is achieved by a fluoroscopy simulation, which relies only on 3D models of anatomy and instruments and therefore requires no X-ray radiation. The rigid tools necessary for transpedicular bone access were equipped with sensor coils providing position and orientation in 3D using an electromagnetic tracking system.

Computer Application

Central to the hardware abstraction layer is a C++ console application; the device server. Connected clients can easily send scripting commands to communicate with certain hardware (e.g. the tracking system) via User Datagram Protocol (UDP) or Transmission Control Protocol (TCP). On the application layer, a graphical user interface (GUI) is responsible for a detailed visualization of the anatomical data. The GUI is programmed in C++ using various open source applications like Qt or Kitware’s visualization toolkit (VTK). Each desired anatomical structure can be loaded as a CAD model and registered using the characterized insertion instruments. The area of surgery is visualized in two-dimensional projections to improve the surgeon’s spatial perception. During each insertion trial, task-specific parameters (duration, instrument path length, etc.) are recorded for post-processing.

Patient Phantom

Artificial vertebrae as well as artificial soft tissue are the main components of the patient phantom. Three artificial vertebrae are embedded and clamped into a torso, which is covered by artificial soft tissue. A suitable polyurethane foam recipe with comparable haptic feedback was developed imitating the inner cancellous core of vertebrae. Additionally, different resin materials were tested covering the spongy structure as a layer of cortical bone. The resulting, artificial imitations were validated in reference to human specimen. Similar to the artificial vertebrae, the soft tissue imitations were developed and validated (see research area “Artificial Bones and Soft Tissue” for more details). After training of the bone access the penetrated vertebrae are removed from the clamping device and replaced by new vertebrae. Both, task-specific parameters as well as the accessed artificial vertebrae, which can be opened by using a band-saw, can be used for a final discussion with a medical expert.

[1] [doi] D. Fuerst and A. Schrempf, “Patientsim – development of an augmented reality simulator for surgical training of vertebroplasty and kyphoplasty.,” BioMed 2012, Innsbruck Austria, 2012.
Title = {PatientSIM - Development of an augmented reality simulator for surgical training of vertebroplasty and kyphoplasty.},
Author = {Fuerst, David and Schrempf, Andreas},
Journal = {{BioMed 2012, Innsbruck Austria}},
Year = {2012},
Abstract = {Surgical interventions have become more complex over the last years and the number of new surgical technologies is still rising. Surgical simulators represent safe and reliable training environments were novice surgeons can acquire the surgical skills required for new or complex procedures. Augmented reality or hybrid simulators combine the advantage of a detailed visualization with haptic feedback to make complex interventions as realistic as possible. As common treatment options for vertebral compression fractures, which are characterized by high incidence rates, kyphoplasty and vertebroplasty represent such complex interventions. Currently the training for both procedures is performed on cadavers with fluoroscopic guidance which results in two serious problems. First, the need for cadavers and second the radiation exposure limiting the duration of the surgical training. This paper reports the current stage of development of an augmented reality simulator which allows novice surgeons to train kyphoplasty and vertebroplasty without having these two disadvantages. We describe how surgical instruments are modified to allow position and force tracking during insertion into artificial vertebrae, and present a first concept for a processing and visualization unit. Preliminary data provide requirements for manufacturing patient specific vertebral phantoms.},
Doi = {10.2316/P.2012.764-078}
[2] D. Fuerst, M. Hollensteiner, and A. Schrempf, “A novel augmented reality simulator for minimally invasive spine surgery,” Proceedings of the 2014 summer simulation multiconference, p. {28:1–28:5}, 2014.
Title = {A Novel Augmented Reality Simulator for Minimally Invasive Spine Surgery},
Author = {Fuerst, David and Hollensteiner, Marianne and Schrempf, Andreas},
Year = {2014},
Pages = {{28:1--28:5}},
Abstract = {Currently the surgical training of kyphoplasty and vertebroplasty is performed on patients or specimens. To improve patient safety, a project was initiated to develop an Augmented Reality simulator for the surgical training of these interventions. Artificial vertebral segments should be integrated to provide realistic haptic feedback. To reach this, resulting forces during needle insertions (trans- and extrapedicular) into formalin-fixed vertebral specimens were measured. The same insertion procedure was also performed on six customized polyurethane blocks with varying mechanical parameters. Based on the results of these measurements, a specific foam phantom was generated and the insertion force measured. Additionally a parametric model for the needle insertion into bone was designed calculating three characteristic parameters for all insertion measurements. The resulting insertion force for the foam phantom was comparable to the specimen measurements and the parametric model provided comprehensible characteristic parameters. Based on the resulting force during needle insertion into human vertebrae, a possible foam recipe for manufacturing artificial segments was found. Furthermore, the parametric model provides characteristic parameters for the assessment of phantoms as well as the development of its production process.},
Journal = {Proceedings of the 2014 Summer Simulation Multiconference},
Publisher = {Society for Computer Simulation International},
Url = {http://dl.acm.org/citation.cfm?id=2685617.2685645}
[3] [doi] L. J. Melton, M. Thamer, N. F. Ray, J. K. Chan, C. H. Chesnut, T. A. Einhorn, C. C. Johnston, L. G. Raisz, S. L. Silverman, and E. S. Siris, “Fractures attributable to osteoporosis: report from the national osteoporosis foundation.,” J Bone Miner Res, vol. 12, iss. 1, p. 16–23, 1997.
title = {Fractures attributable to osteoporosis: report from the National Osteoporosis Foundation.},
author = {L. J. Melton and M. Thamer and N. F. Ray and J. K. Chan and C. H. Chesnut and T. A. Einhorn and C. C. Johnston and L. G. Raisz and S. L. Silverman and E. S. Siris},
journal = {{J Bone Miner Res}},
year = 1997,
month = {Jan},
number = 1,
pages = {16--23},
volume = 12,
abstract = {To assess the cost-effectiveness of interventions to prevent osteoporosis, it is necessary to estimate total health care expenditures for the treatment of osteoporotic fractures. Resources utilized for the treatment of many diseases can be estimated from secondary databases using relevant diagnosis codes, but such codes do not indicate which fractures are osteoporotic in nature. Therefore, a panel of experts was convened to make judgments about the probabilities that fractures of different types might be related to osteoporosis according to patient age, gender, and race. A three-round Delphi process was applied to estimate the proportion of fractures related to osteoporosis (i.e., the osteoporosis attribution probabilities) in 72 categories comprised of four specific fracture types (hip, spine, forearm, all other sites combined) stratified by three age groups (45-64 years, 65-84 years, 85 years and older), three racial groups (white, black, all others), and both genders (female, male). It was estimated that at least 90\% of all hip and spine fractures among elderly white women should be attributed to osteoporosis. Much smaller proportions of the other fractures were attributed to osteoporosis. Regardless of fracture type, attribution probabilities were less for men than women and generally less for non-whites than whites. These probabilities will be used to estimate the total direct medical costs associated with osteoporosis-related fractures in the United States.},
doi = {10.1359/jbmr.1997.12.1.16},
keywords = {African Continental Ancestry Group; Aged; Aged, 80 and over; Aging, pathology; Asian Continental Ancestry Group; Databases, Factual; Delphi Technique; European Continental Ancestry Group; Female; Fractures, Bone, epidemiology/etiology/physiopathology; Hip Fractures, epidemiology/etiology/physiopathology; Humans; Male; Middle Aged; Osteoporosis, Postmenopausal, complications/physiopathology; Osteoporosis, complications/physiopathology; Radius Fractures, epidemiology/etiology/physiopathology; Sex Factors; Societies, Medical; Spinal Fractures, epidemiology/etiology/physiopathology; United States, epidemiology},
language = {eng},
medline-pst = {ppublish},
owner = {P40701},
pmid = 9240721,
timestamp = {2011.07.26},
url = {http://dx.doi.org/10.1359/jbmr.1997.12.1.16}
[4] [doi] “Incidence of vertebral fracture in europe: results from the european prospective osteoporosis study (epos).,” J Bone Miner Res, vol. 17, iss. 4, p. 716–724, 2002.
title = {Incidence of vertebral fracture in europe: results from the European Prospective Osteoporosis Study (EPOS).},
journal = {{J Bone Miner Res}},
year = 2002,
month = {Apr},
number = 4,
pages = {716--724},
volume = 17,
abstract = {Vertebral fracture is one of the major adverse clinical consequences of osteoporosis; however, there are few data concerning the incidence of vertebral fracture in population samples of men and women. The aim of this study was to determine the incidence of vertebral fracture in European men and women. A total of 14,011 men and women aged 50 years and over were recruited from population-based registers in 29 European centers and had an interviewer-administered questionnaire and lateral spinal radiographs performed. The response rate for participation in the study was approximately 50\%. Repeat spinal radiographs were performed a mean of 3.8 years following the baseline film. All films were evaluated morphometrically. The definition of a morphometric fracture was a vertebra in which there was evidence of a 20\% (+4 mm) or more reduction in anterior, middle, or posterior vertebral height between films--plus the additional requirement that a vertebra satisfy criteria for a prevalent deformity (using the McCloskey-Kanis method) in the follow-up film. There were 3174 men, mean age 63.1 years, and 3,614 women, mean age 62.2 years, with paired duplicate spinal radiographs (48\% of those originally recruited to the baseline survey). The age standardized incidence of morphometric fracture was 10.7/1,000 person years (pyr) in women and 5.7/1,000 pyr in men. The age-standardized incidence of vertebral fracture as assessed qualitatively by the radiologist was broadly similar-12.1/1,000 pyr and 6.8/1,000 pyr, respectively. The incidence increased markedly with age in both men and women. There was some evidence of geographic variation in fracture occurrence; rates were higher in Sweden than elsewhere in Europe. This is the first large population-based study to ascertain the incidence of vertebral fracture in men and women over 50 years of age across Europe. The data confirm the frequent occurrence of the disorder in men as well as in women and the rise in incidence with age.},
doi = {10.1359/jbmr.2002.17.4.716},
keywords = {Age Distribution; Aged; Comorbidity; Europe, epidemiology; Female; Humans; Incidence; Male; Middle Aged; Osteoporosis, epidemiology; Prevalence; Prospective Studies; Sex Distribution; Spinal Fractures, epidemiology},
language = {eng},
medline-pst = {ppublish},
owner = {P40701},
pmid = 11918229,
timestamp = {2011.07.26},
url = {http://dx.doi.org/10.1359/jbmr.2002.17.4.716}
[5] [doi] F-D. Zhao, P. Pollintine, B. D. Hole, M. A. Adams, and P. Dolan, “Vertebral fractures usually affect the cranial endplate because it is thinner and supported by less-dense trabecular bone.,” Bone, vol. 44, iss. 2, p. 372–379, 2009.
title = {Vertebral fractures usually affect the cranial endplate because it is thinner and supported by less-dense trabecular bone.},
author = {Zhao, F-D. and Pollintine, P. and Hole, B. D. and Adams, M. A. and Dolan, P.},
journal = {{Bone}},
year = 2009,
month = {Feb},
number = 2,
pages = {372--379},
volume = 44,
abstract = {Cranial endplates of human vertebrae are injured more often than caudal, in both young and elderly spines. We hypothesise that cranial endplates are inherently vulnerable to compressive loading because of structural asymmetries in the vertebrae.Sixty-two "motion segments" (two vertebrae and the intervening disc and ligaments) were obtained post-mortem from thirty-five human spines (17F/18M, age 48-92 yrs, all spinal levels from T8-9 to L4-5). Specimens were compressed to failure while positioned in 2-6 degrees of flexion, and the resulting damage characterised from radiographs and at dissection. 2 mm-thick slices of 94 vertebral bodies (at least one from each motion segment) were cut in the mid-sagittal plane, and in a para-sagittal plane through the pedicles. Microradiographs of the slices were subjected to image analysis to determine the thickness of each endplate at 10 locations. Optical density of the endplates and adjacent trabecular bone was also measured. Measurements obtained in cranial and caudal regions, and in mid-sagittal and pedicle slices, were compared using repeated measures ANOVA with age, level and gender included as between-subject factors. Linear regression was used to determine significant predictors of compressive strength (failure stress).Fracture affected the cranial endplate in 55/62 specimens. Cranial endplates were thinner than caudal (p=0.003) by 14\% and 11\% on average, in mid-sagittal and pedicle slices respectively. Caudal but not cranial endplates were thicker at lower spinal levels (p=0.01). Optical density of trabecular bone adjacent to the endplates was 6\% lower cranially than caudally (p=0.004), and the average optical density of trabecular bone in mid-sagittal slices was 10\% lower in women than in men (p=0.025). Vertebral yield stress (mean 2.22 MPa, SD 0.77 MPa) was best predicted by the density of trabecular bone underlying the cranial endplate of the mid-sagittal slice of the fractured vertebra (r(2)=0.67, p=0.0006).When vertebrae are compressed naturally by adjacent intervertebral discs, cranial endplates usually fail before caudal endplates because they are thinner and supported by less dense trabecular bone.},
doi = {10.1016/j.bone.2008.10.048},
institution = {Department of Orthopaedics, Sir Run Run Shaw Hospital, ZheJiang University, HangZhou City, ZheJiang, Peoples Republic of China.},
keywords = {Aged; Aged, 80 and over; Biomechanics; Bone Density; Female; Humans; Male; Middle Aged; Skull, pathology; Spinal Fractures, pathology/radiography; Spine, pathology/radiography},
language = {eng},
medline-pst = {ppublish},
owner = {p40701},
pii = {S8756-3282(08)00867-3},
pmid = 19049912,
timestamp = {2013.11.15},
url = {http://dx.doi.org/10.1016/j.bone.2008.10.048}
[6] [doi] W. C. G. Peh, P. L. Munk, F. Rashid, and L. A. Gilula, “Percutaneous vertebral augmentation: vertebroplasty, kyphoplasty and skyphoplasty.,” Radiol Clin North Am, vol. 46, iss. 3, p. 611–35, vii, 2008.
title = {Percutaneous vertebral augmentation: vertebroplasty, kyphoplasty and skyphoplasty.},
author = {Peh, Wilfred C G. and Munk, Peter L. and Rashid, Faisal and Gilula, Louis A.},
journal = {{Radiol Clin North Am}},
year = 2008,
month = {May},
number = 3,
pages = {611--35, vii},
volume = 46,
abstract = {Percutaneous vertebroplasty is a safe, inexpensive, and effective interventional vertebral augmentation technique that provides pain relief and stabilization in carefully selected patients with severe back pain due to vertebral compression. Complications from percutaneous vertebroplasty can be devastating, but are rare and avoidable with application of a meticulous technique. Percutaneous vertebroplasty has a role in the management pathway of patients presenting with painful vertebral compression fractures. Kyphoplasty uses a balloon tamp with the aim of restoring vertebral body height, improving kyphotic deformity, and creating a cavity into which bone cement is injected. Kyphoplasty is as effective and safe as vertebroplasty in treatment of painful vertebral compression fractures. Skyphoplasty, a modification of kyphoplasty, is a promising new technique.},
doi = {10.1016/j.rcl.2008.05.005},
institution = {Department of Diagnostic Radiology, Alexandra Hospital, 378 Alexandra Road, Singapore, Republic of Singapore. wilfred@pehfamily.per.sg},
keywords = {Back Pain, surgery; Fluoroscopy; Hemangioma, surgery; Humans; Kyphosis, radiography/surgery; Magnetic Resonance Imaging; Osteoporosis, surgery; Patient Selection; Spinal Cord Compression, radiography/surgery; Spinal Fractures, radiography/surgery; Spinal Neoplasms, diagnosis/secondary/surgery; Tomography, X-Ray Computed; Treatment Outcome; Vertebroplasty, contraindications/instrumentation/methods},
language = {eng},
medline-pst = {ppublish},
owner = {p40701},
pii = {S0033-8389(08)00066-3},
pmid = 18707964,
timestamp = {2013.03.13},
url = {http://dx.doi.org/10.1016/j.rcl.2008.05.005}
[7] P. Hardouin, P. Fayada, H. Leclet, and D. Chopin, “Kyphoplasty.,” Joint Bone Spine, vol. 69, iss. 3, p. 256–261, 2002.
title = {Kyphoplasty.},
author = {Hardouin, Pierre and Fayada, Paul and Leclet, Herv and Chopin, Daniel},
journal = {{Joint Bone Spine}},
year = 2002,
month = {May},
number = 3,
pages = {256--261},
volume = 69,
abstract = {A variant of vertebroplasty known as "kyphoplasty" has been suggested for correcting vertebral compression fractures. A balloon placed inside the vertebral body is inflated to create a cavity, thereby restoring vertebral body height and allowing low-pressure cement injection. This procedure is gaining popularity in the United States. Over 1000 patients had been treated by the end of 2000. However, kyphoplasty is costly (chiefly because the balloon is disposable) and has not been evaluated in carefully designed studies. Although retrospective findings have been reported as highly promising, they are not sufficient to validate this procedure. The principle is innovative and the procedure deserves further investigation as a potentially effective means of correcting loss of vertebral height. Furthermore, use of a bone substitute instead of cement deserves investigation.},
institution = {Service de Rhumatologie, Institut Calot, Berck sur Mer, France. phardouin@hopale.com},
keywords = {Humans; Internal Fixators; Orthopedic Procedures, methods; Osteoporosis, complications; Spinal Fractures, etiology/surgery; Spine, surgery},
language = {eng},
medline-pst = {ppublish},
owner = {p40701},
pmid = 12102271,
timestamp = {2013.11.18}