Inside the hospital OR, a dedicated team of health care professionals is preparing for surgery. Operating room protocol requires a systematic verbal confirmation of the medical process and procedures to be followed. The necessary sterile surgical tools are arrayed, the team is ready, the patient is wheeled in.

But this is a new reality.

The surgeon starts waving his hands in the air, gesturing and moving as if in another world. He looks something like a cross between a NASA spaceman and a symphony orchestra conductor, wearing a spacey-looking head-mounted contraption with large, tinted goggles.

He’s about to conduct one of the world’s first shoulder arthroplasty surgeries using a new mixed reality system. Mixed reality, or MR, is a blend of immersive computer-generated environments and the real physical environment.

The new technology combines cameras, trackers, sensors, communication tools and data overlays in a wearable headset that connects the surgeon to not just patient records, X-rays and other medical scans, not just to case histories and precedents but, if needed, to other health care practitioners around the world.

The first shoulder arthroplasty surgeries using what’s known as Blueprint Mixed Reality (MR) Guidance have now been successfully completed by Dr. Joaquin Sanchez-Sotelo at the Mayo Clinic in Rochester, Minnesota, and Dr. George Athwal at St. Joseph’s Health Care in London, Ontario.

They, along with technology developers from Microsoft and leading MedTech company Striker, developed the new Blueprint MR Guidance System which combines dedicated software with the Microsoft HoloLens 2 headset, allowing the surgeon to visually track the position and orientation of surgical instrumentation in the physical environment.

Without disrupting normal operating room workflow, surgeons and their instruments can be ‘guided’ by 3-D images and information display widgets, visually superimposed both over the patient and in the surgeon’s line of sight by the Blueprint MR Guidance systems.

The hardware and software platfrom allows the surgeon to plan an operation based on each patient’s anatomy and perform a surgery based on precise images of the patient (in this case, the shoulder) which are overlaid in the headset directly onto the surgeon’s field of view, mixing actual perceived visual reality with a mediated and augmented one.

“It is a groundbreaking tool that allows me to replicate the surgical plan very precisely,” said Dr. Athwal. “I am able to see issues with impingement of the joint replacement and ensure the best fit for the implant, which will hopefully provide the best outcome for the patient.”

Since 2022, Dr. Athwal has worked with the specialists at the Roth | McFarlane Hand and Upper Limb Centre at St. Joseph’s, developers at Stryker, and technologists at Microsoft, among others, to develop the mixed reality technique; he’s the first surgeon in Canada to perform a mixed reality shoulder replacement, and the second in the world.

Once he dons the specially-designed mixed reality headset, Dr. Athwal uses both verbal commands and hand gestures to manipulate a digital 3-D hologram of the patient’s anatomy and the metallic implant, which is based on a CT scan of the patient’s bones. As he operates, Dr. Athwal has the hologram close at hand – well, hovering in space – for quick and easy reference.

Surgeons often perform surgery on patients with diseased bones or inflamed organs, which can be quite different in shape and size from healthy ones. When the patient’s CT (computed tomography), magnetic resonance image (MRI), X-ray or other diagnostic scan gets displayed over the patient’s body, surgeons can see a more precise and current representation of the patient and treatment area.

Based on landmark orientation, fiducial registration and the calibration of equipment in the operating room and on the doctor’s head, system designers say spatial accuracy of less than 1 mm is achieved.

Microsoft’s HoloLens 2 enterprise edition, which first appeared nearly four years ago now, combines hand tracking, eye tracking, spatial mapping, visual reference and audio command capabilities in various wearable configurations, from simple headset and goggles to more robust hardhats with goggles to complete industrial hazmat full body enclosures.

HoloLens units are equipped with four visible light cameras used for head tracking, two infrared cameras used for eye tracking, as well as 8 MP imaging sensors for still images or 1080P 30 fps video. It uses depth sensors to map the environment and movements with in it, along with an accelerometer, gyroscope, and magnetometer.

“The addition of Mixed Reality Guidance to our Blueprint platform has the potential to benefit the entire shoulder arthroplasty market – surgeons and patients alike,” said Tim Lanier, president of Stryker’s Trauma & Extremities division. “Now that Mixed Reality Guidance has officially entered the operating room, our surgeons will have access to the most innovative technology available, enabling more precise surgical results and allowing for improved results for patients.”

Blueprint Mixed Reality OR Visualization, the previous iteration of Stryker’s Blueprint Mixed Reality, had been used by over 50 surgeons and in more than 3,500 shoulder replacement procedures globally.

Stryker’s Blueprint platform is a surgeon-controlled 3-D pre-surgery planning software for shoulder arthroplasty cases of all kinds, and for surgeons of all experiences.

The Blueprint Mixed Reality (MR) Visualization software solution from Stryker gives shoulder arthroplasty surgeons the tools needed to reference and interact with a 3-D stereo representation of their pre-operative Blueprint plan.

Stryker says it will continue to utilize and develop the Blueprint Mixed Reality (MR) Guidance System as North American and soon, European, regulators and health care professionals continue to gain experience and confidence with advanced surgical technologies – and new realities.

-30-