Technology and Education Combines to Play in Augmented Reality Sandbox

By: Lee Rickwood

October 7, 2016

OK, so playing in a sandbox does not immediately evoke images of higher education or sophisticated technology.

But visit the Muskoka Discovery Centre in Gravenhurst, ON and you will see that sandboxes have come a long way.

The Centre’s newest exhibit, called the Shaping Watersheds Sandbox, makes creative use of state-of-the-art augmented reality technology to get young and old, student and teacher alike to run their hands through real sand to form virtual mountains, valleys, lakes, and rivers.

While playing in the sand is fun in the moment, the technology deployed for this interactive installation, of course, has a longer term goal – helping folks to learn about nature, ecology and the all-so-critical role water plays in the health of our planet.

“Technology is a tool for creativity,” wrote Olivia Simpson, Educator at Muskoka Discovery Centre, in correspondence with

sandbox-at-muskoka-centreOf course, she’s referring to the visitors at the Centre, but she also means the staff and educators there, as well. The entire investigation and production process behind the installation of the Sandbox itself was a terrific learning experience, and the Centre has developed other ideas and applications that combine digital media and computer technology into a powerful package for continued learning.

“We believe that learning should be a creative and exploratory process,” she continued, again speaking as much of the visitors as the staff. “What we personally dove into was a whole other world of opportunity. The stories we were unable to tell our visitors before all of a sudden had an outlet.

“The fact that new technology is eye-catching, and grabs the attention of those generations brought up in this modern world is something we accept. We began by adding to the standard artifact and text panel exhibits we already had on display.”

The “eye-catching” exhibit itself demonstrates in living colour how water moves and interacts with the landscape to create watersheds. Standing at the sandbox, people find that holding their hands above the sand will cast a shadow, causing blue virtual rain to fall from a virtual cloud.

“Water is a critical resource for all life on earth,” Simpson noted. “The dependency on healthy freshwater sources to the Muskoka region supports that this new exhibit is a perfect educational learning lab for all students and teachers in Muskoka and surrounding regions.”

Depending on how the user has designed his or her sandbox terrain, the rain might fall in a river that flows until a dam stops it. Or it could flood farmland and residential areas.

“We were inspired by the design of AR Sandboxes by Oliver Kreylos, a computer scientist at the University of California, Davis,” she explained, after one of the Muskoka Centre’s board members saw it installed at a learning centre in the

He recorded some video of the sandbox, shared it with other staffers at the Centre, and eventually with a fabrication team from the Muskoka Discovery Centre and Friends of Muskoka Watershed.

Kreylos describes the high-tech sandbox concept as a “3D visualization application to teach earth science” and related ideas like watersheds and catchment areas. It has grown to be a publicly-accessible hands-on physical exhibit that combines a real sandbox with virtual topography and water elements, created using a closed loop from a Microsoft Kinect 3D camera, along with some powerful visualization software and a data projector.

Topographic models are created by the user (player!?) shaping real sand, and those created models are then augmented in real time by an elevation colour map, topographic contour lines, and simulated water. An overhead projector shows the rendered results on the sandbox.

The driving software is based on the Vrui VR development toolkit and the Kinect 3D video processing framework, and is available for download under the GNU General Public License.

Although Kreylos was not directly involved in the Muskoka project, technical staff and project volunteers there completed all the research, sourced all of the products, built the box and designed the unit that houses the video projector and additional monitors.

sandbox-display-at-muskoka-centreThere are now two sandboxes at the Centre, each at different heights to better accommodate children and visitors with any mobility compensations.

“The additional monitor’s stationed above the sandboxes help us engage those who are not directly interacting in the sand. People are able to watch what is going on from afar and it also helps to attract people closer from across the room,” Simpson described.

Yes, there’s a huge “Wow!” factor at play when young people first see the sandbox; its popularity is not surprising, based on that eye-catching nature of cool technology these days, and in fact, there’s a growing sandbox movement out there!

Many users, unaffiliated to UC Davis or the inventive team there, are building their own versions, and there’s now an official AR Sandbox support forum on the Lake Visualization 3D website where users can help each other or provide feedback.

So the creative learning and exploration continues, and the sandbox experience encourages the Muskoka education team to consider even more technology applications, Simpson said, adding to existing uses of tech at the Centre, such as its X-Plane flight simulator.simulator-screen-at-muskoka

“Currently we have a few interesting projects still in the funding stage. The first is a floor projection system. This would be a great addition to our building as it would elevate the initial ‘wow factor’ people experience when they walk into the main exhibit space. The projection would be interactive, showing a large scale map of the three big lakes. We have a couple other projects still in the research phase, including one which could bring something VERY unique to our building. Something that would be the first in Canada, as it is not yet even available to the world.”

How’s that for great playground tease?!








Leave a Reply

Your email address will not be published. Required fields are marked *