Augmenting the museum experience

Museum visitors’ experiences can be enhanced with augmented reality and interactive technologies, yet designers must be careful that these features don’t overshadow the objects at the heart of each exhibit. Ryerson researchers have tackled this challenge by creating digital interfaces that provide a more direct link to the content on display.

Making augmented reality more magical

A team led by professor Naimul Khan of the Faculty of Engineering and Architectural Science (FEAS) worked with Toronto-based companies SimentIT Inc. and SEED Interactive on an augmented reality project that is currently deployed at the Canada Science and Technology Museum in Ottawa. Their challenge was to create an app-based experience for a new exhibit of vintage technology called Artifact Alley, where users could see objects come to life on their smartphones without needing to scan a quick response (QR) bar code.

“This was a huge challenge, as there was low light, a lot of movement and the objects being partly obscured by people standing in front of them,” said professor Khan.

To meet this challenge, professor Khan collaborated with FEAS colleagues, including professors Ling Guan and Andy Ye, on an algorithm that harnessed both image processing and artificial intelligence. This powered a new app-based experience and opportunity for SimentIT and SEED Interactive that the museum calls Augmented Alley, which visitors can now download.

The app recognizes each user’s location while they walk through the museum hall, as well as 13 selected vintage items on display. This means that when users point their smartphone camera at objects, they can see animations in perfect proportion to their distance from the display. Through their screens, visitors can watch cars pull up to gas stations and images on early televisions.

“Because there are no QR codes and you don’t have to input anything into the phone, it feels a lot more magical,” said professor Khan.

The researchers presented their work at the International Conference on Image Analysis and Recognition. Their next project, with AWE Company Ltd., will see them apply a similar technology to outdoor events, providing event planners with a tool to visualize and demonstrate how features will look in real life.

Connecting with religious experiences from the 16th century

For professor Ali Mazalek of the Faculty of Communication and Design, it’s important to provide museum visitors with experiences that retain a connection with the physical world, rather than relying on screen-based or purely virtual interaction. To research this topic, she led a team of students in her lab to explore ways of immersing visitors in the history of 16th century Christian prayer nuts, which are the size of golf balls and have miniaturized carved scenes inside.

Their work, which was recently published in the journal Multimodal Technologies and Interaction, experimented with how an exhibit could appeal to different senses. To do this, they incorporated sensors in 3D-printed versions of the prayer nuts.

“Historical artifacts bring with them a whole rich history that’s very much based on the sensory, hands-on, tangible experiences that people had with the objects, at a very specific time and a very specific place,” said professor Mazalek, the Canada Research Chair in Digital Media and Innovation.

“A lot of that is lost when you take the artefact and put it in a glass case with just a short textual description.”

The sensors enabled the user to explore the scene within the prayer nut by touching parts of it and observing a magnified version on a projection. The user’s heart rate was monitored and they were encouraged to slow their breathing to spend more time with the object. When they relaxed, the projected surroundings were transformed into a religious setting accompanied by cathedral music. An aroma was also created to match the original fragrance that would have been put in the prayer nut by the owner centuries ago.

“I think people found a much greater personal connection to the artefacts than they would do normally,” said professor Mazalek. “They related the experience to things in their own lives and things that were meaningful to them.”

Professor Mazalek and colleagues at her Synaesthetic Media Lab are now working on a project with the Thomas Fisher Rare Book Library. The researchers will create a virtual catalogue of artefacts that the library has in storage to allow users to have a more tangible experience with them.

Professor Naimul Khan’s research was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) and SimentIT Inc.

Professor Ali Mazalek is the Canada Research Chair in Digital Media and Innovation and receives support for her research through the Canada Research Chair program.