My Process | Narrative Augmented Reality Experience
Game of Thrones AR
Project Description
Game of Thrones in AR is a project collectively done by Subigya Basnet and Vhalerie Lee, graduate students at NYU, for Narrative in Augmented Reality class. The project is a mobile AR experience where viewers could either explore Westeros and interact with the sigils, or viewers could watch a 3 min animated story of how Aegon Targaryen conquered Westeros.
The inspiration to venture on Game of Thrones particularly started with a deep love for the TV show, and with a realization about the vast possibility of telling its narrative in augmented reality. The core idea of the project is to augment assets and animations on a 24 x 36 inches map of Game of Thrones world. The major objective for building this project was to experiment building a fuller narrative experience using marker based AR technology. We were very keen on trying how far can we push existing technologies, to create something more than just an augment popping up through a image tracker. By doing this we aimed to:
- Allow the viewers to explore the world of Game of thrones, and get a better understanding of the geographical positions of various houses and their strongholds.
- Give a short narrative of the history of Game of Thrones by telling the story of how Aegon Targaryen conquered the seven kingdoms, leading up to the scenario where the actual TV show begins. The tone of the narrative will be introductory to ensure people who are not familiar with the show or the book will also be able to engage with it actively.
The Team
Left to Right: Vhalerie Lee and Subigya Basnet
Technology
Initial Design Document
3D Modelling
To create the experience, we had to ensure that we model every castle of the major houses of Westeros. Vhalerie was the 3D modeler in the team, and she designed all the castles by referring to show footages and other reference material found on the internet. I then worked on texturing these models on Unity, and making sure that it reflects the castles from the original show. We also got a few assets from the Unity store, e.g. the low poly forest, dragon and particle systems.
Initial AR Testing
As we were working on the models, we also wanted to ensure that the models were working properly in AR. We did a few low-fi prototype testing of the finished models on some sample markers, and altered the model if they had higher poly count than what the phone could handle.
Background Sound Design
Sound was an integral part of our AR experience. As our objective for the project was to create a complete narrative experience, we started working for the first draft of the background sound which consisted of voice over of the script, and a background theme. It was also important to have the background voice over at this stage of the production because all the animation had to be timed to the narration and the music. We used a Zoom H4N to record the voiceover.
Prototype v1
In the first prototype, we were trying to figure out how can we sequence triggering different animation across time. Since this was my (and also Vhalerie’s) first AR project, the initial weeks had a huge learning curve for us. However, triggering various animations across time was the backbone of the whole experience, and on this prototype we had a proof of concept for the project.
Prototype v2
For the second prototype, we had completed modelling and textures all the castles for the experience and they had been placed at the right location in the map. We also added a particle system to replicate the weather at the northern most part of the map. In addition to that, we started building out the timing the animation in this prototype. You can get a preview of it in the video below.
Prototype v3
In third prototype, we completed timing all the animation across the background track of 3 minutes. This prototype was the first time one could see the whole experience, and understand what the project was all about. Creating this particular prototype was a considerably time consuming experience, as each animation had to be clearly synced with the music. It was made particularly hard by the fact that Unity does not allow things to be animated in a timeline. All synchronization had to be hard coded through C sharp scripts. In this process, we got assistance from Sam von Ehren, who helped us come up with modules for various triggering actions in the experience. However, even until this prototype, we were using a sample marker to trigger the experience, and were experimenting how can we trigger it through the whole map.
Sound Design
As mentioned above, sound was one of the integral part of the whole experience. Once the animation timings were set, Vhalerie started working on adding special sound effects across the background sound to enhance the narrative experience. Sound bites of dragon flapping their wings, flying, screaming, breathing fire were added to bring the animal to life. Similarly, sound bites of people marching on the battle field, clunking of the swords, and screams were added to dramatize the battle scenes. Once complete, the sound was then added to the final version of the experience.
Documentation
Once the project was complete, we started focusing on working on the documentation for the experience. During this period we made a short teaser for the project, and also made a comprehensive behind the scene making video of the whole project.
Trailer:
Making of Game of Thrones AR: