Starlight / Coca-Cola / Tool of NA

nocomputer is a part of the Tool Experience team.

Ava Max was captured performing all three songs IRL in volumetric video turning her into a photo realistic avatar and hologram experienced in augmented reality.
Each performance featured a different wardrobe and lighting setup that helped inspire the virtual worlds.

With 360 degree volumetric video and Web AR, Tool brought the team's creative direction to life and built out three mini verses for fans to freely explore and watch Ava Max’s exclusive virtual concerts while drinking their Coca-Cola Starlight.

‍

To integrate the Starlight product,  we asked people to scan the Starlight can to begin the experience. A challenge was that during early testing we found that we were getting poor accuracy due to the reflectiveness and shininess of the cans. Through trial and error and using a combination of spherical and rectangular image targets we achieved acceptable scanning results using the built- in 8Th Wall target recognition.

We also created a custom trained AI model - ssd_mobilenet_v2_fpnlite_320x320 -  specifically for the Starlight cans running in the browser. This model is very robust and has a very good tracking rate even in different and more difficult lighting conditions.

‍

All the experiences were built with our own custom WebGL pipeline that we then connected with 8thWall. To make sure we would have a consistent look and feel and to easily integrate all the shaders and effects across the 3 performances we created our own node based timeline system in Javascript/json. This enabled us to easily control, add/remove and time all shaders/effects, particles and geometry within each scene for all the different songs.

To have high fidelity, we spent a lot of effort in creating custom shaders for the hologram effect, the particles, the portals and the shading of the giant base rock.

‍