The Challenge
In a unique entertainment broadcast, Pixotope was asked to facilitate virtual production on the USA’s first-ever augmented reality (AR) talent show. Alter Ego, which aired on Fox, was no ordinary singing competition. The Lulu-produced ‘reality’ show came with a twist - all the contestants were AR avatars.
.jpeg?width=1500&height=1000&name=Promo-Website-Website%20landscape%20content%20(2).jpeg)
The Background
This was a project that pushed the boundaries of AR to new limits, with constant technological innovation and a tightly-scheduled, complex production. The panel of celebrity judges and a 200-strong audience watched the live performances by the augmented reality avatars on studio monitors. Meanwhile, the avatars' human counterparts powered the performance via full motion capture from backstage.
Details at a glance
- Over 20 unique characters with multiple costume changes
- 60 bespoke AR assets
- 10 rehearsals a day
- 11 episodes at a runtime of 44 minutes each
- 8 Pixotope engines and 8 tracked cameras in the production area
.jpeg?width=1500&height=1000&name=Promo-Website-Website%20landscape%20content%20(3).jpeg)
The Pixotope Solution
As a key aspect of the workflow, Pixotope was used as a central hub, assembling all virtual production elements to ensure a seamless viewer experience. Pixotope enabled producers to move rapidly between the fully virtual contestants and the live audience and judging panel, without the need for post-production.
With so many moving parts, Pixotope was the key to successfully coordinating each element from the initial stages to the final show for the viewer at home.
.jpeg?width=300&name=Promo-Website-Website%20landscape%20content%20(5).jpeg)
Exploring virtual production’s full potential
While a compressed time frame and fast-paced production are typical on a broadcast entertainment series, Alter Ego took things to the next level. In order to maneuver over 20 unique characters with multiple costume changes composed of 60 bespoke AR assets for the performances, ten rehearsals a day, and top talent on board, the ability for Pixotope to implement last-minute changes — without the need for pre-rendering — became even more essential. With so many things in motion, the engine’s speed and reliability provided the production team with the peace of mind needed to assemble all the creative components and technical layers that needed to successfully go to air.
"We were doing a full season of television — 11 episodes at a runtime of 44 minutes each — in a very short amount of time, which has never been done to this scale before using AR. There were so many moving pieces to get working in coordination, multiple departments working in tandem to put on a live show in front of the judges and audience, so the whole project was a challenge.”
Juan Rubio, Pixotope Product Specialist
Rapid flexibility in real-time
The added difficulty of fully virtual contestants in front of a live judging panel and audience members necessitated precise real-time engineering. Pixotope enabled producers to move rapidly in this environment without the need for post-production.
Pixotope had eight engines and eight tracked cameras in the production area. The project required assistance not only with the typical compositing and live rendering production elements, but also with locating the augmented reality characters within the physical space.
Harmonizing live action
Pixotope integrated all live elements in the virtual production. This included taking the camera time and tracking data, plus real-time lighting and motion capture data, and integrating all components with the avatar files and toolsets built by Silver Spoon. The data was sent into the hub of Pixotope machines behind the main stage, transforming each live performance into an augmented reality experience.
.jpeg?width=1500&height=1000&name=Promo-Website-Website%20landscape%20content%20(4).jpeg)
Revolutionizing real-time entertainment
The flexibility of Pixotope tools facilitated the connection between the real world and CG, and there was plenty of creativity involved in finding production solutions.
To convincingly merge the avatars’ actions with on-stage elements, the augmented reality characters had to look natural within their performance space. Precise compositing was key. Pixotope harmonized real production elements and their virtual counterparts during performances, including lighting and follow spots. The production team explored integrating the virtual characters with physical on-stage elements and practical effects, such as low-light fog.
These environmental challenges made full use of the Pixotope toolset’s functionality, allowing real-time innovation to occur in a unique production setting.
The Future of Storytelling
“The whole thing was ridiculously impressive. Not only did we try - and successfully manage - to create a project of that scale in that space of time, but we managed to deliver something really cool that so many people enjoyed.”Juan Rubio, Pixotope Product Specialist
Pixotope is proud to provide a toolset that allows creativity to stretch beyond boundaries.
“At Lulu, we believe that the technology behind Alter Ego marks the future of storytelling in the broadcast. We are hugely excited about how Pixotope affords producers and tech solution providers the ability to support people in telling new stories in innovative ways. It opens up a whole new medium in presenting creative stories and storytelling, and we can’t wait to see where our journey with Pixotope takes us next.”
Dirk Sanders, Executive Producer and Director of Lulu
Discover our AR solutions
Pixotope supports advanced augmented reality solutions for ambitious productions. Read more about the Pixotope Augmented Reality Solution.