Bringing avatars to life: Pixotope enhances virtual production in Alter Ego

AR female avatar signing in front of the judge panel of Alter Ego contest

The Challenge

In a unique entertainment broadcast, Pixotope was asked to facilitate virtual production on the USA’s first-ever augmented reality (AR) talent show. Alter Ego, which aired on Fox, was no ordinary singing competition. The Lulu-produced ‘reality’ show came with a twist - all the contestants were AR avatars.

The Background

This was a project that pushed the boundaries of AR to new limits, with constant technological innovation and a tightly-scheduled, complex production. The panel of celebrity judges and a 200-strong audience watched the live performances by the augmented reality avatars on studio monitors. Meanwhile, the avatars' human counterparts powered the performance via full motion capture from backstage.

Details at a glance

  • Over 20 unique characters with multiple costume changes
  • 60 bespoke AR assets
  • 10 rehearsals a day
  • 11 episodes at a runtime of 44 minutes each
  • 8 Pixotope engines and 8 tracked cameras in the production area


Augmented Reality avatar Night Journey powered by Pixotope, performing in front of the live audience

The Pixotope Solution

As a key aspect of the workflow, Pixotope was used as a central hub, assembling all virtual production elements to ensure a seamless viewer experience. Pixotope enabled producers to move rapidly between the fully virtual contestants and the live audience and judging panel, without the need for post-production.
With so many moving parts, Pixotope was the key to successfully coordinating each element from initial stages to the final show for the viewer at home.

Exploring virtual production’s full potential

While a compressed time frame and fast-paced production are typical on a broadcast entertainment series, Alter Ego took things to the next level. In order to maneuver over 20 unique characters with multiple costume changes composed of 60 bespoke AR assets for the performances, ten rehearsals a day, and top talent on board, the ability for Pixotope to implement last-minute changes — without the need for pre-rendering — became even more essential. With so many things in motion, the engine’s speed and reliability provided the production team with the peace of mind needed to assemble all the creative components and technical layers that needed to successfully go to air.

"We were doing a full season of television — 11 episodes at a runtime of 44 minutes each — in a very short amount of time, which has never been done to this scale before using AR. There were so many moving pieces to get working in coordination, multiple departments working in tandem to put on a live show in front of the judges and audience, so the whole project was a challenge.”

Juan Rubio, Pixotope Product Specialist

Rapid flexibility in real-time

The added difficulty of fully virtual contestants in front of a live judging panel and audience members necessitated precise real-time engineering. Pixotope enabled producers to move rapidly in this environment without the need for post-production.

Pixotope had eight engines and eight tracked cameras in the production area. The project required assistance not only with the typical compositing and live rendering production elements, but also with locating the augmented reality characters within the physical space.

Harmonizing live action

Pixotope integrated all live elements in the virtual production. This included taking the camera time and tracking data, plus real-time lighting and motion capture data, and integrating all components with the avatar files and toolsets built by Silver Spoon. The data was sent into the hub of Pixotope machines behind the main stage, transforming each live performance into an augmented reality experience.

AR avatar walking on the stage

Revolutionizing real-time entertainment

The flexibility of Pixotope tools facilitated the connection between the real world and CG, and there was plenty of creativity involved in finding production solutions.
To convincingly merge the avatars’ actions with on-stage elements, the augmented reality characters had to look natural within their performance space. Precise compositing was key. Pixotope harmonized real production elements and their virtual counterparts during performances, including lighting and follow spots. The production team explored integrating the virtual characters with physical on-stage elements and practical effects, such as low light fog.
These environmental challenges made full use of the Pixotope toolset’s functionality, allowing real-time innovation to occur in a unique production setting.

AR female avatar holding microphone and singing

The future of storytelling

Whilst challenging, the project's final delivery was enormously successful. 
“The whole thing was ridiculously impressive. Not only did we try - and successfully manage - to create a project of that scale in that space of time, but we managed to deliver something really cool that so many people enjoyed.”
Juan Rubio, Pixotope Product Specialist

Pixotope is proud to provide a toolset that allows creativity to stretch beyond boundaries.

“At Lulu, we believe that the technology behind Alter Ego marks the future of storytelling in the broadcast. We are hugely excited about how Pixotope affords producers and tech solution providers the ability to support people in telling new stories in innovative ways. It opens up a whole new medium in presenting creative stories and storytelling, and we can’t wait to see where our journey with Pixotope takes us next.”

Dirk Sanders, Executive Producer and Director of Lulu

Discover our AR solutions

Pixotope supports advanced augmented reality solutions for ambitious productions. Read more about the Pixotope Augmented Reality Solution.