WHAT IS TALENT TRACKING?

Talent tracking is a method of identifying the position and pose of on set talent, without the need for motion capture (mocap) suits or other equipment attached to or worn by the talent.

HOW DOES TALENT TRACKING WORK?

Pixotope offers two methods of talent tracking. Included as standard in all Editions of Pixotope Graphics is the Body Pose Estimation feature. This version of talent tracking uses Artificial Intelligence to process the TTL (through the lens) image to establish the position and pose of the on set talent. For more complex scenarios, for example with several people on set, we are able to offer a standalone solution that processes the image from 3 or more “sensor cameras” placed around the studio.

PIXOTOPE

TALENT TRACKING

Let your on-screen talent interact with the virtual environment, with Pixotope real-time talent tracking system.

Harnessing TalenTrack and Body Pose Estimation technology, our full body pose tracking tool is designed to make your virtual production easier and supporting creativity in productions.

Book a demo

TALENTRACK FEATURES

AI driven body pose

Precisely tracks the body and limbs of on-screen talent through a detailed 3D skeletal structure using AI-driven computer vision algorithms.

Cost-effective talent tracking

Enjoy the advantages of tracking one talent precisely without the costs and complexities of additional hardware.

Track multiple talents

With affordable network-connected cameras and clever processing software, our TalenTrack technology makes talent tracking cost-effective and practical.

Tracking from all angles

Pixotope’s talent reality tracking system allows you to track talents from all angles, with no input from the camera operator.

Scale up easily

Our system requires at least 3 cameras to track a defined area.

This area can be expanded with additional cameras to cover any shape, including long aisles and irregular stages.

Adapts to any production

Define areas that should be left out of the detection zone and a 3D tracking shape to ensure that, from all the 3D space covered by the witness cameras, only the volume of interest will track the talent.

CREATE REALISTIC VIRTUAL SHADOWS

FOR LED FLOORS

Natural shadows often disappear on LED floors due to the light they emit, creating an unnatural look. This is where accurate body pose data becomes crucial.

Our TalenTrack, with its precise tracking, solves this problem for both LED floors and virtual sets where a "good key" isn't achievable, effectively killing natural shadows in those scenarios.

By knowing the talent's position and posture in real-time, TalenTrack adds realistic virtual shadows that seamlessly blend the virtual world with the real, for a truly immersive experience.

LET YOUR TALENT MOVE FREELY

WITHIN THE VIRTUAL ENVIRONMENT

Our system seamlessly integrates actors and real objects into your virtual studio or green screen, allowing for natural movement and interaction with virtual elements.

Imagine a layered stage where actors can walk in front of, behind, and even cast shadows on virtual objects – all thanks to Pixotope's advanced tracking technology.

No trackers or markers are needed on the talent, who can maintain freedom of movement, act naturally, and focus on delivering the story.

INTERACT SEAMLESSLY

WITH VIRTUAL OBJECTS

With Pixotope talent tracking system, your talent can effortlessly touch, kick, and manipulate virtual objects as if it's second nature.

The secret? Precise talent tracking and limb movement monitoring.

This technology enables you to know the exact location of talents and accurately track their every move, allowing them to interact within the environment seamlessly.

VIRTUAL REFLECTIONS

THAT MOVE WITH THE TALENT

Realism is the name of the game in virtual productions. One of the most important aspects of realism, and one of the hardest to nail correctly, is talent reflections.

Whether you work in a green screen environment or an LED stage, creating accurate reflections is a hard task.

Talent tracking answers a big part of this task. Accurate talent position information allows you to make sure the reflections will stay with the talent regardless of where he is on stage and maintain realism throughout the show.

Number of talents tracked

Required body visibility

In/Out shoot

Tracking technology

License

BPE

full body

full body in shot required

Through-The-Lens, 2D plane

Included with VS/AR and XR

TalenTrack

partial body

tracking continues if talent is out of camera shot, as long it stays in tracked volume

standalone tracking separated from the camera tracking. 3D volume

separate license

FAQs

Why does virtual production need talent tracking?

Talent Tracking both makes Virtual Production easier, but also enables more creativity in productions. Talent Tracking enables the onscreen talent to interact with the virtual environment, while also enabling us to move and auto-size the compositing plane, making it possible to easily manage elements such as shadows and reflections and create hyper-realistic virtual sets.

How many people can I track with TalenTrack?

TalenTrack tracks a volume of space. Inside this volume, the system can track up to 5 people simultaneously.

Do I need to worry about talent overlapping each other?

Not at all. TalenTrack uses multiple wide-angle witness cameras to track a volume of space in the studio.

These cameras are positioned in different places, and different angles to cover the volume from multiple directions.

In this way, even if the talents overlap from one point of view, one of the cameras will pick them correctly.

Are there limitations to the size or shape of the tracked area?

Not really. The basic setup of TalenTrack requires three witness cameras, but it can scale up to fit larger or odd-shaped spaces.

Keep adding witness cameras to cover your desired tracked area.

Does TalenTrack works for XR or just for VS/AR?

Absolutely! Talent tracking is super important for the realism of the virtual production, whether its a virtual studio, or an XR stage.

From lighting effects that effects the talent correctly, through virtual shadows and reflections, and all the way to interactions with the environment, talent tracking is key to enhance the immersion of the talent in teh virtual environment.

Can TalenTrack be used for "puppeteering" ?

We do not recommend it. TalenTrack recognizes the talents, and creates a virtual skeleton that the designers can use to drive actions in the scene.

However, keep in mind that TalenTrack does not create a volume for the character, it does not track fingers movement, and doesn't track feet.

What equipment is needed for TalenTrack system?

Pixotope TalenTrack kit will arrive with purchased witness cameras kit. You will need to provide workstations to run the TalenTrack software.

Depends on the use case, we recommend either one workstation for 3 cameras, or for lower latency, one workstation for 2 cameras.