Skip to content
Dot 18 January 2023 4 min read

What is Real-Time Camera Tracking for Virtual Production?

Chances are, if you’re interested in virtual production, you’ve come across camera tracking and at least vaguely understand its function in such workflows. If you’re new to virtual production (well hello, you!), real-time camera tracking is the bedrock to making virtual production look believable. 

To put it simply, camera tracking replicates the camera position and movements within the virtual environment. While this sounds straightforward, it has been fairly complicated in practice. And if you are not spot on with camera tracking, your audience will immediately know  – the whole illusion that is virtual production crumbles. 

The good news is that camera tracking technology is improving rapidly and in such a way that this complex process is being simplified, making it easier for more people to adopt virtual production. 


Mechanical Camera Tracking vs. Optical Camera Tracking

To understand camera tracking, you must first understand the differences between mechanical and optical tracking systems. Both work to achieve broadly the same outcome: to accurately determine the position, orientation, zoom and focus of the camera or cameras, such that the physical and virtual worlds can be accurately aligned. This is critical when merging physical and virtual elements. Depending on the project at hand, these different types of tracking can be used independently or in concert with each other. Here’s what they do and how they work:


Mechanical Tracking 

Mechanical camera tracking combines data from the mechanical movement of the head (pan/tilt) and rig with data from the camera lens (zoom) to accurately describe the camera’s location and status. Mechanical tracking has been around for some time and is still commonly used for projects that involve cranes, dollies, and spiders (support rigs), such as live event broadcasts. Each movement or rotation axis of the support rig as well as each parameter of the camera and lens are transmitted in detail to create an exact mathematical model of the physical camera setup. 


Optical Tracking  

Optical tracking uses images, either from a specific “sensor camera” or in certain scenarios from the camera itself in combination with special markers or objects in the natural environment to calculate the camera position and orientation using computer processing. The sensor camera can be equipped with an infrared (IR) pass filter in front of the lens, and a ring of IR LEDs around the lens to illuminate the measurement space with IR light. When combined with retro-reflective markers, this enables the tracking to continue to work reliably even in areas of low light. Sensor cameras are not typically pointed towards the stage or set, and instead, towards the floor or ceiling where, if markers are required, they will not be in the shot. 


So, Which Type Of Tracking Should You Use?

Mechanical tracking has been around a lot longer than optical tracking and is often considered robust and reliable. It does, however, require significant - and sometimes frequent as environmental factors change - (re)calibration. However, since mechanical tracking doesn’t depend at all on markers or distinguishable objects in the natural environment, there are certain scenarios where mechanical tracking still has some advantage, such as very large low lit or low-contrast spaces. 

Meanwhile, optical tracking offers far more flexibility. Mechanical tracking requires the camera rig to be “anchored” at a known location from which the mechanism movement can be referenced. By contrast, cameras with optical tracking can be moved freely even, for example, as a steady cam. Hence for certain productions, they are often used together.


How Pixotope Camera Tracking Helps   

Once you start peeling back the (many) layers of camera tracking, you’ll quickly see that there’s rarely a one-size-fits-all workflow. We’ve got you covered though. 

We’ve been hard at work following our acquisition of TrackMen, the expert one-stop-shop for real-time 3D camera and talent tracking in the media industry for 25 years, to demystify the complexities of camera tracking for all media producers. We like to think we’re doing a pretty good job so far. Pixotope Camera Tracking is unique in the market as the only system that covers all tracking methods, from optical to mechanical to markerless to marker-based and TTL (through the lens) in a single software product. 


Learn More About Real-Time Camera Tracking  

In most cases, you can't do virtual production without real-time camera and talent tracking. It’s the piece of the puzzle that drives photorealism and the suspension of disbelief for your audience. You can’t pull off the illusion if your virtual environment and objects aren’t responding realistically to the changing positions of your camera and talent. 

Contact a Pixotope expert today for a demo of Pixotope Camera Tracking to learn how we’re making it easier to bring into your workflow.