Elevating Visual Effects with Advanced Scanning Solutions
Todd is an Emmy award-winning freelance visual effects supervisor/artist, screenwriter, editor, and director with over 25 years of experience. His diverse experience in video games, film, and television has led him to play key roles in projects ranging from LORD OF THE RINGS: THE TWO TOWERS to recent blockbusters, including AVENGERS: AGE OF ULTRON, BLACK PANTHER, and DOCTOR STRANGE. He has been animating since he was around 8 years old after being inspired by Star Wars.
As visual effects migrated from optical to digital, he started VFX house Max Ink Cafe with producer Jennifer Champagne, working together for over a decade to produce visual effects, full CG animation, and films. Perry writes for both Animation Magazine and Digital Media Net, reviewing software and hardware used in the VFX industry, which keeps him abreast of developing technologies.
The original text was published in the printed version of Animation Magazine in June 2024.
By Todd Perry:
Although scanning and photogrammetry have been a ubiquitous part of the VFX industry for many years, we have recently seen remarkable advances in capture devices and delivery formats. Polycam is a scanner in our phone. NeRF and Gaussian Splats are popular as lightweight, low-rez but surprisingly high-quality formats for games and apps. The question is: Where do we go from here? I think it’s about improving speed and volume, and Mosaic, a hardware tech and service company based in Prague, is one company to look at.
Mosaic has four robust 360° camera devices designed for capturing large areas and quickly — like mounted-on-a-traveling-car kind of quickly. But I’m going to focus this review on the Mosaic Xplor — a scanner designed like a backpack.
For a particular use-case scenario, I’ve been consulting on a short film that takes place in Amsterdam. The areas that we needed scanned were more pedestrian areas and there weren’t enough resources for a formal lidar scan or a traditional photogrammetry session, the latter of which consists of walking up and down the street taking hundreds of photos.
In comes the Mosaic Xplor, which mounts onto the user’s back — the workstation is in the backpack part, while an arm mounts to the back and hangs out above the user’s head, fitted with an array of cameras for the photogrammetry at a total of 14K resolution with two 120-degree lidar scanners scanning at 400K points/sec to a range of 40 meters, which lock down the geo-position of the cameras.
The backpack only weighs 15 lbs. — 5 lbs. of which is the camera array overhead. That allows the user to walk around and scan with your hands free, allowing not only for comfort but also safety. But the capture system is still fast enough that you could capture if you are on a bike or a scooter. The output is thick with data, which can be resolved into a textured mesh via photogrammetry or into NeRFs and Splats.
Mosaic does offer processing services to convert data. They have a proprietary system for dealing with lens distortion from the cameras, but if you are clever you have the option of attempting a solve in the photogrammetry tool of your choice. Just know that you may have to take an extra step to iron out the lens distortion first.
Also, if you do have supplementary data, such as drone footage, traditional photogrammetry and texture photos, lidar scans and other geospatial data, you can get even higher accuracy and detail.
I found that even the decimated meshes have a ton of internal detail, with additional detail being derived from normal maps extracted from the data. The silhouettes of structures can get a little “rubbery,” but this is a limitation of photogrammetry as a process and not the capture system itself. There is plenty of data to accurately represent the volume, which even a junior modeler could make quick work of refining.
The Mosaic Xplor is portable, lightweight, and compact, which allows you to get into areas that otherwise would be really difficult to scan. Yet, the support hardware ties itself to the photos being taken, making less guesswork in the final solve. In fact, being able to capture in a crowded Amsterdam pedestrian thoroughfare just proves how capable and versatile this is!
Check out videos of Mosaic Xplor and examples of data captured by this 360° Mobile Mapping backpack on the Mosaic YouTube channel. You can also find behind-the-scenes footage from our recent Amsterdam project, as mentioned by Todd 😊
See for yourself the details of the 3D models created with imagery by Mosaic Xplor! Find the model from Amsterdam on NIRA and Prevu3D.
For more details on the Mosaic Xplor or our VFX services, please contact our team below.
