How High-Resolution Imagery is Revolutionizing 3D World-Building, Generative AI Models, and Immersive Visual Effects
What to expect in this article
- Discover how reality capture is transforming industries like VFX, AI/ML, autonomous vehicles, and urban planning.
- Learn about the challenges of current reality capture devices and how Mosaic Xplor addresses them.
- Explore the Mosaic Xplor backpack camera’s features, applications, and its role in pushing boundaries in visual effects.
- Dive into real-world case studies, including its use in the VFX production “The Ice Cream Man”.
- Compare 3D modeling techniques—Photogrammetry, NeRFs, and Gaussian Splatting—and see how Mosaic Xplor supports them all.
- See how Odyssey, a leader in generative world-building, leverages Mosaic Xplor to create immersive digital environments.
The Expanding Role of Reality Capture in the Digital Age
In the digital age, reality capture has become a cornerstone of innovation across multiple industries. At its core, reality capture refers to the process of digitally collecting and representing the physical world, enabling users to analyze, interact with, and recreate environments in high fidelity. This technology, once confined to niche applications, now plays a critical role in areas as diverse as film production, AI training, and infrastructure management. The demand for accurate, high-resolution visual data is greater than ever, and its applications continue to expand at a rapid pace.
Within the media industry, specifically film and visual effects (VFX), reality capture enables the creation of hyper-realistic environments that immerse audiences. Filmmakers and VFX artists use detailed 3D models and high-resolution imagery to blend real-world elements with computer-generated content seamlessly. Similarly, AI and machine learning (ML) rely heavily on reality capture to train algorithms for tasks like object recognition and autonomous navigation. High-quality, annotated data captured from the real world is vital for these systems to learn and make accurate predictions.
The AEC (Architecture, Engineering, and Construction) sector benefits from reality capture through enhanced design, planning, and project management. By integrating 3D models into their workflows, professionals can identify potential issues early, improving efficiency and reducing costs. Autonomous vehicles depend on continuously updated, high-resolution street view imagery to navigate safely and accurately. Meanwhile, industries like road maintenance and asset management rely on street-level data to monitor infrastructure, plan repairs, and optimize resource allocation.
And while a variety of applications can benefit from the transformative potential of reality capture technology, the effectiveness of this technology hinges on the quality and accessibility of the data it produces. Industries demand solutions that can deliver accurate, high-resolution imagery at scale, even in environments that are difficult to access with normal surveying and reality capture devices. This is where innovation in capture devices becomes crucial, laying the foundation for advanced workflows in some of the most dynamic and data-driven fields today.
The Limitations of Current Reality Capture Devices
Despite the advancements in reality capture technology, existing devices face significant limitations that can hinder their effectiveness across various industries. Understanding these limitations is crucial for identifying the gaps that innovative tools like Mosaic Xplor aim to fill. Below is an in-depth look at the most common reality capture devices and the challenges they pose.
Vehicle-Mounted Systems
Vehicle-mounted systems are among the most commonly used tools for large-scale reality capture. These systems typically involve cameras, LiDAR scanners, and GPS units mounted on cars or trucks. They are particularly effective for mapping urban environments and capturing vast amounts of data quickly and efficiently.
Limitations:
Accessibility: These systems are limited to areas that are navigable by vehicles. Narrow streets, pedestrian zones, and off-road locations are often inaccessible, leading to gaps in data.
Complex Setup and Maintenance: The integration of multiple sensors can lead to complicated setups that require constant calibration and maintenance, increasing operational costs.
Cost: High-quality vehicle-mounted systems are expensive and demand trained professionals, making them less viable for smaller businesses or projects with limited budgets.
Drones
Drones have gained popularity for their ability to capture aerial views and cover areas that are inaccessible by ground vehicles. They are particularly useful in surveying, construction site monitoring, and environmental studies.
Limitations:
Regulatory Restrictions: Drone operations are subject to strict regulations, including altitude limits, no-fly zones, and licensing requirements. These restrictions can significantly limit their use, especially in urban or sensitive areas.
Battery Life: Most drones have limited flight times, typically ranging from 20 to 30 minutes per battery. This limitation can lead to frequent interruptions during data collection, reducing efficiency.
Weather Dependency: Drones are highly susceptible to weather conditions, with strong winds, rain, and extreme temperatures often grounding operations.
Resolution and Detail: While drones excel in capturing large-scale aerial views, they may lack the resolution and detail required for close-range reality capture tasks like inspecting infrastructure or creating detailed 3D models of small objects.
Static Scanners
Static scanners, such as terrestrial LiDAR systems and tripod-mounted cameras, are used for capturing highly detailed 3D data from fixed positions. These devices are commonly employed in construction sites, archaeological digs, and industrial facility inspections.
Limitations:
Limited Coverage Area: Because static scanners capture data from a fixed point, their coverage is restricted to the immediate vicinity. To map a larger area, the device must be frequently repositioned, which is time-consuming and labor-intensive.
Operational Complexity: Static scanners require careful setup and calibration to ensure accurate data capture. This process often demands skilled operators, increasing the complexity and cost of data collection.
Immobility: These devices are impractical for capturing data over large or dynamic environments due to their lack of mobility.
Handheld Devices
Handheld devices, including portable cameras and LiDAR scanners, offer mobility and ease of use. They are often used for quick, small-scale data capture in confined spaces or for close-up inspections.
Limitations:
Operator Dependency: The quality of the captured data heavily depends on the operator’s skill and consistency. Inexperienced users may produce data with inconsistencies or inaccuracies.
Limited Battery Life and Storage: Handheld devices typically have shorter battery lives and limited storage capacity, which can constrain the amount of data that can be collected in a single session.
Restricted Scalability: While effective for small-scale projects, handheld devices are not designed to handle large-scale reality capture tasks efficiently.
The Common Challenge: Accessibility and Scalability
Across all these devices, a recurring challenge is accessibility. Many environments, such as dense urban centers, underground tunnels, and remote natural landscapes, are difficult or impossible to reach using conventional capture systems. Furthermore, the need to capture high-resolution data at scale introduces additional complexity, requiring devices that are not only reliable and portable but also capable of maintaining data quality across diverse conditions.
These limitations highlight the growing need for innovative solutions that can overcome these barriers. This is why the Mosaic Xplor came into play, providing unmatched versatility and performance in environments that other systems simply cannot handle.
And so the Mosaic Xplor Backpack Camera Came to Life
The Mosaic Xplor backpack camera is a direct response to the evolving needs of our customers in mobile mapping and surveying. Originally, Mosaic focused on developing high-performance vehicle-mounted systems like the Mosaic 51 and Mosaic X camera systems, which deliver exceptional results for urban mapping, GIS, and infrastructure inspection. However, as our customer base expanded, a new challenge emerged: the need to capture data in areas inaccessible by vehicles.
Learn more about the wide range of Mosaic customers here by reading their success stories.
Customer Feedback and Innovation
Many of our customers, ranging from surveying firms to infrastructure managers, sought a solution that would extend the capabilities of their existing Mosaic 51 or Mosaic X systems. They needed a portable, human-wearable device that could navigate environments vehicles couldn’t—narrow alleys, pedestrian zones, remote trails, and even indoor spaces. Their requirements were clear: they wanted a device that combined the high resolution and reliability of our flagship systems with the flexibility to go anywhere.
In response, we here at Mosaic engineered the Xplor backpack camera. This innovative solution was designed to meet the demand for a lightweight, portable system that could deliver high-quality imagery and data in the most challenging environments. The Mosaic Xplor retains the core strengths of our other systems—exceptional image resolution, advanced GPS capabilities, and rugged durability—while adding unparalleled mobility to the mix.
Core Features of Mosaic Xplor
13.5K Resolution Imagery: Offers the same level of detail and clarity as our vehicle-mounted systems, ensuring that no critical data is lost.
Advanced GPS Integration: Supports precision geotagging, critical for applications in surveying and mapping.
Rugged Design: Built to withstand harsh conditions, including extreme weather and rough terrains, ensuring reliability in diverse environments.
Ergonomic and Lightweight: The backpack design makes it comfortable to wear for extended periods, enabling users to capture data on foot without fatigue.
capture with revolutionary 13.5K resolution.
Expanding the Possibilities
With Mosaic Xplor, users can now capture data in places that were previously out of reach. This opens up new possibilities for industries that rely on accurate and comprehensive data collection. Whether mapping a dense urban environment, surveying a remote archaeological site, or inspecting infrastructure in hard-to-reach locations, Mosaic Xplor provides the flexibility and reliability needed to get the job done.
This innovation not only addresses our customers’ immediate needs but also sets a new standard for portable reality capture solutions. By bridging the gap between mobility and performance, Mosaic Xplor continues to empower professionals in mobile mapping, surveying, and beyond.
Proven Success: Mosaic Xplor in Action
Mosaic is no stranger to the VFX industry, having contributed to several high-profile productions using its vehicle-mounted camera systems. These include major blockbusters such as ‘The Gray Man’, ‘War II’ (a top-grossing film out of India), and ‘The Instigators’ (a recently released Apple TV production).
Learn more about several of our VFX customers here.
In these projects, Mosaic’s systems were instrumental in capturing dynamic scenes, particularly high-speed highway chases that required precision and detail to enhance the intensity of the action. These successes highlight Mosaic’s versatility in serving both mobile and wearable capture needs, solidifying its position as a trusted partner in VFX.
However, all previous work for VFX studios was with our vehicle-mounted solutions, mainly the Mosaic X camera system.
In 2024, we put our newly released Mosaic Xplor to the test and conducted our first VFX project using only our backpack camera, demonstrating its transformative potential in real-world applications.
With only a team of two, we traveled from our hometown of Prague to Amsterdam for the production of “The Ice Cream Man.” This project stands as a testament to how Mosaic Xplor can push the boundaries of visual storytelling by capturing immersive, high-resolution data in environments inaccessible to traditional mobile mapping systems.
The Amsterdam Project: “The Ice Cream Man”
In “The Ice Cream Man,” a VFX-heavy production set in the heart of Amsterdam, the challenge was clear: capture intricate street-level imagery in the city’s narrow, winding streets and crowded pedestrian zones. These environments presented a unique set of obstacles that ruled out the use of vehicle-mounted systems. This is where the Mosaic Xplor stepped in as the perfect solution.
The Mosaic Xplor allowed the production team to collect high-resolution, geotagged imagery on foot. Its lightweight design enabled operators to navigate the city’s narrow alleys and busy marketplaces with ease, all while maintaining the image quality needed for complex VFX workflows. The result was a seamless blend of real-world and digital elements, creating a visually stunning and immersive cinematic experience.
Todd Sheridan Perry, a renowned VFX supervisor, praised the Mosaic Xplor’s performance in his detailed review. According to Perry, the Xplor’s ability to capture high-quality imagery in confined and dynamic environments significantly enhanced the production process, streamlining the integration of visual effects.
For a comprehensive breakdown of how Mosaic Xplor was utilized in “The Ice Cream Man”, check out our case study here.
Expanding Horizons Beyond VFX
While its success in VFX is impressive, the Mosaic Xplor’s applications extend far beyond the entertainment industry. In urban planning, the device helps create detailed models of pedestrian pathways and infrastructure, which can be used to optimize city layouts. In AI training, high-resolution imagery captured by Mosaic Xplor is instrumental in refining machine learning algorithms for tasks like pedestrian detection and environmental analysis.
By delivering accurate, scalable data in challenging environments, Mosaic Xplor empowers professionals across various fields to make informed decisions and drive innovation. Its versatility ensures that no matter the project, Mosaic Xplor can adapt to the needs of the user, providing unparalleled value.
Next, we’ll explore the diverse methods of creating 3D models and how Mosaic Xplor supports advanced techniques like photogrammetry, Neural Radiance Fields (NeRFs), and Gaussian splatting.
3D Modeling: Comparing Methods
As reality capture continues to evolve, the demand for creating accurate and realistic 3D models has skyrocketed. Different industries, from film production to urban planning, rely on advanced modeling techniques to visualize and interact with their data. Thanks to tools like the Mosaic Xplor, capturing high-resolution data is more accessible than ever. But how this data is transformed into usable 3D models depends largely on the method employed.
In the sphere of 3D modeling, three primary techniques dominate: Photogrammetry, Neural Radiance Fields (NeRFs), and Gaussian Splatting. Each method has its strengths and weaknesses, and the choice of technique often depends on the specific requirements of the project. Below is a detailed comparison of these methods, drawn from our extensive experience in the field.
Photogrammetry
Photogrammetry is a process that uses multiple 2D images of an object or environment to generate a detailed 3D model, also referred to as a (textured) mesh model. By analyzing overlapping photos from different angles, this method reconstructs depth and spatial relationships with high accuracy.
Pros:
- Well-supported by a wide range of software.
- Highly effective for capturing hard surfaces, such as buildings, roads, and infrastructure.
- Produces photorealistic models when sufficient image coverage is available.
Cons:
- Struggles with complex or soft structures, such as foliage or fabric and glass
- Requires significant computational resources, especially for large datasets.
- Sensitive to lighting variations, which can introduce artifacts.
Neural Radiance Fields (NeRFs)
NeRFs leverage AI to generate 3D models from 2D images by learning a scene’s lighting, texture, and geometry. This technique excels at reconstructing complex visual phenomena, such as reflections and transparency, which traditional methods often struggle with.
Pros:
- Produces highly realistic models with accurate light behavior.
- Handles complex lighting and reflective surfaces effectively.
- Requires fewer images compared to photogrammetry for similar results.
Cons:
- Limited in terms of editability, as models are implicit rather than mesh-based.
- Computationally intensive and requires a robust AI framework.
- Still an emerging technology with limited support for generative applications.
Gaussian Splatting
Gaussian Splatting represents 3D environments using Gaussian primitives. This technique is particularly fast and well-suited for dynamic scenes or when rapid model updates are needed.
Pros:
- Exceptionally fast rendering and model creation.
- Highly editable, allowing for dynamic scene adjustments.
- Effective for capturing complex geometries and intricate details.
Cons:
- Struggles with flat surfaces and light transport fidelity.
- Requires expertise to fine-tune for optimal results.
- Limited applications outside specific use cases, such as dynamic scene modeling.
Limitations in Relighting
Relighting Challenges: Both NeRFs and Gaussian Splats have limited capabilities for relighting, meaning they cannot effectively simulate new lighting conditions without introducing visual artifacts. This restricts their use for direct editing, such as inserting objects or manipulating scene illumination.
The Mosaic Xplor Advantage
Mosaic Xplor plays a crucial role in supporting these 3D modeling techniques. Its ability to capture high-resolution imagery in challenging environments ensures that data input remains consistent and reliable, regardless of the method used. Whether generating photorealistic models with photogrammetry or exploring new frontiers with NeRFs and Gaussian splatting, Mosaic Xplor provides the foundational data required for success.
Setting the Stage for a New Way to Capture AI Training Data
Each of these 3D modeling methods has its place, but their true potential is unlocked when paired with innovative projects like those led by Odyssey. By partnering with Mosaic and leveraging the Mosaic Xplor as their capture tool, Odyssey combines the best of reality capture with cutting-edge modeling techniques to create immersive environments for film and gaming. In the next section, we’ll dive into how Odyssey is using Mosaic Xplor to revolutionize the way 3D worlds are built.
The Odyssey Partnership: Pioneering New Frontiers
Odyssey is at the forefront of visual storytelling, pioneering Hollywood-grade generative worlds that blend advanced AI with human creativity. Their mission is to empower professional storytellers by providing tools that enable the generation and precise direction of stunning video content. By developing models capable of producing high-quality geometry, photorealistic materials, dynamic lighting, and controllable motion, Odyssey offers a new approach to film and game creation. This technology allows creators to build immersive worlds and capture cinematic sequences with unprecedented control and efficiency. Odyssey’s commitment to integrating AI into the creative process aims to enhance, rather than replace, human storytelling, ensuring that the art of narrative remains central in the evolving digital landscape.
Founded by Oliver Cameron, former CEO of Voyage, and Jeff Hawke, former VP of Technology at Wayve, Odyssey leverages their deep expertise in AI, machine learning, and computer vision to redefine visual storytelling through cutting-edge generative technology.
Learn more about Odyssey here.
Why This Partnership Makes Sense
Odyssey’s desire to offer environments with high-quality geometry, photorealistic materials, stunning lighting, and controllable motion requires capturing data in environments that are inaccessible to traditional vehicle-mounted systems. For their latest initiative, Odyssey needed a tool capable collecting realistic and diverse environments including, “forests, caves, trails, beaches, glaciers, parks, and architectural masterpieces,” all while maintaining the high-resolution imagery necessary for their advanced 3D modeling workflows.
“Combined, these sensors capture our world in detailed, 13.5K resolution in 360 degrees, with physics-accurate depth information included alongside each panoramic capture. What’s more, since a human is in precise control of the sensors, they can ensure that every possible angle our generative models might find interesting is captured.
Think Google Street View for everywhere that cars don’t drive.”
- Oliver Cameron, CEO of Odyssey
This partnership highlighted Mosaic Xplor’s ability to adapt to the unique demands of generative world-building—a critical component of Odyssey’s innovative approach.
The Role of Mosaic Xplor in Odyssey’s Workflow
Odyssey integrates the data captured by Mosaic Xplor into a sophisticated pipeline that combines Neural Radiance Fields (NeRFs) and Gaussian Splatting. These techniques allow them to create dynamic, photorealistic 3D environments that are used in film and gaming.
“At Odyssey, we’re focused on pioneering this unified way to learn 3D. We're combining the inherent editability and speed of meshes, the ability of splats to model complex photo-real appearance and the real-world learning capabilities of NeRFs. We're developing a representation that integrates within existing 3D tooling for direct editing, fits natively into diffusion transformer pipelines, encodes both dynamic and static scenes, and scales computationally to achieve the highest visual fidelity for stunning visuals”, shares Oliver Cameron.
By leveraging Mosaic Xplor’s high-fidelity imagery, Odyssey can:
Train Generative AI Models: The accurate, real-world data serves as the foundation for AI systems designed to simulate lifelike environments.
Enhance Visual Storytelling: The combination of real and digital elements results in immersive experiences that captivate audiences.
Optimize Workflow Efficiency: Mosaic Xplor’s portable design and seamless integration reduce time spent on data collection and preprocessing.
For an in-depth look at Odyssey’s innovative process, visit their page here.
Impact on Film and Gaming
Odyssey’s use of Mosaic Xplor marks a significant step forward in the evolution of films and game development. The ability to capture and recreate real-world environments with such high fidelity allows filmmakers and game designers to push creative boundaries, producing worlds that feel authentic yet fantastical.
By choosing Mosaic Xplor, Odyssey has set a new standard for how reality capture can fuel the next generation of digital storytelling. Their work demonstrates how high-resolution, portable capture solutions can unlock new possibilities in an industry where realism and immersion are paramount.
A Partnership Built for the Future
Odyssey’s partnership with Mosaic not only underscores the value of Mosaic Xplor but also paves the way for future advancements in 3D modeling and generative AI. As the demand for realistic digital environments continues to grow, this collaboration stands as a model for how cutting-edge technology can drive innovation in creative industries.
Conclusion: Shaping the Future of 3D World-Building
The Mosaic Xplor backpack camera represents a pivotal innovation in the realm of reality capture. From its origins as a response to customer demand for a wearable version of Mosaic’s vehicle-mounted systems, Xplor has evolved into a tool that empowers professionals across diverse industries. Whether it’s navigating the narrow streets of Amsterdam for “The Ice Cream Man,” supporting AI-driven urban planning, or enabling cutting-edge generative world-building models for film and gaming for Odyssey, Mosaic Xplor consistently delivers unparalleled value.
Odyssey’s decision to integrate Mosaic Xplor into their generative world-building pipeline underscores the camera’s versatility and reliability. By capturing high-resolution data in previously inaccessible environments, Mosaic Xplor not only meets the demands of today’s projects but also sets the stage for the future of immersive storytelling. The partnership demonstrates how reality capture and advanced 3D modeling techniques can merge to create stunningly realistic digital worlds.
The Moral of the Story
The Mosaic Xplor is more than just a camera—it’s a gateway to innovation. By providing the foundation for high-quality, scalable 3D environments, it is helping to train generative AI models that will redefine how we experience film, gaming, and beyond. This groundbreaking technology empowers creators to push boundaries and explore new frontiers, turning ambitious ideas into reality.
Are you ready to elevate your projects with the power of Mosaic Xplor? Whether you’re in VFX, AI, or infrastructure, this tool can transform your workflow and help you achieve unparalleled results.
Contact us today to learn more and explore the possibilities of Mosaic Xplor and mobile data capture!
Find out more about applications of 360º mapping solutions in the VFX industry here 👈
