Metaverse cameras – hardware that will physically build the metaverse

Yeah, yeah, yeah, the metaverse is this amazing place, but how do we actually make it happen? Short answer: metaverse cameras – and a lot of ‘em 

Quick background on the metaverse

If you’ve been following anything in the last few weeks, then chances are you’ve seen this crop up: ‘What is the metaverse?”  

Due in part to Facebook’s decision to rebrand itself to ‘Meta’, this word, in partnership with its half-sibling – the metaverse – have become the hot topic of conversation, sparking every media outlet and social media platform to start creating stories, videos, memes, and even parodies (well done Iceland!)

Yet, despite the skyrocketing trend in searches worldwide for ‘metaverse’ (see the image from Google Trends below), to be honest, I wasn’t finding much helpful information. 

What every article on the ‘metaverse’ will tell you

Nearly every article – whether from a major news media outlet, a small blog, a YouTube channel or a Twitter feed – I kept coming across these same, rather abstract, pieces of information about the metaverse. Allow me to cut to the chase and sum it all up for you here:

1. The term came from a 1992 sci-fi novel by Neal Stephenson called “Snow Crash”; the metaverse in “Snow Crash” was a little bit tongue-in-cheek, a virtual world providing an escape from the unpleasant and crazy real world.
2. Metaverse is the combination of two words: ‘meta’ = beyond and ‘verse’ from universe. So essentially it means beyond our current understanding of a universe.
   1. But speaking of definitions – there isn’t actually any one definition anyone can seem to agree on, leading most sites, videos, and articles to go on to quote 2, 3 or even 4 different definitions in order to cover their bases. The Verge even lists out about 5 traits that need to be included in order that this eventual platform/environment/space becomes the Metaverse.  
3. The leading companies who are already working towards the metaverse are Facebook – sorry – Meta, Microsoft, Apple, Nvidia, and Epic Games
4. The leading use cases are expected to be:
   1. Entertainment purposes like attending social events such as concerts or taking a trip; for educational purposes 
   2. Business meetings and trainings for employees
   3. Educational purposes and learning experiences, such as learning a new skill or how to repair a product
   4. Trainings for healthcare workers, military, customer care 
   5. Creating unique items and experiences (similar to the NFT industry)
5. What we need to make the metaverse a reality:
   1. High speed internet and servers which can handle thousands or millions of people at once (for these concert we’re all going to attend)
   2. VR / AR glasses (including w/ motion tracking, hand tracking)
   3. A persistent, always on, online world

So now that we’ve gotten that all out of the way, let’s move on to a much more tangible, concrete answer:

How are we going to build the metaverse?

A “collective, virtual shared space created by the convergence of virtually enhanced physical reality and physical persistent virtual space, including the sum of all virtual worlds, AR and the internet” (from the Metaverse Mogul) does not just happen on its own. There are a number of hurdles ahead of us to make this still abstract idea, a reality. 

So what are the challenges, how can we deal with them, and what hardware do we need to not just access the metaverse, but to build it? 

What are the issues stopping us from achieving the metaverse? 

1. Software and devices are just not ready. VR is still just evolving and AR is even more in its infancy. Not to mention a whole host of other issues with VR, most especially VR headsets. Most are heavy, uncomfortable, and don’t really make you say, “Let me walk around in this all day.”
2. Another big roadblock is the lack of extensive compute power and there are many issues which are causing this issue, including physical limitations, increasing data regulations, costs and a lack of a computing-supportive ecosystem. 
3. Plus we need people to actually design, build, run, and maintain the metaverse. And there are certainly not enough engineers, designers and network admins to do so. In fact this is a common occurrence in the IT sector, where businesses say that the lack of talent is the biggest barrier to adopting 64% of new technologies.
4.  Also along the lines of computing: a robust edge computing infrastructure is missing. This can be aided by smart city networks as those develop. 
5. Realistic graphics and images to bring the virtual world to a new, believable level.
   
   Reuters Elizabeth Howcroft reflected recently, “so you might have seen science fiction films that are set in fully fledged metaverses or alternate digital worlds that are almost indistinguishable from real physical life, but that’s still the stuff of fiction at the moment, the most virtual spaces I’ve seen look more like the inside of video games than real life.”
   

If you’ve seen what some are calling the early stages of the metaverse – mostly in the form of video games such as Fort Nite, Roblox, and Minecraft – then you’ll understand and likely agree with Howcroft. While it works just fine that your kids are playing in an extremely artificial world that is too absurd to be real, this will not fly when businesses are trying to teach their employees how to maneuver heavy machinery, or when training doctors to perform surgery or actually performing surgery. For these situations, extremely realistic, true-to-life and actual visuals will be extremely important and vital for a functioning metaverse. 

So how do we do this? We’re so glad you asked. With visual data, and lots of it. 

Software already set to help build the metaverse

Remember when I mentioned Epic Games earlier? You probably remember them – they’re the video game company behind Fortnite. We wrote about them earlier this year when they acquired Capturing Reality, the creators of RealityCapture, a leading photogrammetry tool. 

In the article, our CEO Jeffrey Martin wrote, 

Epic Games, along with its larger competitors EA, Valve, Sony, and others, is among the ten biggest video game companies in the world. Their most famous product is Fortnite, which has helped propel them into this top ten list. What is more interesting is the 3D engine that Epic Games makes, which is the software that allows them to build these games which take place in 3d worlds. This is called Unreal Engine.

He goes on to explain how Unreal Engine is essentially a software that can create a simulation that is not able to be distinguished from reality. Right now it is used in the video game and movie industries, and he even says, “and more and more other industries as time goes on.” (*hint hint*) 

Not only did Epic Games acquire Capturing Reality, but they also acquired Quixel which is an enormous library of photorealistic 3D photogrammetry scans. 

Well it seems like time has gone on and now the time has come that other industries or other use cases start to take advantage of this photorealistic technology. 

All of this is to say that in order to have a metaverse where we can meet virtually with our colleagues, ‘travel’ to far-off places, or train to be an astronaut, we will need true-to-life environments without a video game vibe. 

What hardware is necessary to make the metaverse a reality? 

Someone who has been following the evolution of the metaverse for some time is Matthew Ball. He has been writing about this subject for several years now and is considered one of the experts in the field. 

In an article he published earlier this year, he divided the necessary hardware into two categories:consumer-facing and enterprise hardware.

Consumer-facing hardware for the metaverse

These are things that the metaverse users will need to have in order to access and participate in the metaverse. We mentioned a few earlier that some of the major players are already investing in and developing as we speak. This includes: VR headsets, mobile phones, and haptic gloves

For consumer-facing hardware, two of the bigger companies leading the way are Oculus and Hololens. Meta acquired Oculus some time back and have been working on making a sleek headset that is affordable, user-friendly, and extremely immersive (according to reviews). 

There are plans to eventually phase out Oculus and replace it with a pair of smart glasses, made in cooperation with Rayban, the thinking being that you can actually walk around the streets with a pair of Raybans rather than an Oculus headset (I would tend to agree).

While the Oculus has mainly been marketed to users for  VR and entertainment purposes, it’s rival Hololens 2 has been marketing itself as ‘enterprise-ready’ for training employees, such as engineers and mechanics, healthcare workers, construction, and education, through the Mixed Reality technology that it possesses. 

On top of the visuals, you need to be able to feel and touch your surroundings in VR, MR, and AR environments, as well as the looming metaverse. In order to do so, we need some incredible technology which pairs what we see with sensations in our bodies and enables us to pick things up and move them. 

You might have experienced this with game consoles which have haptic feedback abilities. This means the game controller can vibrate, shake, or apply motion to the user’s hand(s) during play to add an extra element of realism to the game. 

Besides game controllers, you can now find haptic gloves and even haptic vests or suits. There are some great videos on YouTube where you can watch people test out these various haptic wearables. We highly suggest checking those out. 

Enterprise hardware for the metaverse

This refers to the hardware which will be paramount in operating or creating the environments for virtual or AR platforms. He includes here: industrial cameras, projection and tracking systems, and scanning sensors. 

And when I read that, I can’t help but combine all of those parts together and get one clear answer – a high performing, high resolution, mobile mapping camera system – or better yet – a metaverse camera as part of a mobile mapping system.

Why the metaverse camera will come in the form of a mobile mapping system

Mobile mapping systems (MMS) are hardware systems that combines image sensors (aka cameras) with GNSS devices (GPS, IMU, INU), add in some serious storage capacity or ability to stream to an onboard computer, maybe top if off with LiDAR as well, strap it to the top of a moving vehicle and hit the road (literally).

MMS are capable of collecting data on the go, at city and highway speeds. This greatly reduces the workflow often necessitated by stationary surveying techniques, as well as reduces traffic caused by the same. Cars, vans, trucks, trams, trains, bikes, and buses can all be used to collect data, while keeping up with the flow of traffic. You can even add a one-two punch and install a MMS on top of an already in-use vehicle such as a city bus, tram, or garbage truck, reducing carbon emissions, and also maintaining a constantly refreshed view of the streets and buildings on that vehicle’s daily or weekly routes. 

Depending on the type of system, it might be focused on highly accurate location information for building maps or 3D reconstructions of a city street, manufacturing warehouse, or an entire city. This is vital when measurements need to be taken between the sidewalk and buildings, or a construction company wants to see how fast their builders are progressing with the project, without having repeated site visits.

Or the mobile mapping system might specialize in highly detailed images, with extreme resolution images. This can be especially useful for companies reviewing the wear and tear on street signs or bus stops, or to look for deterioration on infrastructure including utility poles and real estate. 

We recently did a piece on mobile mapping systems, so check that out for the top 5 mobile mapping systems for 2022.

How metaverse cameras will tie it all together

Remember our hinting at how Epic Games and its recent acquisitions factor back into all of this? Unreal Engine, Capturing Reality, and Quixel are going to make Epic Games the go-to for finding, accessing, and creating realistic environments which are unable to be distinguished from reality. 

But remember, that is the software. So we still need the hardware – in the form of enterprise-ready cameras plus image sensors and scanning sensors – to provide the visuals for Epic Games’ impressive library and software.

If we can expect to interact in the metaverse in a wide variety of scenarios, environments, locations, etc. we’ll need A LOT of street-level images, captured in the fastest way possible. And right now, mobile mapping systems are the way to go. 

Some MMS can capture data for the creation of maps or 3D models of a whole city in the time it takes to drive that area. Then through the use of high-powered photogrammetry software (Reality Capture), we will be able to start adding to the photorealistic 3D photogrammetry scans (Quixel) – all currently owned and operated by Epic Games. 

So let’s sum it up here:

+

+

+

Mobile Mapping Camera 

=

THE FOUNDATION

OF THE METAVERSE