From FOV to Inside-Out Tracking: A Beginner’s Guide to VR Vocabulary

VR, AR, MR, and XR

Alright, four acronyms all using the letter R at the end. Each of these are something reality. Most of the time they just get lumped together, especially when talking about things like market growth or usage statistics. Let’s go into each in detail.

XR – Extended Reality

First up is the most straightforward. XR, also known as extended reality, is an umbrella term that encompasses a range of technologies that provide users with immersive experiences, including virtual reality (VR), augmented reality (AR), and mixed reality (MR). These technologies use computer-generated content to create a simulated environment that users can interact with. XR has become increasingly popular in recent years, with applications across a wide range of industries, including gaming, education, healthcare, and retail. Let’s dig into the specific types now.

VR – Virtual Reality

VR is up next, which you probably already know. It stands for virtual reality. VR applications are designed to immerse you in a totally separate reality from our own. The whole idea is to get you to suspend belief and interact with an artificially constructed world. VR mostly is experienced through headsets that take up your full range of vision and add audio. A growing number of companies are developing cool technologies to add in support for other senses like touch, smell, and even taste. We’ll be posting more about some of these awesome advancements as they grow.

Currently, the majority of the public thinks of games whenever VR is mentioned and, while that is true, there are many other use cases. In the medical fields, VR is being used to train staff and allow surgeons to practice operations before the actual procedure. In some instances, it’s even being used to help patients during rehab or encourage health and wellness through gamifying exercise or providing immersive landscapes. Training in VR is gaining traction overall as entry costs have decreased. Pilots, police, and even Walmart has increasingly been using VR in their training programs to name a few.

Now take that same technology and start applying it to recreational activities. Games are coming out every day for people like you and I to enjoy in VR. You can go fishing with your friends, play a tabletop game together, or hang out with people from around the world if you would like. I think that’s enough about VR for the moment, let’s move on to AR and MR.

AR – Augmented Reality & MR – Mixed Reality

These two are close. Both are very similar to each other, it’s mostly about where your focus lies and what the application does that makes the difference. While VR mostly uses full headsets, AR/MR use a wide variety of hardware. Some do use the same headsets as VR but with cameras positioned to let you see what’s around you. Other ways both are usable is through phones, glasses with displays, and even windows. Let’s look at augmented reality first.

Augmented reality adds onto the real world. Your focus is still on your physical surroundings but there’s digital pieces added on. Pokemon Go is the usual go to example. You explore your surroundings to find Pokemon that are added in through AR. Other examples could be playing cards that, when viewed in AR, have digital figures that pop out or overlays that tell you more about landmarks you’re looking at.

Mixed reality, on the other hand, combines physical and virtual elements in a more connected way. While AR is more ‘virtual overlaid on picture’ MR gathers data around you and allows the technology to interact with the surroundings. Think about how different Pokemon Go would be if you could chase Pokemon around corners or they’d disappear behind a tree. Or how about setting up new wallpaper for your house and being able to tour every corner? That’d fall into MR as well.

It’s obvious at this point that AR and MR are both closely related. In some cases, there may be aspects of both in applications that further blur the lines. Regardless of which it is, they’re both a merging of the virtual and physical that companies are trying to get just right.

FOV – Field of View

Field of view (FOV) refers to the extent of the user’s visual field in the virtual environment. It determines how much of the virtual world the user can see and plays a crucial role in creating a sense of immersion. A wider FOV provides a more immersive experience, as it enables the user to see more of the virtual environment and reduces the feeling of being trapped in a box. The FOV of a VR system is determined by the design of the headset and the lenses used to project the images onto the user’s eyes.

Degrees of Freedom

Degrees of freedom (DoF) refer to the number of independent ways a user can move within a virtual environment. The more DoF a VR system offers, the more freedom the user has to move around and interact with the virtual world.

In older VR systems, users typically had 3 DoF, which allowed them to move their head up and down, left and right, and forward and backward. These systems were often limited to seated experiences and were designed primarily for gaming and entertainment.

Newer VR systems offer 6 DoF, which adds the ability for users to move their head and body in all six directions – up and down, left and right, forward and backward, and also roll, pitch, and yaw. This level of freedom allows for more natural and intuitive interactions with the virtual environment and opens up new possibilities for applications in fields such as education, healthcare, and training.

6 DoF is achieved through the use of positional tracking technology, such as inside-out or outside-in tracking, which allows the VR system to track the user’s movements in real-time. This enables users to move around and interact with virtual objects as if they were really there.

Head Mounted Display – HMD

Head-mounted displays (HMDs) are a key component of virtual reality (VR) systems, as they enable users to fully immerse themselves in a digital environment. An HMD typically consists of a head-mounted unit with two small displays or screens, one for each eye. These screens are usually positioned very close to the user’s eyes to create the illusion of a wide field of view, and are often combined with specialized lenses to enhance the overall VR experience. HMDs may also include built-in headphones or earbuds for audio immersion and an array of sensors to track the user’s head movements for a more realistic experience.

There are many different types of HMDs available, ranging from high-end devices designed for professional use, to more affordable consumer-grade products. HMDs can be tethered to a computer or console for a more powerful and immersive experience or may be standalone devices that do not require any external hardware. Some HMDs also include features such as eye-tracking or hand-tracking, which can further enhance the level of immersion and interaction in a virtual environment. In essence, when someone thinks of a VR headset, this is what is usually on someone’s mind.

Headstrap

VR headstraps are an important part of any VR headset, providing a comfortable and secure fit for the user. They typically consist of adjustable straps that go around the user’s head, along with additional padding or cushions for extended use. Headstraps play a crucial role in ensuring that the VR headset remains stable and properly positioned on the user’s head, which is essential for an immersive VR experience. There are many different types of headstraps available, ranging from simple elastic bands to more complex systems with ratcheting straps and built-in headphones.

Additionally, some headstraps hold the headset just above the face to remove pressure from around the eyes. Going a step further, the rear of some straps have a battery or other counterweight. This can be very important for some people who are more sensitive and can give anyone a better, more even experience.

Facial Interface

VR facial interfaces are an important part of a VR headset that comes into direct contact with the user’s face. They typically consist of foam padding or cushions that sit against the user’s cheeks, nose, and forehead, helping to provide a comfortable and secure fit while preventing light leakage. A good facial interface is crucial for both comfort and hygiene, as it can help to distribute the weight of the headset more evenly across the face and prevent the build-up of sweat or bacteria. Some VR headsets feature interchangeable facial interfaces, allowing users to easily swap out different sizes or materials depending on their needs. Silicone is a very common covering that fits most of these requirements and is used in many interfaces for superior comfort and hygiene.

Prescription Lenses

If you wear glasses, you probably will want to look into a set of prescription lenses to add onto your headset. While most headsets have some room or extra pieces to make glasses work, I’ve personally scratched lenses trying to wear my glasses in VR. It also further limits your field of vision more than the limitations of the headset. The set I picked up for my Quest 2 ran me about $60 but I already had my prescription in hand from my optometrist.

Tethered HMD

Tethered headsets are virtual reality headsets that require a wired connection to a computer or gaming console to function. The wired connection allows for higher-quality graphics and faster processing, making it possible to run more advanced VR applications that require high performance. Tethered HMDs typically include a range of sensors and features, such as positional tracking and motion controllers, to create a more immersive VR experience. While the wired connection can limit the user’s mobility and create potential tripping hazards, tethered HMDs remain popular among VR enthusiasts and professionals due to their high quality and advanced capabilities. As technology continues to advance, it is likely that we will see more wireless VR solutions emerge to address the issue of mobility.

Standalone HMD

Standalone HMDs are VR headsets that operate without being tethered to a PC or console. They are self-contained devices that include all the necessary hardware, such as the display, processor, and battery, to power the VR experience. This allows for a more flexible and untethered VR experience, where users can move around and interact with the virtual environment without being restricted by wires or cables. However, standalone VR headsets often have lower processing power and graphics capabilities compared to tethered VR headsets, which can result in a lower-quality visual experience. Nonetheless, standalone VR headsets have become increasingly popular due to their convenience and portability, and advancements in technology have led to the development of more powerful standalone VR headsets that can rival tethered VR headsets in terms of performance.

Latency

Latency in VR refers to the delay between a user’s actions and the corresponding visual and auditory feedback they receive, and achieving low latency is a key challenge in VR development. If not handled with care, high latency can cause motion sickness, eye strain, and a reduced sense of immersion, which is why developers strive to minimize it by using high-speed displays, optimizing graphics performance, and reducing the processing time of tracking data. Additionally, developers must consider the latency of other components, such as audio and haptic feedback, to ensure a seamless and responsive VR experience. High latency, and the nausea it can cause, is especially problematic for fast-paced VR experiences such as first-person games.

Screen-Door Effect

The screen-door effect is a visual artifact that can occur in some VR headsets, where the user can see a grid-like pattern of fine lines or gaps between individual pixels on the display. This effect is caused by the space between the pixels being visible to the user, giving the appearance of looking through a screen door. The screen-door effect can be distracting and can reduce the overall visual quality of the VR experience. However, newer VR headsets with higher-resolution displays and more advanced optics have greatly reduced this effect.

Boundary Wall

VR boundary walls, also known as guardian or chaperone systems, are an essential safety feature in many VR systems. These walls help users stay within a defined area, preventing them from accidentally walking into walls or other obstacles in the real world while immersed in a VR experience. When users come close to the boundary walls, they are alerted through visual and/or auditory cues to help them stay within the safe area. Some VR systems even allow users to customize the shape and size of the boundary walls, making it easy to accommodate different room sizes and shapes.

Locomotion

Locomotion refers to how users move through virtual environments. There are various methods of VR locomotion, including teleportation, arm-swinging, and natural walking. Teleportation is a common locomotion method in VR, where the user can point and “teleport” to a different location in the virtual world. Arm-swinging is another method, where the user swings their arms to simulate walking or running. Natural walking is the most immersive form of VR locomotion, but it requires specialized equipment and a large physical space. VR developers must carefully consider the locomotion method for their applications, as different methods can have different effects on user comfort and immersion. Some users may experience motion sickness or discomfort with certain locomotion methods, so it’s important for developers to provide a range of options to cater to different user preferences.

Motion Tracking Systems

One of the key components of a seamless and immersive VR experience is the ability of the hardware to track your movement and translate it into the virtual world. There are several ways that VR headsets accomplish this, including inside-out tracking, outside-in tracking, and a variety of other systems. Each method has its own unique advantages and disadvantages, and the choice of tracking system often depends on the specific use case and application.

Inside-Out Tracking

Inside-out tracking is a way that VR headsets can track your movements without the need for any external sensors or devices. Instead, the headset has cameras built into it that capture your surroundings and use computer vision algorithms to figure out where you are and how you’re moving.

One of the advantages of inside-out tracking is that it’s much easier to set up and use compared to other tracking methods. You don’t need to worry about placing sensors or extensive calibration – you just put on the headset, set your play space, and start using it.

However, inside-out tracking does have some limitations. It can be less accurate than other tracking methods, especially in low-light environments or when there are a lot of reflective surfaces around. Additionally, if you move your hands or controllers out of the camera’s field of view, the tracking may become less accurate or stop working altogether.

Despite these limitations, inside-out tracking has become very popular in the VR industry because it allows for more portable and convenient VR experiences as well as cool features such as pass-through which we touch on below. With ongoing improvements in computer vision and AI technology, the accuracy and reliability of inside-out tracking are likely to continue improving in the future.

Outside-In Tracking

Outside-in tracking is a way that VR headsets can track your movements using external sensors or cameras. These sensors or cameras are typically placed around the room or mounted on walls, and they track the position of your headset and controllers as you move around.

Unlike inside-out tracking, which uses cameras built into the headset itself, outside-in tracking requires additional hardware and setup. This can make it more complicated to use, but it also provides more accurate tracking and better performance overall. Tracking like this can minimize or altogether stop issues like occlusion, where the camera loses sight of your hands or controllers.

However, outside-in tracking requires more setup and can be less portable than inside-out tracking. Additionally, it may not be ideal for people who have limited space or who want to use VR in different locations.

Overall, the choice between inside-out and outside-in tracking depends on your specific needs and preferences. If you prioritize convenience and portability, inside-out tracking may be a better choice. If you want the highest level of accuracy and performance, outside-in tracking may be the way to go.

Base Stations

Base stations, also known as lighthouses or sensors, are a key component of many VR systems that rely on outside-in tracking. These base stations emit infrared light that is detected by sensors on the VR headset or controllers, allowing the system to accurately track the user’s movement in three-dimensional space. Typically, base stations are mounted on walls or tripods in opposite corners of the play area, creating a field of view that covers the entire space. Base stations are highly precise and can track even small movements, making them essential for a many different outside-in tracking systems.

Magnetic, Inertial, and More

There are several other tracking systems used in VR besides inside-out and outside-in tracking. Magnetic tracking, for example, uses magnetic fields to track movement and is often used in high-end VR setups, such as those used for scientific research or military training. Inertial tracking, on the other hand, uses sensors to detect changes in motion and orientation and is often used in portable VR systems, such as those used for training or therapy.

Acoustic tracking is another method that uses sound waves to measure distance and position. This method is less common but has been used in some specialized VR systems. Finally, wireless tracking uses radio frequency signals to track movement and is often used in wireless VR setups, such as those used for gaming. While it’s not necessary to have a deep understanding of these tracking methods, it’s helpful to have a basic understanding of how they work as there are a variety of systems out there and some combine a few different tracking techniques.

Room Scale Tracking

Room-scale VR is a highly immersive subcategory of virtual reality that allows users to explore a physical room as if it were an extension of the virtual world. With the ability to walk, jump, and touch virtual objects all around them, users are granted an unprecedented level of freedom and interactivity in the virtual environment. Achieving this level of immersion requires the use of specialized sensors and trackers, which are strategically placed throughout the room to monitor the user’s every move. The VR headset or controllers worn by the user are then used to track their position and movements, allowing them to be accurately placed in the virtual environment and interact with objects within the room-scale space. Many headsets currently on the market support room scale tracking in one form or another.

Full-Body Tracking

Full body tracking is an advanced VR technology that allows users to track and reproduce their full body movements in a virtual environment. This technology typically uses multiple sensors or cameras to capture the user’s movements and translate them into a digital space, creating a more immersive experience. With full body tracking, users can walk, run, jump, and interact with virtual objects in a more natural way, increasing the feeling of presence and immersion. One of the most popular applications of full body tracking in VR is dancing. With full body tracking, users can move their entire bodies and see their virtual avatars mimicking their every movement, creating a highly immersive and entertaining experience.

Dancing in VR has become increasingly popular in recent years, with many VR applications and games designed specifically for this purpose. Some of these applications even offer virtual dance parties where users from around the world can come together and dance together in a shared virtual space. Full body tracking allows users to fully engage in the dance experience, feeling the beat of the music and moving their bodies in a way that closely mirrors real-life dance movements.

Eye Tracking

Eye tracking is a technology that enables VR systems to detect the movement and position of the user’s eyes. By tracking eye movements, VR headsets can determine what the user is looking at and adjust the virtual environment accordingly. This can lead to more realistic and immersive VR experiences, as eye tracking allows for more natural interactions with objects in the virtual world. Eye tracking can also be used for foveated rendering, a technique that reduces the rendering workload by only rendering the part of the display that the user is looking at in high resolution while rendering the rest in lower resolution. This can lead to significant performance improvements, making VR more accessible to a wider audience. However, eye tracking is a relatively new technology in VR, and not all headsets have it. As the technology continues to improve and become more widely adopted, it has the potential to greatly enhance the VR experience.

Facial Tracking

Facial tracking is a technology that enables VR systems to track the movements and expressions of the user’s face in real-time. This can be achieved through a variety of sensors and cameras placed on the VR headset. By tracking facial movements, VR headsets can create more realistic avatars and animations, and enable more natural communication in social VR applications. Facial tracking can also be used for emotional analytics, where the system analyzes the user’s facial expressions to determine their emotional state. This can be useful for a variety of applications, such as market research and psychology research. However, facial tracking raises privacy concerns, as it can capture sensitive information about the user’s facial features and expressions. As with any technology that involves personal data, it’s important to consider the potential risks and benefits before using facial tracking in VR.

Passthrough

Passthrough is a feature in some VR headsets that allows you to see the real-world environment around you without taking off the headset. While crossing into the AR realm, it can be useful to quickly check your surroundings or to interact with physical objects while still in VR.

Passthrough works by using cameras on the front of the VR headset to capture images of the real-world environment. These images are then displayed on the headset’s screens, allowing you to see your surroundings while still immersed in the virtual world. Some VR headsets also include the ability to switch between different levels of transparency, allowing you to adjust the amount of real-world visibility based on your needs.

Passthrough is particularly useful in situations where you need to interact with objects in the real world while still in VR. For example, if you need to type on a keyboard or use a joystick while playing a VR game, passthrough can allow you to do so without taking off the headset. As passthrough becomes more common, developers have even been creating experiences entirely based around passthrough like making rollercoasters in your living room.

Phones as Headsets?

While using phones as VR headsets can be an affordable and convenient option to test the waters of VR, the reality is that this approach has become less popular in recent years. One of the main reasons for this is that dedicated VR headsets offer a more advanced and immersive experience, with features such as advanced motion tracking and better graphics quality.

Additionally, many mobile devices are simply not powerful enough to provide a high-quality VR experience. While some high-end smartphones can provide a decent VR experience, many lower-end devices struggle to run VR apps smoothly and overheat during extended use.

Another challenge with using a phone as a VR headset is that it can be uncomfortable for extended periods of use. Some older systems had you hold the phone in front of your eyes for an extended period of time which was tiring and caused neck strain. Even with fully wearable mounts, some devices wouldn’t fit with some mounts or the headset itself didn’t fit properly, leading to discomfort or pain.

Despite these challenges, using a phone as a VR headset can still be a viable option for those on a tight budget or who are just looking to try out VR for the first time. However, as dedicated VR headsets become more affordable and offer better features, it’s likely that using phones as VR headsets will become less common over time.

Haptic Feedback

Haptic feedback refers to the use of touch sensations to provide feedback in VR experiences. This can include sensations such as vibration, pressure, and temperature changes that are designed to simulate the sense of touch.

Haptic feedback can be used in a variety of ways in VR, from simulating the sensation of firing a gun to providing tactile feedback when interacting with virtual objects. For example, a haptic glove could be used to simulate the sensation of holding and manipulating virtual objects, while a haptic vest could be used to simulate the sensation of being hit or pushed.

The use of haptic feedback can greatly enhance the sense of immersion in VR experiences, making them feel more realistic and engaging. Additionally, haptic feedback can provide valuable feedback to users, allowing them to better understand their virtual environment and interact with it more effectively.

However, haptic technology is still relatively new and there are some challenges to overcome. For example, current haptic feedback systems can be expensive and bulky, making them difficult to incorporate into consumer VR devices. Additionally, the technology is still evolving and there are limitations to what can be realistically simulated using haptic feedback.

---

Last updated 4/19/2023