Spatial Computing as a New User Experience: An Overview of Leading SDKs

Imagine walking into your living room, pointing your phone at an empty space, and seeing a full-size couch appear right in front of you as if it were really there.

Or looking at a machine in a factory and getting step-by-step repair instructions floating next to it. That’s the kind of experience spatial computing makes possible.

Spatial computing is a broad term for technologies that bring the virtual and physical worlds together. It uses motion sensors, computer vision, and AR/VR to create more natural ways to interact with apps and information, using hands, eyes, or voice.

What used to feel like science fiction is quickly becoming part of our everyday lives costing around $100 billion. As demand grows for more immersive interaction with digital, spatial computing is already transforming how we shop, learn, work, and play.

In this article, we’ll look at real-world examples of how spatial computing is changing user experiences, and explore the top SDKs (like Apple’s visionOS, Microsoft’s MRTK, Meta’s Presence Platform, and Niantic’s Lightship) that developers are using to build these exciting new apps.

What Is Spatial Computing and Why It’s Changing User Experience

According to Statista, the concept of spatial computing environments was spawned by the desire to bridge the gap between the real and the virtual world.

This way, spatial computing is a technology that overlays digital data onto the world around it to recognize the environment, respond to movement, gestures, or voice, as well as place digital information in the right place and make it feel more realistic and applicable.

Spatial computing underlies augmented reality (AR) and virtual reality (VR), which enable corresponding applications by using sensors, cameras, and smart software to figure out where people are, what they’re looking at, and how they’re moving.

Perhaps one of the most notable applications of spatial technology and extended reality on a large scale is Pokémon Go, which assembled over 200 million downloads within a short period after its launch.

Since the fourth quarter of 2024, the Pokémon GO app remains a stand-out performer, with more than 8.2 million downloads worldwide.

Estimated AR/VR Users, Statista

Why it matters for user experience:

Put simply, spatial computing implies creating a more true-to-life way of interacting with technology. Instead of having to tap on a screen, users can simply reach out and take a 3D object, gaze at something to find out more, or step back and visualize digital objects staying fixed in space.

This means:

• More natural interaction – using hands, eyes, or voice instead of buttons.
• More sensory-rich simulations – digital content feels part of the real world.
• Smarter apps – that know where you are and adapt to your surroundings.

Real-Life Use Cases Across Industries

A pivotal chapter in the spatial computer narrative unfolded with the introduction of Microsoft’s HoloLens in 2015.

This headset marked a significant leap by overlaying holographic images onto the physical world, bringing the best of both AR and VR which Microsoft called “mixed reality”.

Now spatial computing spans lots of industries and directions, offering new ways to interact with digital content through natural gestures, voice, and movement.

Retail

Shopping becomes more fun and convenient with spatial computing. Customers can try on clothes, see how glasses will look on their faces, or test makeup without having to try on the real thing.

Home equipment stores such as IKEA, in turn, allow people to view virtual furniture in their house through a phone or tablet app so they know how well it fits and looks before buying.

Other ways spatial computing helps in retail include:

• Interactive displays that respond when customers walk by
• Personalized offers based on where shoppers are in the store
• Extra product info shown through AR

Healthcare

Healthcare spatial computing helps doctors and students work way better. Instead of using 3D images that sometimes distort the view on the monitor, surgeons can now see 3D images right on the patient’s body using AR glasses.

Medical students can practice surgeries in VR, learning faster and safer. It’s also used in physical therapy, guiding patients through exercises, and giving them real-time responses.

Some benefits include:

• More precise surgeries
• Better medical training with realistic practice
• Improved rehab with interactive guidance

Manufacturing and Maintenance

In factories, spatial computing creates safer working conditions. Workers wear smart glasses that show step-by-step instructions right on the machines they’re fixing.

This hands-free service means fewer mistakes and faster repairs. It also lets workers see live data, such as temperature or warnings, overlaid on equipment.

It helps by:

• Showing live info on machines
• Predicting problems before they happen
• Visually identifying safety risks

Education and Training

Spatial computing makes learning more fun and useful. For instance, it is possible to study 3D models of human bodies or planets.

Training simulations allow individuals to practice tricky skills such as flying or emergency response without dangerous consequences.

In some cases, schools make use of mixed reality classrooms, where digital objects appear in the room to help explain lessons better.

Examples include:

• Virtual science labs and interactive models
• Realistic flight and emergency training
• Mixed reality lessons with digital overlays

Entertainment and Gaming

This is one area where spatial computing happens and makes it that much more exciting to experience with games and entertainment.

In complete immersive VR headsets like Meta Quest, it feels like you can move and interact with nature. It also blends elements of real and digital to enjoy the concert, the theater, and the museum.

Some key features are as follows:

• Location-based AR games
• Fully immersive VR experiences
• Mixed reality concerts and events

Architecture and Real Estate

In architecture and real estate, spatial computing helps people see and explore buildings before they’re built. Clients can walk through virtual homes or offices to check layouts and designs.

Real estate agents use AR apps to give remote tours to buyers. On construction sites, digital plans can be overlaid in the real environment to improve precision and teamwork.

It helps by:

• Letting buyers tour homes virtually
• Catching design issues early
• Improving collaboration on building projects

What Are Spatial Computing SDKs?

Spatial computing SDKs (Software Development Kits) are development environments through which programmers can craft applications where digital content blends into the physical world. Spatial computing SDKs include all the building blocks required to create immersive experiences, such as code libraries, APIs, and testing environments, so developers do not have to start from scratch.

In short, a spatial computing SDK allows developers to create AR glasses, VR headsets, or smartphone apps supporting augmented or mixed reality. It simplifies the process of:

• Mapping and understanding the physical environment (spatial awareness)
• Placing and anchoring 3D digital objects in real-world locations
• Tracking user position and movement in space
• Recognizing hand gestures, eye gaze, or voice input
• Rendering realistic visuals and spatial audio

It would be much more complicated and time-consuming to develop spatial apps without these SDKs.

In fact, all popular SDKs are provided by companies such as Apple, Microsoft, Meta, and Niantic and are utilized in everything from AR gaming and virtual meetings to healthcare simulation and navigating indoors.

Key Capabilities of Spatial Computing SDKs

The global spatial computing market was already worth $102.5 billion in 2022, and it continues to surge. That’s a result of tireless tech creation, growing adoption across sectors, and strong investor interest. But why is it growing so fast?

Most important of all is spatial mapping. Put simply, spatial mapping is the capability of the app to map the environment — walls, floors, and furniture — so that it knows where to place virtual objects.

For example, people can use an AR app to see a virtual chair in their room, and it will be on the floor instead of above it.

Furthermore, SDKs allow for position and head tracking, which means the app can track movements in real time. So if a person moves their head or stands up and walks around, their viewpoint in the app moves organically, just like it would in real life.

The other essential features are hand, eye, and gesture recognition, which people can use without a keyboard or touchscreen. They can grab, move, or rotate virtual items using just their hands, or even do things by looking at something or by voice command.

Besides, SDKs also stimulate anchoring and persistence which allow digital objects to be anchored in the real world even if users close down the app or come back later.

For example, users can leave a digital sticky note on their fridge, and it will still be there when they come back, just like a real one.

To deliver that added sense of presence, the majority of SDKs include realistic 3D graphics and spatial audio capabilities. So the objects look sharp and realistic, and the sounds come from the right direction, picking up a sound behind and turning to look where it went, for example.

Some SDKs (which is truly a nice bonus) also offer multi-participant experiences that allow those in different locations to interact with the same virtual scene at the same time, for instance, during virtual meetings, training sessions, or multiplayer games.

Lastly, there are also SDKs that provide occlusion support, where virtual objects may be seen behind real-world objects. For example, if your virtual cat is wandering around in your living room, it can wander behind your sofa rather than unnaturally in front of it.

SDK Overview: Building Blocks of the Spatial UX

The development process for traditional applications is undeniably complex, but it follows proven methods and uses well-understood technologies.

Meanwhile, AR/VR development is a relatively new direction, where the development cycle is more problematic due to the need for specialized skills, tools, and out-of-the-box thinking.

According to various surveys and polls, over 60% of developers rely on powerful SDKs that help them regulate the tough technical effects and make digital content feel like part of the real world.

Apple visionOS SDK

Apple’s visionOS SDK is the main toolkit for producing apps for the Apple Vision Pro headset. It brings Apple’s polished design and strong ecosystem into the world of spatial computing.

If you’ve already built apps for iOS or macOS, you’ll find this SDK familiar: it uses Swift, SwiftUI, and RealityKit to help create 3D content that feels reflexive and genuine.

Apps created with visionOS can run in a variety of formats: immersive environments, floating 3D windows, or room-scale environments. The SDK supports advanced interaction with just your voice, eyes, and hands — no controller needed.

What distinguishes visionOS SDK is how everything, from tracking to visuals, is just so flowing and of such high quality. It’s excellent for developers creating next-generation productivity apps, media experiences, or utilities that fit into Apple’s larger ecosystem.

Best for:

• Creating immersive apps for productivity, design, and media
• Taking full advantage of the Apple ecosystem (iOS, macOS, iPad)
• Fluid, gesture-based interfaces

Key features:

• Eye and hand tracking
• 3D windowed and full-screen app environments
• Tight integration with SwiftUI and RealityKit

Microsoft MRTK (Mixed Reality Toolkit)

Microsoft’s open-source software development kit for assembling mixed reality apps (mainly for the HoloLens headset but also for other Unity-supported platforms) is the Mixed Reality Toolkit or MRTK.

MRTK is built with professionals and enterprises in mind and offers all the components developers need to build apps for training, manufacturing, remote assistance, or medical use.

MRTK has ready-made structure blocks for hand gestures, voice commands, and spatial awareness. Plus, it helps create apps where users engage with holograms in a real environment without needing to program every interaction from scratch.

What sets MRTK apart is its strong support for real-world deployment in enterprise settings, where integration with enterprise systems matters the most.

Best for:

• Enterprise and industrial apps (like training or remote assistance)
• Fast development using Unity
• Apps for HoloLens and similar devices

Key features:

• Gesture and voice interaction
• Spatial mapping and understanding
• Cross-platform support (including VR and mobile)

Meta Presence Platform

Meta’s Presence Platform is the central SDK for developing apps on Meta Quest headsets (e.g., Meta Quest 2, Meta Quest 3, and Quest Pro). It’s all about making digital experiences feel more natural and immersive through the use of capabilities like hand tracking, voice, and mixed reality passthrough.

This SDK includes everything from advanced gesture recognition to room scanning, allowing users to interact with digital objects directly in their own physical environments. Developers can also use the Voice SDK to add speech commands and conversational AI.

The best thing about the Presence Platform is that it is such a wonderful platform for games, social apps, and creative responsive tools that are interactive. It’s a developer favorite when building entertainment apps, fitness experiences, or shared environments where becoming fully immersed is fundamental.

Best for:

• VR games and social apps
• Mixed reality experiences on Quest 2, 3, and Pro
• Natural, hands-free interaction

Key features:

• Full-color passthrough (on newer headsets)
• Hand tracking and voice input
• Realistic spatial anchoring

Niantic Lightship ARDK

Niantic’s Lightship ARDK is simply mobile AR in the real world. It’s the technology behind hit titles like Pokémon GO, and it helps creators make apps that respond to outdoor environments, landmarks, and real-world locations.

Features of lightship technology include environment mapping in real-time, shared AR in multiplayer, and anchoring of content to GPS coordinates. This means that users can travel to real cities, parks, or events and interact with content that developers have anchored there.

What makes Lightship stand out is its emphasis on mobile, outdoor, location-based AR. It’s great for building AR games at scale, city guides, or social AR experiences that multiple users can share together simultaneously in real time.

Best for:

• Outdoor AR games and social experiences
• Real-time multiplayer features
• Large-scale, location-aware apps

Key features:

• Shared AR maps and networking
• Real-world meshing and occlusion
• Support for both iOS and Android

Choosing the Right SDK: Criteria to Think of

Choosing the right SDK is a massive step in developing a spatial computing application. Every SDK differs in its support for devices, development tools, and user interactions, so your choice needs to match your goals and target users.

Basic selection criteria such as price, reviews, or popularity may not work because spatial computing solutions are highly specialized: what works well for one use case (like a mobile AR game) may not work for another (like an enterprise training tool).

Rather, it’s important to center around technical capabilities, hardware compatibility, developer support, and how well the SDK supplies the type of user experience you’re trying to create.

The major criterion is the platform and the hardware. If you target the Apple Vision Pro, the visionOS SDK is definitely your best bet.

It integrates nicely with Apple’s tools, such as Swift and RealityKit, and it’s equipped for spatial apps that take input from eye movement, hand gestures, and voice.

For producing an enterprise or training app on Microsoft HoloLens, MRTK is a perfect alternative because it supports Unity and provides out-of-the-box components for voice and gesture input, spatial mapping, and others.

If you’re concentrating on VR or mixed reality experiences for Meta Quest, the Meta Presence Platform gives you the capability to utilize features like passthrough, hand tracking, and voice control.

And for mobile-first, location-based AR experiences, Niantic’s Lightship ARDK is a solid option. It’s great for shared experiences in real-world spaces, like outdoor games or interactive tours.

Your app’s purpose also plays a big role in your decision. Each of the different SDKs is better suited to different kinds of experiences:

• visionOS: Ideal for spatial productivity apps, immersive media, or hands-free creation tools
• MRTK: Great for industrial training, remote support, or workplace tools
• Meta Presence: Great for gaming, narrative, and social VR
• Lightship ARDK: Great for multiplayer mobile AR and outdoor location-based experiences

Also, think about your development environment. If you already work with Swift, visionOS will feel familiar. If you’re more experienced with Unity, you’ll likely prefer working with MRTK, Meta Presence Platform, or Lightship, since they all have strong Unity support and sample assets to help you get started.

Consider how users will interact with your app. Will they use hand gestures, voice commands, or geolocation? SDKs vary in the types of input they support:

• Hand and eye tracking: visionOS and Meta
• Voice input: MRTK and Meta
• Geolocation and shared AR: Lightship

Finally, take into account the quality of documentation and developer support. SDKs from major tech companies like Apple, Microsoft, Meta, and Niantic tend to have well-maintained guides, tutorials, and community forums, which can really help when you’re troubleshooting or looking for inspiration.

Future Horizons for Spatial Computing UX

In the future, the way we interact with technology will change fundamentally, and spatial experiences will lead the charge.

Instead of flat screens, buttons, or taps, we will interface with digital information that emulates the real world.

One of the most prominent shifts will be in device control. Future apps will use eye movement, hand control by gesture, and voice commands instead of keyboards or touchscreens.

All these more natural interfaces will become easier and more enjoyable to accomplish with digital tools, almost as if you are just using your body or talking to someone.

Spatial applications will also become more intelligent about where people are and what’s nearby. Objects will be detected by devices in the environment, with items like furniture, lights, or other people, and react accordingly.

For example, virtual objects might step out of your way when someone walks by or provide you with some helpful information depending on what you’re looking at.

Another trend will be the ability to transition between devices without losing your spot. You can start to work on something on your phone, then move over to an AR headset to finish it in a 3D environment. Regardless of the device you use, be it a tablet, headset, or smart glasses, everything stays in sync.

The line between digital and real will also blur even more. In the future, we’ll be able to place digital objects, such as notes, tools, or artwork, in real spaces and have them stay there over time.

This could make things like learning, working, or playing much more engaging and collaborative.

At the same time, developers and designers will focus more on making such experiences accessible to everyone. Voice control, gesture customization, and assist features will be standard to make more people engage in this new way of interacting with technology.

Why Select SCAND for Spatial Computing Development?

We at SCAND have more than 20 years of experience developing bespoke software and are now assisting companies in making the most out of spatial computing.

If you plan to create an immersive training app, an interactive shopping experience, or a cross-platform AR/VR solution, our augmented reality development company knows how to turn your ideas into reality using the latest SDKs, such as Apple visionOS, Microsoft MRTK, Meta Presence Platform, and Niantic Lightship ARDK.

FAQ: Spatial Computing & SDKs

Q: What is spatial computing in simple terms?

A: Spatial computing is just one way of bringing together the physical and digital. It allows individuals to engage with 3D digital information as though it were in their world.

Q: Does spatial computing differ from AR or VR?

A: AR and VR are both types of spatial computing. AR puts virtual objects onto actual space, while VR creates an entirely virtual space. Spatial computing is the general term that covers all of the technology used in these experiences: sensors, tracking, 3D mapping, etc.

Q: Where is spatial computing technology used today?

A: Spatial computing already making a difference in a variety of industries:

• Healthcare: in surgery simulation and patient education
• Retail: try-on in virtual environments and AR product demonstrations
• Education: interactive learning experiences
• Manufacturing: hands-free instructions and 3D design previews
• Entertainment: games, concerts, and more

Q: Do I need to use special devices in order to use spatial apps

A: Some apps work on smartphones, especially those using AR. But for more advanced spatial interactions, you’ll need a headset such as Apple Vision Pro, Microsoft HoloLens, or Meta Quest. These devices have sensors to track your movement and surroundings.

Q: What are spatial SDKs and why do they matter?

A: SDKs (Software Development Kits) are toolkits for developers. They provide pre-made tools for hand tracking, voice commands, and 3D graphics, as well as save development time and allow apps to work adequately with hardware.

Q: What’s the right way to choose the appropriate SDK for my project?

A: Actually, it depends on what you create and what devices you target. Apple visionOS SDK is best suited for Apple Vision Pro apps.

Microsoft MRTK, in turn, gets along with enterprise or industrial use with HoloLens. Meta Presence Platform is the best match for VR or mixed reality experiences with Quest. Niantic Lightship ARDK — for mobile AR games and location-based ventures.

Q: Is spatial computing just for large companies and developers?

A: Not anymore! It’s becoming more affordable every day. There are free tools, open-source SDKs, and platforms that work on regular smartphones.