CHAPTER 2
Spatial Computing 101

Like any place in Reality, the Street is subject to development. Developers can build their own small streets feeding off of the main one. They can build buildings, parks, signs, as well as things that do not exist in Reality, such as vast hovering overhead light shows, special neighborhoods where the rules of three-dimension spacetime are ignored.

– Snow Crash by Neal Stephenson1

Chapter Cheat Sheet Powered by AI

I asked an AI to create a cheat sheet for this chapter. If you only have five minutes to spare, here are the three must-know insights to help you level up your spatial computing knowledge.

• Spatial Computing History: Spatial computing has evolved significantly since its inception in the 1980s, initially centered on geospatial information within the Geographic Information System (GIS) community and later expanded to include interactive 3D environments in the 1990s through innovations in virtual reality.
• Integration of Multiple Technologies: Modern spatial computing encompasses a broad range of technologies, including augmented reality (AR), virtual reality (VR), and mixed reality (MR), which collectively enable computers to perceive, interact with, and navigate three-dimensional spaces. This integration is enhancing applications in numerous industries, such as automotive, health care, and entertainment.
• Evolution of User Interaction: Spatial computing is shifting how users interact with digital information, moving from traditional flat screens to immersive three-dimensional interactions. This is facilitated by sophisticated sensor technologies and artificial intelligence, which together enable more natural and intuitive user experiences.

 Want to talk to an AI about this book? Scan the QR code or visit ai.thenextdimensionbook.com in your browser to access The Next Dimension Book GPT.

40 Years in the Making

To many, “spatial computing” is a new term recently made popular by Apple as part of its launch of the Apple Vision Pro in June of 2023 as its “one more thing.” “Introducing the era of spatial computing” was a campaign tagline for Apple’s first MR headset, which it refers to as a “spatial computer.”2 But while the 2020s will be looked back at as a major milestone for spatial computing and certainly a major leap forward toward its mainstream impact, this wave of computing goes as far back as the 1980s.

According to Wikipedia, the GIS community first used the term “spatial computing” in the mid-1980s to describe computing large-scale geospatial information.3 Geospatial roots make sense for spatial computing, seeing that mapping the world around us is fundamental to this next wave of computing. At that time, the GIS community was focused on mapping much larger areas versus the more human-centric scale that spatial computing also encompasses today.

In the early 1990s, when VR made its way out of academia and the military into its first commercialization attempt for consumers, the definition of spatial computing moved beyond its geographic use to encompass machine-to-human interactions. A start-up from Seattle named Worldesign used spatial computing to describe how people interact with three-dimensional environments.4 Specifically, Worldesign used the term for its immersive CAVE-like space known as the Virtual Environment Theater, which simulated a flight over the Giza Plateau in 3000 BCE. Robert Jacobson, who was the CEO of Worldesign, has said that this definition of spatial computing was largely influenced by the research conducted at the University of Washington’s Human Interface Technology Lab, which was under the leadership of Thomas A. Furness III, who has been referred to as the grandfather of VR.5

But perhaps the most frequently cited definition of spatial computing in recent times is from the eponymous 2003 thesis by MIT researcher Simon Greenwold.

In his thesis, Greenwold states:

Spatial computing is human interaction with a machine in which the machine retains and manipulates referents to real objects and spaces. Ideally, these real objects and spaces have prior significance to the user. For instance, a system that allows a user to create virtual forms and install them into the actual space surrounding him is spatial computing. A system that allows a user to place objects from his environment into a machine for digitization is spatial computing. Spatial computing differs from related fields such as 3D modeling and digital design in that it requires the forms and spaces it deals with to pre-exist and have real-world valence. It is not enough that the screen be used to represent a virtual space – it must be meaningfully related to an actual place.6

Greenwold’s human-centric explanation still holds true decades later.

Spatial computing promises us a computing experience beyond the screen where our digital interactions take place in 3D space. The essence of spatial computing is to integrate digital information with our physical environment in a seamless and interactive manner. By blending the virtual world with the physical world, computing can then perceive, interact with, and navigate three-dimensional space. This greatly enhances applications in every industry by providing more intuitive, efficient, and immersive ways to interact with technology.

Today, spatial computing has become synonymous with AR and virtual reality. While AR and VR are examples of spatial computing, it is more appropriate to think of spatial computing as an umbrella term for a broad range of technologies. These technologies all enable computers to be present in and perceive our physical real-world space. Spatial computing ushers in a brand-new wave of applications, including AR, VR, and MR (together often referred to as extended reality, or XR), as well as autonomous vehicles, virtual assistants, robotics, and the Internet of Things (IoT).

AR and VR devices represent the next phase of consumer hardware in the era of spatial computing, moving us beyond the smartphone. XR technologies benefit from all aspects of spatial computing, leveraging the computer’s new sense of awareness to edit our perception of reality. This can be achieved by changing our experience of the physical world with the blending of digital content with the physical real-world experience, such as AR and MR or completely transporting us into a simulation that removes our physical world completely, as in VR.

Autonomous vehicles and robotics use technologies such as sensors, AI, and in particular, machine learning, to understand and navigate surroundings and perform complex tasks in the physical world. This includes manufacturing robots that can work alongside humans, drones for delivery and surveillance, robots used in surgery, machines that vacuum our houses, and, eventually, androids in our homes.

Virtual assistants and the Internet of Things are examples of technologies that benefit greatly from the context spatial computing brings. In smart homes, spatial computing can help devices understand the layout and context of their environment, improving how they automate tasks and respond to user needs. For virtual assistants, spatial computing enhances their ability to interact with users through more natural, context-aware responses.

While we are still in the early days of the spatial computing era, it is important to look back and understand that we have been on this journey for a while now. The innovations and advancements of the last few decades have made it possible for us to see the opportunity in this next wave of computing more clearly. Our job at hand is to continue this evolution by investing in technologies and applications that will allow spatial computing to deliver on its promise.

The Spatial Computing Tech Stack

To understand what opportunities there are for your business in spatial computing, it is important first to understand its technology stack. While there are various perspectives on the tech stack within the computer programming world, this approach is designed to provide you with what you need to understand from a practical business standpoint.

A technology stack is everything you need to build and run an app. It includes the physical computers or hardware where the app runs to the various software components that handle different tasks. The software stack has two main sides: the front end, often referred to as the client, is what you see and interact with on your device, and the back end, often referred to as the server, works behind the scenes to manage data and ensure everything runs smoothly. All parts of this stack are necessary to enable applications and use cases for your business.

HARDWARE AS THE FOUNDATION

If we consider the structure of a house as an analogy for the spatial computing tech stack, the hardware would be the foundation. The previous waves of computing introduced groundbreaking consumer devices such as the personal computer, laptop, feature phone, and, eventually, the smartphone. Each played a pivotal role in shaping user interactions and capabilities, paving the way for more sophisticated applications.

Similarly, this new computing wave is introducing categories of devices designed for everyday use. Like the wave before, we will see these hardware options evolve over time. Among these are MR headsets. These headworn wearables offer both AR and VR experiences,...