What’s Next for Augmented Reality
by Owen van Dijk, Firstborn
Augmented reality has enjoyed a massive resurgence in recent years as advances in technology have made it easier for developers and consumers alike to embrace it.
The introduction of software frameworks like Apple’s ARKit and Google’s ARCore has lowered the bar for developers to create markerless apps that augment the real world with virtual content. New hardware such as Microsoft’s HoloLens 2 and the Magic Leap have shown us what “Always On” AR looks like, and consumer apps like Snapchat, TikTok, and Instagram have introduced AR into the lives of millions of users without ever mentioning the technology itself.
We’ve also seen the expanding potential in marketing and advertising that AR represents thanks to brilliant campaigns from some of the world’s biggest brands. Burger King set competitors’ billboards on fire through the power of AR and Foot Locker reinvented the sneaker drop with their AR scavenger hunt campaign for Nike’s highly coveted “Court Purple” Lebrons.
From innovative ad campaigns to fun time-passers like Snapchat filters to useful tools like Google’s AR-enabled navigation, it seems that every day there is a new application for augmented reality, and the future of this technology is very exciting.
The Technology Behind Augmented Reality
The magic of markerless augmented reality is made possible by complex and computationally intensive computer vision algorithms. In order for those algorithms to correctly project virtual content into the real world, they must perform three primary tasks:
They must determine where the device is in space, known as localization.
They must understand what the world and its geometry look like by capturing sparse “point clouds”.
They must see the world the way humans do by detecting and recognizing objects and images.
It was only recently that our mobile devices became powerful enough to handle those complex tasks, and the algorithms behind AR only get more and more complex and advanced each year. For instance, recently Apple announced ARKit 3, with cutting-edge new features like motion capture and people occlusion — the ability to detect humans so virtual content is projected behind and in front of them.
But advanced algorithms are only one part of the equation. Good augmented reality also requires virtual content that looks and interacts as realistically as possible, and real-time 3D game engines like Unity and Unreal Engine are pushing the boundaries of what’s currently possible. Real-time raytracing, a technology that seemed impossible only a few years ago, allows these engines to render and blend images with the real world with maximum realism. For example, Unity recently released a video showcasing a virtual BMW next to the real thing, and many people couldn’t correctly identify which was which. Currently, this does require powerful desktop hardware, but it will be a matter of time before it’s available on mobile devices.
Finally, for virtual objects to exist persistently in the real world and facilitate more efficient processing and projection, companies are utilizing a concept called the AR Cloud. In the AR Cloud, the point data captured by all individual devices is stored in the cloud and reused and shared in real time with every other AR user in the same physical location. That can significantly reduce the processing power required from each device, but unfortunately, most vendors are keeping their collected data to themselves, limiting the AR Cloud’s impact.
The Future of Augmented Reality
There are a lot of sceptics that dismiss augmented reality outright. They say AR is a feature rather than a product; they deride it as nothing more than entertainment, useful only for face filters and games; many criticize the UX or point out that users are only willing to hold up their phones for a few minutes at a time. These arguments are as old as the technology itself, and yet AR continues to become more ubiquitous with over 70 million users reporting to have explored AR in the last 12 months. However, that doesn’t mean there aren’t any drawbacks.
There are significant issues with the current state of AR that need to be addressed. Privacy issues when capturing data and the question of who owns the rights to AR Cloud data are genuine concerns. There are also ethical concerns with the ways constant improvements in machine learning could be combined with augmented reality. The world has already seen the Mona Lisa come to life thanks to machine learning, so it isn’t hard to imagine a future in which hyper-realistic AI driven personas could be continuously projected into our world, fighting for our attention – a scenario right out of an episode of Black Mirror. Will AR benefit all of us or only the privileged few?
While these are valid points, most critics are missing the point. In many ways, AR is a huge part of the future of computing. Just as the smartphone has changed the way we communicate, find a ride home, and order dinner, AR is changing how we interact with computers. Thanks to AR, the days of pointing, clicking, and dragging are coming to an end. Instead, AR will enable us to use our voices or virtual hands to control objects projected directly into our real-world surroundings.
Industries as varied as healthcare, industrial manufacturing and retail will be transformed by AR. Imagine a remote surgeon planning and executing an operation using AR glasses projecting a virtual patient in meticulous detail, or industrial designers working on a new product in a shared real-time design sessions via telepresence and AR, or augmented try-on’s allowing consumers instant visualization before having to make a buying decision. There are many unknown and untapped potential opportunities with hands-free always on AR glasses — everyone will have access to contextual information about everything around them, all in the blink of an eye.
Truth to be told, the technology isn’t there yet. The algorithms need to get better, the rendering needs to be more realistic, and the hardware needs to be smaller and more powerful. But despite the current limitations of the technology and the valid concerns surrounding it, AR is being adopted by both businesses and consumers at a rapidly increasing rate making it undeniable that Augmented Reality is here to stay.
About the Author: Owen van Dijk is a Principal Software Architect at Firstborn where he develops and architects digital experiences, platforms & products for global brands through the innovative use of software and emerging technology. He brings 15+ years of international experience as a technologist for clients such as Apple, Adidas, Absolut, Jaguar Land Rover, Red Bull, Supercell & Toyota. He works and lives in New York.