Building the Future with ARCore, Google's Augmented Reality SDK

ARCore, Google’s augmented reality SDK for Android, has largely taken a backseat to Apple’s ARKit SDK in the press. With the usual talk of a tech war popping up within days of Google’s ARCore launch announcement, it’s understandable that the two development environments are often talked about as if they are fundamentally different products engaged in a kind of augmented reality arms race.

Google Research

At the SilverLogic, we know it’s not a simple question of one or the other when it comes to technology as game-changing as augmented reality. We get excited about any tool that helps our team Make Ideas Happen for clients, regardless of the platform. ARKit is a revolutionary tool for building AR experiences on iOS, but less than 20% of mobile users are connecting to the world on an Apple device. Developers looking to reach the more than 2 billion monthly active Android devices around the globe can’t do it with ARKit, so ARCore has rapidly become the go-to SDK for building AR on Android.

What is ARCore?

Google

Until the launch of ARKit, Google had positioned itself as the unquestioned leader in augmented reality R&D with Project Tango. Tango focused heavily on hardware development for AR - specifically sensors - but Google announced that it would be shuttering the project effective March 1. Much of the tech from Tango that handled environmental mapping and motion sensing is present in ARCore, and Google’s pivot to focusing on mobile AR means it is placing a much heavier bet on its enormous installed Android base utilizing AR on its mobile devices more and more frequently.

Compared to iOS, building any SDK for Android is a tricky proposition. Since hundreds of devices run versions of Android, developers don’t have the advantage of working with Apple’s notoriously tight specifications: Basically, AR for Android has to work on a lot of different devices made by a bunch of different companies, each with its own specifications and hardware nuances. Because Android AR has to scale across so many different types of devices, certain ARCore features had to be developed in a somewhat more abstract way than what shows up in ARKit.

The Key Difference: Motion Tracking

Google Research

While some of its techniques look very similar to ARKit’s - both SDKs handle environmental mapping and directional light estimation in basically the same way - ARCore does a few key things differently. Its “hit testing” tracks coordinates relative to the phone’s screen by projecting rays into the device camera’s field of view, and ARCore also seems to handle object anchoring a bit better by allowing the user greater control over how inserted objects are placed into the environment.

But the biggest difference between the two SDKs - and the one Google is banking on as the key to ARCore's future - involves motion tracking.

ARKit does motion tracking via a technique called visual-inertial odometry, which merges the data coming in from the iPhone’s proprietary TrueDepth camera system cameras and its core motion sensors to create an extremely accurate sense of where the device is in the environment. However, this technique relies heavily on both the optics of the camera and the proprietary sensors in iPhone, which is not an option for the more device-agnostic Android OS.

Google’s solution here was to rely on inertial measurement unit (IMU) data, which is a combination of feeds from an accelerometer and a gyroscope. The big advantage in doing this is that IMUs are essentially in all modern smartphones, so scaling across relatively minor variations in hardware is less of a problem.

Google Research

From Google’s Developers blog:

As your phone moves through the world, ARCore uses a process called concurrent odometry and mapping, or COM, to understand where the phone is relative to the world around it. ARCore detects visually distinct features in the captured camera image called feature points and uses these points to compute its change in location. The visual information is combined with inertial measurements from the device's IMU to estimate the pose (position and orientation) of the camera relative to the world over time.

By relying on IMU data, Google has built the initial iteration of ARCore with scalability in mind. While the move means that ARCore will theoretically work on lots of different devices, issues can arise when the IMUs of specific devices aren’t all that great. For AR to look right (and not super glitchy), IMUs need to be extremely well-calibrated. Google tackled this by only supporting devices running Android 7.0 or higher - the newer Samsung Galaxy and Google’s own Pixel 2 - with ARCore, devices with the most state-of-the-art accelerometers and gyroscopes in them. At the moment, this means ARCore (like ARKit) only works across a few types of devices, but this will look very different in the next 12-24 months.

The Tortoise and the Hare 

Google targeted 100 million devices with its first build of ARCore, and one firm estimated that it reached about 71.5 million devices by the end of 2017. This is a tiny fraction of the total Android device market - between 3.5 and 5% of what’s out there. By comparison, ARKit was estimated to have an installed base of around 380 million in the same time window.

But Google is betting that slow (at least by tech industry standards) and steady is going to win this race. The intelligence firm ARtillry estimates that a single hardware replacement cycle (2.5 years) will make a staggering 92% of Android phones ARCore friendly. Combined with growth forecasts, this puts the number of ARCore-ready devices at 3.6 billion as soon as 2020. Compared with a similarly derived estimate of 850 million for ARKit, it looks like ARCore is poised to capture roughly the same 4:1 ratio of the AR market as the longstanding Android-to-iOS ratio.

Even with all the talk of competition between the two SDKs, that’s not where we focus at TSL: Why pick a single SDK when you can just learn and master both of them? We’re thrilled to be working with both: The most important takeaway from these figures has nothing to do with competition - ARKit and ARCore are just two means to the same end, which is making augmented reality a useful part of everyday life for billions of people around the world.

What's a fun way I can see ARCore in action right now?

We really can't recommend the AR Stickers app enough. Especially the Porgs. You owe it to yourself on a Friday to play around with the Porgs: