Apple's Augmented Reality Tool Kit Can Now Detect Walls And 2D Images In Beta

Apple has just rolled out an updated version of its augmented reality platform, ARKit, to developers in a beta version of its latest iOS software, and it includes some new features that true AR fans will likely be happy with.

The latest version of ARKit, which is version 1.5 supports vertical plane features. That's a technical way of saying that it allows sensors on an iPhone (or iPad) to recognize not only the floor you're on but the windows and walls around you as well. So when app developers now create AR apps, they can incorporate functionality that would use the vertical spaces around them as well as the horizontal space.

The new version of ARKit also features 2D image recognition, which means you can point your phone at a flatbed print or hang it on the wall and the ARKit app will display pop-up information. A flat image can also trigger some other action in the app. An obvious use case is for museums: You point your phone at a board, and the AR app identifies the image for you, then you can also generate a 3D object or a virtual experience in the app depending on the picture you are looking at. Let’s say, for example, the Apollo 13 wall art released a full virtual tour of a lunar module when the iPhone pointed to the wall art. And finally, ARKit will now support 1080p HD images, which was not the case before.

Apple first announced ARKit, its augmented reality platform for iPhone and iPad, at WWDC in June last year; although it was officially launched in September. Before that, app makers used 2D or flat overlays in apps (like Pokémon Go), but apps didn't use advanced depth sensors or computer vision technology. With frameworks like ARKit or ARCore from Google, they are.

Since Apple first announced ARKit, over 2,000 App Store apps have been created with ARKit functionality. It might not sound like a lot compared to the size of the App Store, but it's a lot of augmented reality apps.

The biggest change in functionality in this new version of ARKit is almost certainly the support for vertical planes. Google's ARCore supports two types of horizontal planes (horizontal up and horizontal down), but as far as we know, it does not support vertical planes. Threads on GitHub indicate that this is a high-demand feature for ARCore.

The consumer beta of iOS 11.3 and ARKit 1.5 is expected to go live in the next few days, though Apple hasn't said when.