A New Wave of AR Realism with the ARCore Depth API

ARCore is a developer platform meant to create astounding Augmented Reality (AR) adventures for the users. Right from its inception, the emphasis had been to provide powerful APIs to developers, who would help them, create AR apps that combine the real and digital world together perfectly.

Google showcased a preliminary view of the ARCore Depth API last year. This API helps developers in the depth map creation using an RGB camera. ARCore Depth API is able to create depth images and depth maps by making use of a depth-from-motion algorithm. Since then, Google has been working closely with specified collaborators to examine how data provided by depth maps can make virtual objects appear precisely, either in front or behind real world objects. This would provide a more enriched and immersive real-life experience for AR apps.

Occlusion is a key feature of Depth API. It refers to the accurate appearance of digitized objects behind the real objects in the actual world we live in. For example, if a small digital cup has to be hidden behind a table, depth API would come in handy. Using Depth API, the cup can be effectively occluded behind the table. Without Depth API, the cup would overlap with the edge of the table and appear unrealistic. Thus, virtual objects carry a real look and feel when placed behind real objects.

Five Nights at Freddy’s AR: Special Delivery is a first person horror game. In this game, players are confronted with animatronic creatures (monsters) and they have to survive. Illumix, the developer of this AR app leverages the occlusion technology to provide a realistic experience while playing the game. Using occlusion, some animatronic monsters are hidden behind real world objects such as doors, tables, etc. giving a more terrifying experience to the players.

At Snap Inc., the team of developers and designers has amalgamated many such concepts into a group of Snapchat lenses such as the Dancing Hotdog and the Undersea World Lens, a latest Android exclusive.

In addition to the feature of occlusion, ARCore Depth API unleashes novel ways of how depth maps and images can be utilized. Some of these ways include:

  • Surface interactions
  • Environmental traversal
  • Realistic physics
  • And many others

The openly accessible GitHub project allows AR app developers to utilize these ideas for creating splendid realistic experiences for the audience.

Another AR app that combines the power of occlusion and depth is Lines of Play. This game by Google Creative Lab enables users to design domino art lines and place these in the real world environment around them.

Depth can be applied for other usage scenarios and not merely self-expression and gaming alone. A great example of this is the TeamViewer Pilot app. It utilizes depth to identify and solve real world problems more efficiently.