I always wanted to test out the AR possibilities that smartphones have nowadays. With all these fancy Virtual Reality headsets coming, I wondered if it is possible to create a similar experience while using the front camera of a phone. To archieve this, I simply cut a hole in my cardboard so that the camera could stick out.
360° tracked AR
Google offers an Unity Framework if you want to use the accelerometer data to track the head movement. In a first test, this worked perfectly well. I had a box floating in front of me. Even the cardboard push button worked and I was able to interact with the box. To make this whole thing go Augmented- instead of Virtual Reality, I cut the hole and put a video feed into the background of my scene. By the way, my test Phone is a Samsung Galaxy S7. Here is a list why this did not work at all:
- the camera resolution was way too bad, so the background was fading completely
- the accelerometer tracking is not fast and accurate enough to turn the ingame camera as fast as the camera screen was
- the phone couldn’t keep up with the needed CPU power to display the video feed while continiously calculating the position of the box (I had like 10 fps)
To sum it up: A todays phone is not capable to generate a nice AR 360° experience. Sure, there might be tweaks here to improve the performance but not in a way that the outcome would be a satisfying experience. Is this the reason google didn’t make their Daydream View AR ready?
Augmented Reality without 360° tracking
This seemed like traveling back in time several years. First I tried that new fancy VR/AR Headset tracking stuff and now I am going back to hold your phone in your hand and look at it normally. Okay, who needs a Headset to use 360° AR? You obviously can look at it normally too, just like a youtube 360° Video. Sadly, the problem with the tracking speed of the accelerometer stays. The object was moving too slowly to keep up with the camera. Additionally, if you shake your phone slightly, the object wiggles a bit more than the camera. It just felt like its attached to a rubberband.
I had to go one step further back. Augmented reality with your phone, without 360° tracking. The key point here is, that it is possible to track objects through your phone. So unlike the floating cube, you are able to pin a gameobject directly on a surface or object. No wiggles, no rubberband. The magical tracking of images or objects can be archieved through the vuforia plugin.
With this technique I was able to create a very nice application in no time. We are creating a custom advent calendar every year and send them to our customers. One was just lying around. Take a look at what I did with it here.
If you want to see how I made this app, please come back later or hit me up. The tutorial is on its way.