augmenting

Here’s a good example to show that AR only looks good when done well with eye-popping graphics (not a surprise to anyone, really).

Where as the AR for YanMingYuan – a tourist attraction in China – has the idea and concept of bringing the ruins to life, it looks kind of fake and bordering on the cheesy side…

On the other hand the AR digital binocular station by NZ based MindSpace Solutions looks a bit more appealing (though, why not design the binocular to look more updated if they have to reconstruct it anyway?) and the integration of video content is always nice:

Note that in both cases, since it is using a stationary device, GPS or heavy video feed analysis might not even be needed to assemble this AR experience. As long as the computer knows the position of where the camera is looking (and we can calculate this using a rotary encoder), overlaying video and special fx onto the camera feed should be relatively simple.

It’s been a little more than a month (or is it two?) since mobile AR apps have started to make an appearance on the mobile market. And, it’s clearly not making that big of a buzz as a lot of AR developers and future forecasters were drumming it up to be back in early August. Not a big surprise, as these apps are really quite limited in their functionalities. Our biggest peeve: why overlay geo-data on camera view when it’s not all that accurate and it’s easier to use with a vector map view anyway? Having said that though, we’ll acknowledge that it has to start somewhere.

Augmented Planet, an extensive AR blog, has just posted a review of AR browsers. The author used several browsers to locate his local Indian restaurants, and the results are really not that great. Read the details here: http://www.augmentedplanet.com/2009/11/augmented-reality-browsers-head-to-head-part-1/

Here’s a bit of summary from the post:

Layar: “Out of all the applications tested Layar local search performed the best with the least amount of data errors, of the 8 restaurants shown I only spotted 2 that were misplaced in residential areas.”

Robotvision: “..The app however lists JPF Drum Tuition and Olga Piano Tuition as local Indian restaurants as well as plotting restaurants in residential streets. The augmented reality view has a number of problems, for starters there is no compass to tell you where things are making it really hard to find anything. The other problem is if places are close together then its impossible to select all but the front option”

Yelp: the two benchmark restaurants were relocated on the map and vanished from the map, respectively. “One last comment about the AR view is it’s so unresponsive, items in the view sit around regardless of where you point the camera then gradually slide away. Yelp is not without it’s mapping errors, further afield from my home location I noticed missplaced pubs, Chinese resturants and even a few resturants placed on the motoway.”

Urbanspoon:
“UrbanSpoon surprisingly for an application that is dedicated to eating and finding restaurants performed the worst with the most mapping errors. Brashwamy’s is completely missing despite having submitted it via the applications add a restaurant function several weeks back. Sharod is shown but also located in completely the wrong location.
…
Putting the application in to AR mode has a useful feature where the restaurants are represented by a bubble with the bubble colour reflecting the customer feedback. Clicking the bubble you can get the phone number, vote or add a menu with your camera. It’s just a shame that the data is so inaccurate that you’ll never be able to find the restaurant to see if it lives up to its hype.”

Granted we’re still in the early days of AR and the technology still need a little time to mature, but erratic data and bad user experience like these, if it becomes a norm with AR app, then we’ll be reduced to merely using AR as advertising gimmicks and this promised ‘terminator vision’ could very well fizzle away. Anyone remember VRML?

Layar announced last week their new 3D feature coming in November. This feature will enable 3D rendering for Layar’s API, so that you can have something like this:

on the mobile application developed using Layar.

Interesting because: while we’ve seen a lot of AR apps on the desktop using 3D renderings (from Total Immersions to all the projects using ARToolKits, FLAR.. etc.) the AR apps on mobile have all been limited to drawing flat sprites/billboards on top of live video. Also note that this is *not* using a marker but location data in order to draw the 3D objects. Could be interesting to learn how they determine camera translation and smooth out the jitter.

However, 3D in AR on mobile phone still has lots of ways to go. Layar 3D: “With the limited processing power of phones we try to keep 3D objects below 1000 polygons. The platform can process objects up to 5000 polygons but starts to slow down.” Which means we probably won’t be able to get a lot of smooth-looking stuff w/ the current smart phones, but alas, it is a step forward.

Also, this looks like something out of Fringe:

Source: http://layar.com/3d/

Over the past week, we’ve looked into several “tech tools” for doing augmented reality applications. Here’s a round up of what we’ve found so far:

1. Marker-based AR

The one and only open-source tool available seems to be the ARToolKit library, of which many variation and flavors have been created using the GNU-licensed library to make AR development faster and in more areas, some of these include:

OSGART:  ARToolKit + OpenSceneGraph

OSGART is a library that simplifies the development of Augmented Reality or Mixed Reality applications by combining the well-known ARToolKit tracking library with OpenSceneGraph.

With OSGART, users gain the benefit of all the features of OpenSceneGraph (high quality renderer, multiple file type loaders, community nodekits like osgAL, etc.) directly in their augmented reality (AR), mixed reality (MR) or mediated reality applications.

FLARToolkit
An AS3 library for developing AR in Flash.

FLARManager

FLARManager is a lightweight framework that makes building FLARToolkit augmented reality applications easier. it is compatible with a variety of 3D engines (currently: Alternativa3D,Away3D, Papervision3D, and Sandy3D), and provides a more robust event-based system for managing marker addition, update, and removal. it supports detection and management of multiple patterns, and multiple markers of a given pattern.

simpleARToolKit

a very primitive and limited Processing library …made as a wrapper from another open source project jARToolKit which is a Java port of the original ARToolKit. It only works with an OpenGL sketch in Windows.

BuildAR
a “no-programming required” software tool to generate your own AR scene on desktop. From HitLab, developers of ARToolKit. Currently (Sept 09) is PC only.

ARToolWorks
Distributer of commercial license for ARToolKit

Our notes: we’ve experiment with using ARToolKit with Cocoa (our bread n’ butter) – not as easy as it sounds, but eventually got it to work.  The difficulty for all the AR development seems to be the 3D drawing part.  With just ARToolKit, you’d have to do the drawing in OpenGL yourself – which, perhaps, explains the existence of OSGART.  The Flash route probably gives the developer more flexibility with 3D drawing, although we haven’t tried it ourselves.

2.  Mobile AR

See earlier post on Locative AR, these are markerless AR using the smartphone’s data.

There are lots of development done for the Android phone already, but really the big game player is the iPhone (and is the one we’re most interested in).  Most of the dev for AR on the iPhone is limited to the developer themselves at this point (as of Sept 09), with a few apps that slipped through to the app store such as Yelp! Monocle and the Paris Subway Map.  Both are using the combination of GPS, magnetometer and accelerometer data, then overlaying the info graphic on the video feed.  Useful, maybe.  Suspect that the novelty might wear off after a while if doesn’t offer any more functionality, but all is still in development.

Acrossair claims they have iPhone AR apps ready and waiting for launch when 3.1 releases (which happened yesterday (10 Sept 09), but we’re not sure if Apple released the video API w/ 3.1).

On the Android:  Wikitude is doing lots of ground work for this type of stuff.  Layar as well.

Tools:  Wikitude API for Android, Layar API for Android

ARToolKit has a version of the library that works with iPhone, but not released yet.

3.  Marker-less AR

This is where augmented reality gets more exciting and a bit out of “geek-mode”.  The basis is the same, but using advanced camera vision technique, the software would be able to do image recognition and, without using the black-and-white markers, render the computer generated visuals on top of it.  Development and applications in this area are happening, but not as wide spread as the opensource marker-based (which is thanks to ARToolKit and others).  We assume that in the future, AR markers will be obsolete in a few years’ time.

Proprietary tools:
Total Immersion – lots and lots of commercial and advertising work for big companies from consumer products to expos, games, and film industries.  Impressive portfolio.  They also offer a packaged software.  PC-based and final results require DirectX plugin on the user part.

Metaio – offers PC-based software for developing AR applications.  Also PC based.

Code libaries:
As mentioned above, augmented reality is based on two things working together:  camera vision to determine where/what the user is looking at and rending 3D onto that video frame in real time.  This combination could be technically possible using OpenCV and OpenGL, both libraries packages are free and open for commercial usage.

Overall Note: camera vision requires a lot of resources, so while it is possible to do complicated AR scenes on today’s desktop computers, we’re wondering a bit about the capabilities on the smartphones, especially for the marker-less option.