augmenting

Over the past week, we’ve looked into several “tech tools” for doing augmented reality applications. Here’s a round up of what we’ve found so far:

1. Marker-based AR

The one and only open-source tool available seems to be the ARToolKit library, of which many variation and flavors have been created using the GNU-licensed library to make AR development faster and in more areas, some of these include:

OSGART:  ARToolKit + OpenSceneGraph

OSGART is a library that simplifies the development of Augmented Reality or Mixed Reality applications by combining the well-known ARToolKit tracking library with OpenSceneGraph.

With OSGART, users gain the benefit of all the features of OpenSceneGraph (high quality renderer, multiple file type loaders, community nodekits like osgAL, etc.) directly in their augmented reality (AR), mixed reality (MR) or mediated reality applications.

FLARToolkit
An AS3 library for developing AR in Flash.

FLARManager

FLARManager is a lightweight framework that makes building FLARToolkit augmented reality applications easier. it is compatible with a variety of 3D engines (currently: Alternativa3D,Away3D, Papervision3D, and Sandy3D), and provides a more robust event-based system for managing marker addition, update, and removal. it supports detection and management of multiple patterns, and multiple markers of a given pattern.

simpleARToolKit

a very primitive and limited Processing library …made as a wrapper from another open source project jARToolKit which is a Java port of the original ARToolKit. It only works with an OpenGL sketch in Windows.

BuildAR
a “no-programming required” software tool to generate your own AR scene on desktop. From HitLab, developers of ARToolKit. Currently (Sept 09) is PC only.

ARToolWorks
Distributer of commercial license for ARToolKit

Our notes: we’ve experiment with using ARToolKit with Cocoa (our bread n’ butter) – not as easy as it sounds, but eventually got it to work.  The difficulty for all the AR development seems to be the 3D drawing part.  With just ARToolKit, you’d have to do the drawing in OpenGL yourself – which, perhaps, explains the existence of OSGART.  The Flash route probably gives the developer more flexibility with 3D drawing, although we haven’t tried it ourselves.

2.  Mobile AR

See earlier post on Locative AR, these are markerless AR using the smartphone’s data.

There are lots of development done for the Android phone already, but really the big game player is the iPhone (and is the one we’re most interested in).  Most of the dev for AR on the iPhone is limited to the developer themselves at this point (as of Sept 09), with a few apps that slipped through to the app store such as Yelp! Monocle and the Paris Subway Map.  Both are using the combination of GPS, magnetometer and accelerometer data, then overlaying the info graphic on the video feed.  Useful, maybe.  Suspect that the novelty might wear off after a while if doesn’t offer any more functionality, but all is still in development.

Acrossair claims they have iPhone AR apps ready and waiting for launch when 3.1 releases (which happened yesterday (10 Sept 09), but we’re not sure if Apple released the video API w/ 3.1).

On the Android:  Wikitude is doing lots of ground work for this type of stuff.  Layar as well.

Tools:  Wikitude API for Android, Layar API for Android

ARToolKit has a version of the library that works with iPhone, but not released yet.

3.  Marker-less AR

This is where augmented reality gets more exciting and a bit out of “geek-mode”.  The basis is the same, but using advanced camera vision technique, the software would be able to do image recognition and, without using the black-and-white markers, render the computer generated visuals on top of it.  Development and applications in this area are happening, but not as wide spread as the opensource marker-based (which is thanks to ARToolKit and others).  We assume that in the future, AR markers will be obsolete in a few years’ time.

Proprietary tools:
Total Immersion – lots and lots of commercial and advertising work for big companies from consumer products to expos, games, and film industries.  Impressive portfolio.  They also offer a packaged software.  PC-based and final results require DirectX plugin on the user part.

Metaio – offers PC-based software for developing AR applications.  Also PC based.

Code libaries:
As mentioned above, augmented reality is based on two things working together:  camera vision to determine where/what the user is looking at and rending 3D onto that video frame in real time.  This combination could be technically possible using OpenCV and OpenGL, both libraries packages are free and open for commercial usage.

Overall Note: camera vision requires a lot of resources, so while it is possible to do complicated AR scenes on today’s desktop computers, we’re wondering a bit about the capabilities on the smartphones, especially for the marker-less option.