tracking augmented reality

Cool video


These effects are done in post-production, but would be super sweet as mobile AR:

street tests from Najork on Vimeo.

September 21st, 2009

AR tech round up #1


Over the past week, we’ve looked into several “tech tools” for doing augmented reality applications. Here’s a round up of what we’ve found so far:

1. Marker-based AR

The one and only open-source tool available seems to be the ARToolKit library, of which many variation and flavors have been created using the GNU-licensed library to make AR development faster and in more areas, some of these include:

OSGART:  ARToolKit + OpenSceneGraph

OSGART is a library that simplifies the development of Augmented Reality or Mixed Reality applications by combining the well-known ARToolKit tracking library with OpenSceneGraph.

With OSGART, users gain the benefit of all the features of OpenSceneGraph (high quality renderer, multiple file type loaders, community nodekits like osgAL, etc.) directly in their augmented reality (AR), mixed reality (MR) or mediated reality applications.

An AS3 library for developing AR in Flash.


FLARManager is a lightweight framework that makes building FLARToolkit augmented reality applications easier. it is compatible with a variety of 3D engines (currently: Alternativa3D,Away3DPapervision3D, and Sandy3D), and provides a more robust event-based system for managing marker addition, update, and removal. it supports detection and management of multiple patterns, and multiple markers of a given pattern.


a very primitive and limited Processing library …made as a wrapper from another open source project jARToolKit which is a Java port of the original ARToolKit. It only works with an OpenGL sketch in Windows.

a “no-programming required” software tool to generate your own AR scene on desktop. From HitLab, developers of ARToolKit. Currently (Sept 09) is PC only.

Distributer of commercial license for ARToolKit

Our notes: we’ve experiment with using ARToolKit with Cocoa (our bread n’ butter) – not as easy as it sounds, but eventually got it to work.  The difficulty for all the AR development seems to be the 3D drawing part.  With just ARToolKit, you’d have to do the drawing in OpenGL yourself – which, perhaps, explains the existence of OSGART.  The Flash route probably gives the developer more flexibility with 3D drawing, although we haven’t tried it ourselves.

2.  Mobile AR

See earlier post on Locative AR, these are markerless AR using the smartphone’s data.

There are lots of development done for the Android phone already, but really the big game player is the iPhone (and is the one we’re most interested in).  Most of the dev for AR on the iPhone is limited to the developer themselves at this point (as of Sept 09), with a few apps that slipped through to the app store such as Yelp! Monocle and the Paris Subway Map.  Both are using the combination of GPS, magnetometer and accelerometer data, then overlaying the info graphic on the video feed.  Useful, maybe.  Suspect that the novelty might wear off after a while if doesn’t offer any more functionality, but all is still in development.

Acrossair claims they have iPhone AR apps ready and waiting for launch when 3.1 releases (which happened yesterday (10 Sept 09), but we’re not sure if Apple released the video API w/ 3.1).

On the Android:  Wikitude is doing lots of ground work for this type of stuff.  Layar as well.

Tools:  Wikitude API for Android, Layar API for Android

ARToolKit has a version of the library that works with iPhone, but not released yet.

3.  Marker-less AR

This is where augmented reality gets more exciting and a bit out of “geek-mode”.  The basis is the same, but using advanced camera vision technique, the software would be able to do image recognition and, without using the black-and-white markers, render the computer generated visuals on top of it.  Development and applications in this area are happening, but not as wide spread as the opensource marker-based (which is thanks to ARToolKit and others).  We assume that in the future, AR markers will be obsolete in a few years’ time.

Proprietary tools:
Total Immersion – lots and lots of commercial and advertising work for big companies from consumer products to expos, games, and film industries.  Impressive portfolio.  They also offer a packaged software.  PC-based and final results require DirectX plugin on the user part.

Metaio – offers PC-based software for developing AR applications.  Also PC based.

Code libaries:
As mentioned above, augmented reality is based on two things working together:  camera vision to determine where/what the user is looking at and rending 3D onto that video frame in real time.  This combination could be technically possible using OpenCV and OpenGL, both libraries packages are free and open for commercial usage.

Overall Note: camera vision requires a lot of resources, so while it is possible to do complicated AR scenes on today’s desktop computers, we’re wondering a bit about the capabilities on the smartphones, especially for the marker-less option.

September 11th, 2009

Posted in Tech

No comments

AR for architects


Great use of AR for architectural presentations:

via arvertising

September 5th, 2009

A good primer for Locative AR on mobile devices


Video explanation of the technology behind locative AR on mobile from GigantiCo blog:

September 4th, 2009

There are inspirations, but then there are also other things


Our path to the world of augmented reality probably began like most curious coders out there:  it started with a link to some YouTube video and a “hey that’s cool, let me figure out how they did that” and naturally this led to Googling, finding some opensource codes, staying up late and opening up 20 more browser windows, etc.

Wait, actually, maybe it was seeing that GE ad… or was it that Japanese video… or was it this blog post?

Anyway, there’s a lot out there, and it’s steadily gaining street creds despite the gimmicky nature of 3D graphics.  Now that the venerable iPhone is about to get some AR applications in the App Store (they said September, right?), the time seems right to pay attention to this thing.  We’re starting this website as a part of our R&D effort to collect resources, notes, and track the world of augmented reality development, mobile platforms and otherwise.  Who knows, it might be useful.

September 2nd, 2009

Posted in Notes

No comments

Back to top