Studierstube Tracker

Studierstube Tracker (StbTracker) is a computer vision library for detection and pose estimation of 2D barcodes. It is a successor to the well known ARToolKitPlus library. Its concept is very similar to that of ARToolKit (ARTK), ARToolKitPlus (ARTK+) and ARTag, but its code base is completely different. StbTracker has been written from scratch with high performance for PCs as well as mobile phones in mind.

How does it compare to ARToolKitPlus?

StbTracker has been developed from scratch after 4 years of experience with ARToolKitPlus. It was designed from ground up to support mobile phones as well as PCs. Hence, its memory requirements are very low (100KB) and processing is very fast (about twice as fast as ARTK+ on mobile phones). While ARTK+ follows a monolithic approach, requiring configuration at compile time, StbTracker is highly modular: Developers can extend StbTracker in any way, creating new features for it. Different to ARTK+, StbTracker is not open source though.

StbTracker does not share any source code with ARToolKitPlus. It is a separate project and has no legal connections to ARToolKit or ARToolKitPlus.

Feature Overview

StbTracker's main features over other solutions are:

  • Fully class-based API (see API documentation)
  • Many different marker types
    • All marker types can be used concurrently.
    • Template markers (ARTK-style)
    • ID-markers (simple-id & BCH)
    • DataMatrix markers (see example 2 below)
    • Frame markers (see examples 3 & 4 below)
    • Split markers (see example 5)
  • Highest performance on low-end devices
    StbTracker is the fastest solution for tracking on mobile phones. It is about twice as fast as ARTK+, which used to be the fastest marker tracking library so far. At the same time tracking on phones is much more stable (less jitter) than with ARTK+.
    Benchmarks show that up to 185 images per second can be tracked on a 312Mhz smartphone (see below).
  • Low memory consumption
    The memory footprint of StbTracker mostly depends on the actual camera resolution (1 byte per pixel required). For a typical phone setup with a video stream of 320x240 pixels in YUV12 format the memory usage is below 100KB.
  • Small Executable
    StbTracker.dll for Windows CE devices is only 240KB storage.
  • Up to 4096 id-based markers
    The BCH marker system allows up to 4096 markers that are reliably detected at no speed penalty due to large number of markers. No markers need to be trained.
  • Camera pixel-formats (RGB24, RGB32, RGB565, YUV12)
    StbTracker natively supports pixel formats that are common on mobile phones such as RGB565 or YUV12.
  • Variable marker border width
    The marker border with can be modified freely giving more design choices.
  • High-quality pose estimation gives more stable tracking (less jitter).
    StbTracker includes an Implementation of "Robust Planar Pose" by G. Schweighofer and A. Pinz
  • Support for MATLAB camera calibration toolbox
    Usage of the MATLAB camera calibration toolbox gives high accuracy camera calibration and works with any device (PC, mobile phone, etc).
  • Automatic thresholding (watch this video)
    Adapts fully automatic to changing lighting conditions.
  • Fully configurable at runtime
    All aspects of the tracking pipeline can be configured at runtime.
  • No limits to video resolution
    There are no limits to the size of input images (except for memory consumption). Arbitrarily sized images can be used for tracking. Images with 8 Megapixels and 2000 markers have been successfully tested.
  • Easy to extend
    StbTracker's was designed from ground up to be easily extensible. Every step of the tracking pipeline is configurable and can be exchanged with custom code - even when using the binary distribution of StbTracker.
  • Highly portable
    StbTracker's code base is highly portable since it does not depend on any external libraries. It further obeys restrictions specific to mobile phones only for optimal portability.

 

Platform Support

StbTracker currently runs on Windows XP, Linux and Windows CE. A Symbian port is planed.

DataMatrix Markers

DataMatrix is an ISO standard for 2D barcodes. StbTracker supports markers with embedded DataMatrix patterns. Hence, markers can store large amount of binary (e.g. simple 3D models) or text (e.g. URLs) data. See example 2 below.

Frame Markers

Frame-Markers are recent addon to StbTracker. Different to conventional markers, the data is encoded in the border itself, thereby allowing arbitrary content inside the marker. The marker can even just be a frame (no interior part) which allows putting it around objects to augmented (like a picture frame) rather then next to them. See examples 3 and 4 below.

Extendibility

StbTracker is completely modular. Every part of the pipeline is implemented as a separate class, called a "Feature". The following features make up the complete tracking pipeline:

  • Thresholding
  • Fiducial detection
  • ARToolKit
  • Corner filtering
  • Pose estimation
  • Pose filtering

 

StbTracker comes with at least one implementation for each feature:

  • Thresholding
    • ThresholderAuto
      Automatically adjusts to varying lighting conditions.
  • Fiducial detection
    • FiducialDetectorRect
      Detects rectangles in the image
  • Marker detection
    • MarkerDetectorSimpleId
      Checks if fiduals found are markers of type simple-id.
      Implemented in floating-point as well as pure fixed-point.
    • MarkerDetectorBCH
      Checks if fiduals found are markers of type BCH.
      Implemented in floating-point as well as pure fixed-point.
    • MarkerDetectorDataMatrix
      Checks if fiduals found are markers of type DataMatrix.
      Implemented in floating-point as well as pure fixed-point.
    • MarkerDetectorDataTemplate
      Checks if fiduals found are markers of type template (ARTK-style markers).
      Implemented in floating-point as well as pure fixed-point.
    • MarkerDetectorSplit
      Checks if fiduals found are markers of type template (ARTK-style markers).
      Implemented in fixed-point.
  • Corner filtering
    • CornerFilterDESP
      Filters corner position using the DESP (Double Exponential Smoothing Prediction) algorithm.
  • Pose estimation
    • PoseEstimatorLM
      Pose estimation using non-linear refinement (Gauss-Newton iteration).
      Implemented in floating-point as well as pure fixed-point.
    • PostEstimatorRPP
      Pose estimation using the Robust Planar Pose algorithm.
      Only implemented in floating-point (not suitable for mobile phones).
  • Pose filtering
    • PoseFilterDESP
      Pose filtering using the DESP algorith

Examples

 


Example 1 : Benchmark image. Tracked at a speed of
185 images/sec on a Motorola Q phone with 312 Mhz.

 


Example 2 : Tracking a DataMatrix Marker.
(encoded message: "http://www.imagination.at/")

 


Example 3: Tracking and augmenting a Frame-Marker.

 


Example 4: Tracking from a Frame-Marker with image content.

 


Example 5: Tracking a Split-Marker. FPS include complete AR workflow (most time goes into rendering...)

 


Example 6: StbTracker detecting 383 BCH-markers (marked with crosses) in a 640x480 image.
Crosses are not centered exactly on the marker due to lens undistortion.

Want to know more?

Send us an email requesting more information.