You're half right. here is the full explanation.
The segmentation and tracking is done with our software called VideoClix, in development since 1999.
It allows you to tag objects within video. Once tagged the tracking algorithms take over and track the object throughout the video.
This is what the authoring app looks like: http://www.videoclix.com/Presentation/VCX_authoring.swf
The Tracking algorithms track color, contour, texture and movement. Then with AI the results are combined to detect the object in each frame.
Then when the object is detected a primitive hotspot (simple shape) is created that best fits the shape of the object.
Here is a sample: http://www.videoclix.com/Presentation/tracking_demo.swf
(red is detection algorithm white is the vector based hotspot)
Then when all objects are tracked, info, fun facts, story line, bios, download links are entered into each object info panel through an online tool, which looks like this: http://www.videoclix.com/Presentation/content_campain.png
Then the video goes live and it can be embedded in any website. To answer the question how long... well, the tracking is real time. But it can be even shorter than real time if a technique like tweening, which you mentioned, is used. Basically the tracking system would track each 4th or 8th frame... then tween between the keyframes. The software then interpolates the rest of the hotspot positions.
For you guys wondering what tween means... it originates from the word "Between" keyframes.
I'll let the boys know about the cb_32 and S1N mis tags.
Hope you still enjoyed the video.
The VideoClix team.
I'm very interested in this software! Very awesome tracking algorithms.