What is VidSync?

VidSync is a free, open-source Mac application for scientific video analysis. It was designed for 3-D measurement with a pair of cameras, but is also useful for 2-D measurement applications and those requiring efficient, organized event logging without any spatial measurements at all. It is equally applicable in freshwater, saltwater, laboratory, and terrestrial settings.

Major features

• Measure 3-D points and lengths with sub-millimeter accuracy.
• Quick, simple, inexpensive 3-D calibration.
• Powerful, flexible playback controls.
• Easily organize and navigate thousands of measurements.
• Multiple data export formats.
• Extensive, helpful visual overlays.
• Capture analyzed video and still frames.

Applications

VidSync was developed for field research on freshwater fish behavior, but it is equally useful in the laboratory, and in terrestrial or marine environments. It lets you record well-organized measurements of:

• Positions.
• Lengths.
• Times.
• Anything you can calculate from the three above.

Length measurements are especially simple and show up instantly on-screen (velocities can, too):

Exporting VidSync data for analysis in other programs (such as R, Mathematica, Matlab, or Excel) enables complex, creative applications. For example, below are the territories of Chinook salmon filmed in the Chena River, Alaska, on August 14, 2009 during twenty minutes of feeding. Each sphere represents the position of a prey capture attempt. Conflicts between fish are represented by pyramids (winner) and square (loser). The yellow arrow shows the direction of flow from the upstream end of the school of fish. The graphics were generated in Mathematica using 3-D measurements exported from VidSync.

The main restrictions on VidSync's applications are 1) you have to be able to see the thing you're measuring, so darkness and turbid water are problematic, and 2) you have to use a calibration frame with dimensions not more than 10-100x smaller the size of the area you're measuring. For example, with a very manageable half-meter calibration frame, you can make decent measurements within 10-20 meters, but you cannot measure the height of a building a mile away.

To see some other things people have done with VidSync already, go to the Project Showcase.

 

Feature details

 

Easy calibration

The two objects required to calibrate videos for measurement are easy and inexpensive to build (a few hundred dollars at most). Actual calibration takes about 5 minutes in the field or lab, and about 10 on the computer once you're familiar with the process. A new user can learn the process from scratch in less than an hour.

Measurement accuracy and the magnified preview

Although VidSync's mathematical methods are capable of sub-millimeter accuracy, your measurements are only as good as your input. VidSync has some powerful tools to let you quickly input points for measurement with as much precision as you want.

When you click on a point to measure in VidSync, you don't need to worry about exactly where you're clicking, or which pixel on under your mouse pointer is being measured. A magnified, live preview image of the video places a tiny dot (or an elaborate reticle) directly over the point you're measuring. After clicking, use the arrow keys to manipulate the measurement position with sub-pixel precision.

 

Advanced playback control

VidSync replaces the standard Quicktime playback controls with a new control bar tailored for scientific analysis. We often need to be able to stop immediately when we observe something of interest, then back up and re-watch it two or three times, maybe at different speeds. Or maybe we need to jump around the video at fixed intervals (or systematically randomized ones). In a normal video player these tasks are distractingly awkward. In VidSync they're so easy that after the first hour or two you aren't even thinking about them.

Easy organization, annotation, and retrieval of measurements

VidSync organizes measurements into a hierarchy of "objects," such as fish, and "events," such as prey capture attempts. Each event may have multiple measured points. Events (such as a conflict between fish) can also be associated with multiple objects (i.e. Vidsync supports many-to-many relationships).

Objects determine the color of the symbol that overlays the video to indicate measured point (for example, I give different fish different colors). Events determine the size and shape of the symbol, how long it stays onscreen, whether it's connected by a line to other points in the same event (e.g., for a length measurement event), and other visual properties. You don't have to set these attributes for every individual event. Instead, you define event types (i.e. length measurement, prey capture attempt, etc.) and individual events take on the properties of their type.

Many small features of this system have a large effect on the ease with which you can record and retrieve measurements:

• You can add arbitrary notes to every object or event, useful for sophisticated coding of observations.
• The point selected in the lists shows a square crosshair selection indicator around its symbol on the video overlay.
• You can select a point through the menus and click "Go" to jump the video to its location.
• You can right-click near a point on the video overlay to select it in the lists, and click "Go" to jump to the frame on which it was recorded.
• You can split an object into two, or merge two objects into one, preserving all the measured events.
• Sort the lists by any field, or use the "Sort" button for the default order (first type, then index, then name).

This system has some very flexible applications. Below, I used it to digitize the boundary of the woody debris behind the fish, for estimating distance-to-cover, as well as the river bottom.

 

Flexible data export and image/video capture

You can export measurements from VidSync to a CSV spreadsheet file, which is adequate for most simple measurements, or an XML file, which fully reflects the underlying organization of VidSync's objects and events including many-to-many relationships. Almost any programming language you might use for data analysis (including R, Mathematica, Matlab, and Python) has XML import capabilities you can use to parse VidSync's XML output.

You can also export still images or video clips, with or without measurement overlays. VidSync videos with measurement overlays help you construct dynamic, stand-out presentations at professional conferences. You can get creative with the object/event hierarchy to make very interesting illustrations. For example, I used the interval-stepping playback feature to create a presentation video showing how a juvenile Chinook salmon captures a particle of potential prey and then "spits" it back out.

 

Ease-of-use features

One of the main things setting VidSync apart from its alternatives is the cumulative effect of dozens of minor features that make each task a little bit faster or a little bit easier. Here's an example of how a few tasks are handled.

For locating things to measure:

• Fine-scale controls with keyboard shortcuts allow custom playback at any speed, with random or interval sampling.
• Mouse wheel steps through the video frame-by-frame.
• Non-measurement text annotations mark items of interest.
• “Hint lines” to find matching objects in other views.

For editing past measurements:

• Measurements are organized in an intuitive hierarchy of objects (e.g., fish) and events (e.g., conflicts).
• Retrieve measurements through tables or clicking markers on video.

For inputting precise spatial coordinates:

• Arrow keys relocate measurements with sub-pixel precision.
• Magnified preview shows precise measurement location clearly.

For initiating new measurements:

• New events are auto-created by clicking on video, if input from the previous event is complete.

For interpeting past measurements:

• Measurement marker size, color, and shape represent different measurement types or objects.

For data organization and sharing:

• Full analysis is self-contained in one file, edited by one program.
• Reusable information (e.g., calibrations and object/event types) can be exported separately for sharing across projects.
• Full object and event hierarchy is exported in CSV or XML files.

 

System Requirements

The current version of VidSync requires Mac OS X Mavericks (10.9) or later. Because it relies on several Mac-specific underlying technologies, no Windows version is possible. VidSync itself requires very little processing power, so performance is limited by the computer's ability to play multiple videos simultaneously. Most Macs from 2010 or later (and some older machines) can handle a stereo pair of 1080p (full HD) videos, or several standard definition (640x480) videos.