Lighting and cinematography for VidSync data

Working with video means some basic principles of the arts are required for our science. To get the highest-quality data, we need to understand what conditions and settings transfer the most useful information from the scene to the recorded video frame. The relevant standard is whether the objects, landmarks, and behaviors needed for analysis remain visible when the recording is examined frame by frame.

Designing footage for analysis

A scene that looks good during normal playback may become difficult when an observer pauses on a single frame and tries to place a point, identify an individual, or determine whether a brief behavior occurred. Fine markings can disappear in noise, a moving edge can become a blur, and a subject can merge into a complicated background. Camera placement and lighting should therefore be tested using the same kinds of frames and viewing scale that will be used during analysis.

The required image quality depends on the observation. Detecting the presence of a large fish may tolerate difficult footage. Identifying an individual from its markings, locating the corner of a fin, or deciding whether a fish opened its mouth demands much clearer detail. Write down the smallest or least distinct feature that observers must see, then design the recording around that requirement.

Lighting

For video analysis of large, obvious objects, such as the presence or lengths of fish, a wide range of footage may suffice. For analyses of fine behavioral details, including whether a fish opened its mouth to ingest a possible prey item or whether markings identify an individual fish, lighting can make or break the data.

The light must be adequately bright. Dim lighting forces the camera toward a slower shutter speed, a wider aperture, a higher light sensitivity (ISO), or some combination of the three. A slower shutter increases motion blur. A wide aperture reduces depth of field. A high ISO increases noise and may obscure small edges or markings. More light gives the camera greater freedom to preserve the details needed for analysis.

The distribution of light also matters. Very bright and very dim areas in the same scene may exceed the range that the camera can record. Exposing for the bright areas can leave important shadows without visible detail; exposing for the shadows can remove detail from the highlights. Examine the actual subject, reference objects, and calibration targets for clipped highlights and blocked shadows.

Stationary cameras may remain in place while the sun moves, clouds pass, shadows lengthen, or artificial lights change. Consider the full recording period when choosing a camera position and exposure strategy. A short test made at only one time of day may miss conditions that dominate later footage.

Two types of natural lighting often work well for fine detail:

1. Diffuse light that evenly illuminates the scene. This is common on brightly overcast days, when sunlight scatters through the clouds and arrives from many directions. It can also occur in the shadow of a large object during a sunny day when the subject remains well lit by the open sky. A dark crevice with little sky exposure and no direct light may produce footage that is inadequate for detailed analysis.
2. Direct sunlight entering through a relatively flat water surface from a favorable direction. Sunlight is generally easier to work with when the sun is behind the cameras or to the side. Light coming toward the cameras can produce glare and reduce contrast. Refraction through a wavy surface can also cast rapidly moving bands of bright and dim light across the subject. This may be visually attractive, but the changing illumination can make analysis more difficult, as in the scene below.

Motion blur and frame rate

Shutter speed and frame rate affect different parts of the recording. Frame rate determines how often an image is recorded. Shutter speed determines how long each frame gathers light and therefore how far a moving feature can travel while that frame is being exposed. A high frame rate does not by itself guarantee sharp frames if the exposure time remains long.

Choose a shutter speed by considering the fastest movement and the smallest moving feature that must remain distinct. Fish turns, fin tips, prey strikes, drifting particles, and rapidly moved reference objects may blur even when the body of the subject appears sharp. Faster shutter speeds reduce blur but require more light, a wider aperture, or a higher ISO. Pilot recordings should include representative fast movements rather than only stationary subjects.

Focus and depth of field

Focus should be checked at the actual subject distance and through every part of the optical path used during recording. A focus-distance scale developed in air may be misleading when the camera records through a housing port and into water. Test the complete assembly under field conditions.

Depth of field must cover the region in which observations will be made. Focusing on a central point does little good if subjects regularly move into a soft foreground or background. A smaller aperture can increase depth of field, but it also reduces the available light. Increasing the subject distance may expand the usable depth range while reducing the number of pixels devoted to the subject. These tradeoffs are best resolved with recordings of the intended scene.

Continuous autofocus can shift attention between the subject, suspended material, a foreground obstruction, and the background. If focus is fixed manually, secure the focus and zoom controls when practical and record their settings. With multiple cameras, inspect each view independently. A feature that is clear in one camera may be soft or obscured in another.

Exposure, color, and changing conditions

Manual exposure can keep the appearance of a scene stable, which helps when observers compare frames or identify markings. Automatic exposure may be useful when illumination changes substantially during a long deployment, but it can react to a bright object, a passing shadow, or a change in the amount of open water in the frame. Test how the chosen mode responds to events expected during the study.

Stable white balance is useful when color or markings contribute to identification. Automatic white balance can change between frames as the scene changes. When color is unimportant, sharp edges and adequate tonal separation usually matter more than aesthetically pleasing color. Underwater light also changes with depth, dissolved material, and the distance between camera and subject, so a setting tested at one site may not transfer directly to another.

Glare, suspended material, and obstructions

Reflections from the water surface, aquarium walls, housing ports, or nearby lights can cover useful detail. Examine the scene from the position of each camera and change the camera, light, or background angle when practical. Keep housing ports and aquarium walls clean. Water droplets, condensation, fingerprints, and residue are especially conspicuous when they are close to a short-focused lens.

Turbidity and suspended particles reduce contrast with increasing distance. Nearby particles can also become bright, out-of-focus obstructions. Shorter camera-to-subject distances often improve visibility in these conditions, provided the required field of view is retained. Allow disturbed sediment or detritus to settle before recording important observations or reference material. Field procedures should minimize walking, equipment movement, bubbles, and other disturbances that introduce material into the viewing volume.

Vegetation, structural habitat, other animals, and parts of the subject itself can obscure the feature being recorded. Multiple camera views help only when at least one view retains a usable line of sight. Review likely subject positions and movements from every camera before committing to a long deployment.

Background and contrast

The visual contrast of the background behind a subject can be as important as the overall brightness. Practical constraints may determine the camera direction, but a relatively uniform background generally makes outlines and landmarks easier to see. Texture that resembles the subject can make point placement and individual identification difficult even when the exposure is good.

For example, in my own studies of juvenile Chinook salmon around logjams in Alaska, I tried to avoid filming fish with a dense mat of roots and leaves directly behind them. I would instead point the cameras toward the open water of the main channel, or toward a dark recess of the logjam where brightly lit foreground fish contrasted against a dark background.

Background choice also affects automated or semi-automated image processing. A method that works on a clean silhouette may fail when shadows, vegetation, surface shimmer, or other moving objects have similar brightness and texture. Record a pilot sample from the intended background before designing a large analysis around it.

Reference objects and calibration footage

Reference objects must be recorded with the same attention given to the subject. Their edges or points should be visible, in focus, and large enough to identify consistently. Avoid placing a bright target in deep shadow or allowing white areas to lose their internal detail through overexposure. For underwater work, record the target through the same housing and water conditions used for the observations.

Keep cameras and mounts still while collecting observation footage and its associated reference material. If a camera moves or a relevant optical setting changes, document the change and collect the reference footage required for the new setup. The Field protocol page provides a general recording and documentation sequence.

Review pilot recordings

Review pilot footage on a computer before finalizing the field design. Step through representative sequences one frame at a time and view them at the magnification likely to be used during analysis. Include fast movements, difficult subject positions, changing light, and the least favorable parts of the field of view. Check whether another observer can identify the required features without relying on knowledge of what happened during recording.

A useful pilot review asks whether observers can see the subject boundaries and landmarks, distinguish individuals when required, interpret brief behaviors, find synchronization cues, and identify points on reference objects. It should also reveal autofocus shifts, exposure changes, motion blur, glare, obstruction, and periods when the subject leaves the usable viewing volume.

Recording checklist

• The smallest required feature is visible in representative frames.
• Fast movements remain sufficiently sharp for the intended observation.
• The expected subject area falls within the usable depth of field in every camera.
• Highlights and shadows retain the details needed for analysis.
• Backgrounds provide adequate contrast at likely subject positions.
• Glare, shimmer, turbidity, debris, bubbles, and obstructions have been checked.
• Synchronization cues and reference objects are visible in every required view.
• Camera settings and any changes are recorded.

See Camera settings for a fuller discussion of resolution, frame rate, shutter speed, aperture, ISO, focus, and related controls.