Skip to content

Camtrap datetime cleanup

Background

In practice, media timestamps cannot be assumed to be reliable. Different camera manufacturers write different datetime fields, and the same dataset may contain images, videos, XMP sidecars, and files whose modified time has already been altered during transfer. So the goal of datetime cleanup is usually not to "read one field directly", but to initialize timestamps as robustly as possible first, and then leave abnormal cases for manual review.

Our usual approach is to iterate through all available timestamp metadata fields and initialize the datetime according to a priority order; if none of them are available, we fall back to the file modified time as the final backup. This logic is already implemented in serval; see Serval tags.rs. As far as I remember, trapper-client used a very similar strategy when generating data packages.

In real projects, this initialization method usually covers more than 95% of the data. The remaining issues are typically caused by wrong camera time settings or missing metadata, and need to be corrected with deployment records and manual judgment.

Common problematic cases

The camera clock was set incorrectly or reset during deployment

This is the most common source of timestamp errors. For example, the clock may not have been calibrated before deployment, or it may have been reset because of hardware failure or battery issues. In such cases, users can often still infer the correct time from deployment records, setup and retrieval times, or neighboring media from the same site, and then apply a uniform offset correction to the affected batch.

The timestamp is burned into the image but not written into metadata

Some cameras only overlay the timestamp onto the image or video frame, but do not write it into EXIF, QuickTime, or any other metadata field. If the file modified time is also changed during copying, compression, or synchronization, it becomes very difficult to recover the true capture time from file properties alone. In that case, OCR is basically the only remaining option; see Serval issue #225.

1. Initialize timestamps and generate an XMP sheet

It is recommended to start with serval xmp init --info to generate an XMP sheet for review. The purpose of this sheet is not to "automatically fix everything", but to gather all currently available timestamp information into one table so that users can inspect and correct it efficiently.

At minimum, the initialized sheet should serve two purposes:

  • Keep the media path and the current initialized datetime result.
  • Preserve as many available timestamp sources as possible for comparison.

This makes the process much easier to audit: even when the initialized result is not fully correct, users can still see the basis for the result instead of receiving a black-box output.

2. Import the sheet into the web visualization tool

After that, import the sheet generated by serval xmp init --info into serval-charton to inspect whether the time distribution looks reasonable. In practice, the most useful views include:

  • Site overview, for checking the overall temporal coverage and suspicious clusters.
  • Site media timeline view, for examining how records from different sites are distributed over time.
  • Time heatmap, for spotting obvious offsets, bursts, or discontinuities.
  • Site info table, for direct inspection and export of tabular results.

The key point here is to find anomalies first and decide how to fix them afterward. Compared with reading the table row by row, the visual views make it much easier to detect batch-level offsets, site-specific clock resets, or isolated device failures.

3. Correct timestamps directly in the table

Once anomalies are identified, timestamps can be edited directly in the CSV table. In the current workflow, the main field to edit is xmp_update_datetime, which should contain the corrected datetime. Batch datetime updates can also be used when a whole group of files needs to be shifted together.

Typical correction patterns include:

  • Apply a uniform time offset to an entire site.
  • Correct only one abnormal segment.
  • Manually refill clearly incorrect start or end segments.

4. Reload the data source and review the charts again

After editing the CSV, there is no need to rebuild the whole dataset. Just reload the data source in the web tool and the charts will update accordingly. This makes it easy to form a quick loop of "edit once, review once" until all anomalies are either explained or corrected.

5. Export a trap_info template

Once the timestamps look correct, the web tool can export a trap_info template. The exported table will already contain the inferred start and end times for each deployment, and users only need to fill in the remaining fields before continuing with the rest of the workflow.

Practical suggestions

  • Do not overwrite the original time information too early; keep both the initialized result and all visible time sources.
  • When you find an anomaly, first decide whether it is a single-file problem or a batch-level offset.
  • File modified time should only be treated as the last fallback, not assumed to be the capture time by default.
  • If the timestamp exists only as visible text in the image and file properties are already unreliable, move to an OCR workflow early instead of continuing to rely on metadata.
  • In most cases, it is safer to finish datetime cleanup before generating or revising trap_info.