AutoMagicCalib#

Overview#

AutoMagicCalib (AMC) is an automated calibration tool that estimates both intrinsic and extrinsic camera parameters for single-camera and multi-camera systems. It provides camera projection matrices and lens distortion coefficients essential for accurate 3D reconstruction and multi-view applications.

AMC eliminates the need for traditional calibration patterns (like checkerboards) by using tracked moving objects in the scene as natural features for calibration. It leverages DeepStream’s object detection and tracking capabilities to identify and follow objects (particularly people) across frames, then analyzes these trajectories across camera views to automatically derive camera parameters from regular operational footage. This approach enables calibration without interrupting normal operations, allows retroactive calibration using archived footage, and performs calibration in the actual deployment environment.

Note

This document offers a quick overview for AutoMagicCalib. For detailed step-by-step instructions, configuration options, and advanced usage, please refer to the AutoMagicCalib GitHub Repository.

Key Features#

  • Automatic Calibration: Uses DeepStream-tracked moving objects as natural features, enabling calibration from operational footage without service interruption

  • Lens Distortion Correction: Estimates radial distortion parameter (k1)

  • Camera Projection Matrix: Computes 3x4 projection matrix including focal length, rotation, and translation

  • Multi-View Support: Computes extrinsic parameters between cameras in multi-camera systems

  • Ground Truth Integration: Optional focal length override and evaluation against known parameters

  • Evaluation and Visualization Tools: Score metrics, rectified videos, and trajectory overlays

  • Format Conversion: Export calibration for downstream applications like MV3DT

System Requirements#

  • x86_64 system

  • OS Ubuntu 24.04

  • NVIDIA GPU with hardware encoder (NVENC)

  • NVIDIA driver 570

  • Docker (setup to run without sudo privilege)

  • NVIDIA container toolkit

Quick Start#

The AutoMagicCalib source code and detailed documentation are available at: NVIDIA-AI-IOT/auto-magic-calib

  1. Prepare Video Files

    Ensure your video footage contains moving objects for tracking, with sequential cameras having overlapping views.

  2. Repository Setup

    git clone https://github.com/NVIDIA-AI-IOT/auto-magic-calib
cd auto-magic-calib
git submodule update --init --recursive
git lfs pull
cd scripts
bash setup.sh

  3. Run Calibration

    For calibration with ground truth for evaluation:

    bash launch_EndToEndCalib.sh -v <video_dir> -o <output_dir> -g <groundtruth_dir> -l <layout_image> -f <focal_lengths>

    For calibration without ground truth:

    bash launch_EndToEndCalib.sh -v <video_dir> -o <output_dir> -f <focal_lengths>

Calibration Output#

AMC generates comprehensive calibration results:

output_dir/
├── single_view_results/
│   └── cam_XX/
│       ├── camInfo_hyper_XX.yaml        # Camera parameters
│       └── distortion.yaml              # Lens distortion
└── multi_view_results/
    └── BA_output/
        └── results_ba/
            └── refined/
                └── camInfo_XX.yaml      # Final refined parameters

Each camera’s calibration includes:

  • K: 3x3 intrinsic matrix

  • R: 3x3 rotation matrix

  • t: 3x1 translation vector

  • projectionMatrix_3x4: Complete projection matrix

Integration with MV3DT#

AMC produces calibration results in a generic format that may require conversion for specific downstream applications. Each application has its own expected input format, coordinate system conventions, and metadata requirements.

For integration with Multi-View 3D Tracking (MV3DT), convert AMC output to the required format using the provided script:

bash launch_ConvertToMv3dt.sh -i <amc_output> -o <mv3dt_output> [-g]

This generates MV3DT-compatible format with proper coordinate transformations and metadata.

The following figure shows MV3DT performing real-time multi-view 3D tracking using camera calibration parameters generated by AutoMagicCalib. The system tracks objects across multiple camera views while maintaining consistent identities, demonstrating successful integration of AMC calibration results with downstream tracking applications.

MV3DT visualization using AMC calibration

MV3DT real-time multi-view tracking with AMC-generated calibration parameters#

Additional Resources#

  • GitHub Repository: NVIDIA-AI-IOT/auto-magic-calib

  • Sample Data: Available in the repository for testing

  • Support: File issues on the GitHub repository

For detailed documentation including:

  • Complete command-line options

  • Manual alignment procedures

  • Evaluation metrics interpretation

Please refer to the comprehensive README in the GitHub repository.