In this Electronics and Communication project, we demonstrate for the first time a proof-of-concept multi-target detection, segmentation, and tracking system using a commercially available 3D Flash LIDAR video camera. The system will accomplish the detection, segmentation, and tracking of multiple stationary and non-stationary human-sized objects which are arranged on a roughly planar surface. We have used a combination of fundamental point cloud video processing algorithms to illustrate our proof-of-concept. Since point cloud video sequences depicting moving objects are relatively uncommon, we will first discuss important considerations for dealing with this data type.
This work has direct applicability in many 3D vision scenarios most specifically remote geometric measurement of objects, target tracking, site security, and perimeter surveillance. With concepts such as these one can easily envision security systems which autonomously detect intruders and pinpoint their locations accurately for the purpose of passive surveillance or interdiction. In addition to measuring the precise location of the intruding object, its size can accurately be measured. Partial 3D models of the object can easily be visualized. With marginal optimization and refinement efforts, the result of this work can already be of use in the stated application areas.
LIDAR is an acronym which stands for Light Detection And Ranging based on a laser-radar paradigm. RADAR (Radio Detection And Ranging) is the process of transmitting, receiving, detecting, and processing and electromagnetic wave that reflects from a target, a technology that was developed by the German Army in 1935. All ranging systems function by transmitting and receiving electromagnetic energy with the primary difference between RADAR and LIDAR being the frequency bands. LIDAR uses ultraviolet, visible, or near infrared light to image objects where as RADAR uses radio waves.
LIDAR is an optical remote sensing technology that can measure the distance to, or other properties of a target by illuminating the target with light, often using pulses from a laser. LIDAR technology has application in geomatics, archaeology, geography, geology, geomorphology, seismology, forestry, remote sensing, and atmospheric physics, as well as in airborne laser swath mapping (ALSM), laser altimetry and LIDAR contour mapping.
3D FLASH LIDAR
The term 3D Flash LIDAR refers to the 3D point cloud and intensity data that is captured by a 3D Flash LIDAR imaging system. 3D Flash LIDAR enables real-time 3D imaging capturing 3D depth and intensity data characterize by lack of platform motion distortion or scanning artifacts. When used on a moving vehicle or platforms, blur-free imaging is expected. This is as the result of using single laser pulses to illuminate the field-of-view (FOV) creating an entire frame. Because the integration time is fast (e.g. at 100 meters range, capture requires 660 nano-seconds), the ability to produce real-time 3D video streams consisting of absolute range and co-registered intensity for use in autonomous applications is a natural fit.
Because there 3D Flash LIDAR systems are solid state (no moving parts), they carry less mass than alternative LiDAR cameras (i.e. scanners). Capture distances range from 5cm to 5km. 3D Flash LIDAR camera’s readout semiconductors enable each pixel in the focal plane array to act independently and measure range and intensity of every pixel (point) in the camera's field of view. It provide accurate 3D representation (models) of the scene including measurements, real-time imaging through obscuration such as dust, cloud or smoke. The "framing camera" nature of 3D Flash LIDAR cameras and the real-time 3D video output makes them ideal for fast moving-vehicle solutions such as automotive or aviation.
3D FLASH LIDAR Characteristics
- Full frame time-of-flight data collected with single laser pulse
- Unambiguous direct calculation of absolute range
- Full frame rates with area array technology
- Blur-free images without motion distortion
- Co-registration of range and intensity for each pixel
- Pixels are perfectly registered within a frame
- Ability to represent objects that are oblique to the camera
- Non-mechanical (no need for precision scanning mechanisms)
- Calibration done at manufacturing time
- Smaller and lighter than point scanning systems
- Low power consumption
- Ability to “see” into obscuration (range-gating)
- Eye-Safe laser assembly
- Combine 3D Flash LIDAR with 2D cameras (EO and IR) for 2D texture over 3D depth
- Possible to combine multiple 3D Flash LIDAR cameras for a full volumetric 3D scene
Future development should include the use of higher resolution 3D Flash LIDAR video cameras which will soon become available. Future development should also include optimization of the proof-of-concept algorithms described in this paper to run in real-time at higher frame rates. Also, fusion of the 3D point cloud video with high resolution visible near infrared (VNIR) or short-wave infrared (SWIR), and the ubiquitous long-wave infrared (LWIR) camera data can be used to enhance the functionality of the described detection and tracking system in the context of perimeter security and surveillance operations. Please go through the attached Project report for more information.