Government, municipalities, financial institutions, and businesses are driving new uses for video surveillance technologies beyond crime prevention or security into applications such as asset management, risk mitigation, and safety.

The challenge for camera manufacturers, however, is developing “smarter” cameras at lower price points. More and more, digital high-definition (HD) Internet protocol surveillance cameras are replacing analog cameras because of lower installation costs, scalability, and the ability to add intelligence.

Wide dynamic range (WDR) CMOS image sensor technology, for example, provides high-quality images with better resolution and higher performance under very harsh lighting conditions. The following figure illustrates how an Intel® FPGA is the central interface for WDR CMOS sensors.

Advanced video analytics is replacing simple motion detection to automate the monitoring of video, which can result in significant cost savings. Learn how Intel® FPGAs can be used for advanced video analytics.

Intel® FPGAs play a key design role in these next-generation high-definition Internet protocol cameras:

  • Flexibility to interface to many types of image sensors, including HD WDR video image sensors that support up to 1080p60
  • Fast processing to incorporate a full image sensor pipeline (ISP) intellectual property (IP) that includes techniques such as defect pixel correction, gamma correction, dynamic range correction, and noise reduction
  • Cost-effective solution that can incorporate functions such as sensor interfacing, image compression, and even pan-tilt-zoom (PTZ) control in a single device
  • High-performance HD video analytics
  • Integrate a complete IP camera design in as little as 11x11 mm2 with small form factor Cyclone® V packaging

Simplified Block Diagram of an FPGA-Based Surveillance Camera

Industry's First HD WDR Video Surveillance Chipset

Intel® has simplified video surveillance camera system development with the industry’s first HD WDR video surveillance chipset. This chipset streamlines the delivery of ISP IP and provides a single vendor source for the IP and FPGA. With ISP provider Apical and HD WDR sensor manufacturer AltaSens, Altera provides a verified ISP optimized for the Altasens WDR sensor. No other ASSP or DSP platform offers a comprehensive pipeline that incorporates WDR technology using a 1080p60 sensor and data path. And FPGAs are the only devices that can handle the large bandwidth of data from 1080p and 720p WDR CMOS sensors.

High-Definition Wide Dynamic Range Video Surveillance Chipset

The industry's first high-definition (HD) wide dynamic range (WDR) video surveillance chipset from Intel® simplifies the development of video surveillance camera systems. It provides a single vendor source for the intellectual property (IP) and FPGA.

Our exclusive chipset combines an Intel® Cyclone® IV E FPGA with a security chip that supports Apical's HD WDR full image signal processing (ISP) pipeline IP and AltaSens' 1080p60 A3372E3-4T image sensor. We are streamlining access to Apical's IP and reducing your risk by giving you a complete sensor processing solution. To learn more, contact your local Altera sales representative.

No other ASSP or digital signal processing (DSP) platform offers a comprehensive pipeline that incorporates WDR technology using a 1080p60 sensor and data path. And FPGAs are the only devices that can handle the large bandwidth of data from 1080p and 720p WDR CMOS sensors.

Image Sensor Processing Pipeline Challenge

WDR sensors output up to 20 bits of raw image data, but lack an ISP, making it a challenge to connect them to standard ASSP or DSP devices.

Figure 1 shows two photographs of a street scene comparing what the output would look like from a standard camera (on the left) versus a camera with a WDR image sensor using appropriate image processing (on the right). Compared to the unprocessed left image in Figure 2, the right image shows how the high-performance ISP IP core allows the maximum detail to be extracted from a high-contrast scene. In particular, the example reveals the dark areas without corresponding overexposure in the bright areas.

Figure 1. Standard Sensor vs. WDR Sensor Output Image

Figure1. Standard Sensor vs. WDR Sensor Output Image
Images courtesy of Apical Ltd

Although WDR sensors support high-quality resolution and higher frame rates, it requires more computing power to process these images such as pixel-by-pixel correction, local toning, and white balancing. This poses a challenge for off-the-shelf DSPs and even some ASSPs because CMOS WDR sensors have no on-chip image pipeline processing and output the image data in RAW/Bayer format at up to 20 bits per pixel. The large amount of raw data coming from the sensor can be computed as:

  • 20 bits/pixel x (1280 x 720) pixels/frame x 60 frames/s = >1 gigabits per second (Gbps)

This large amount of data makes it difficult to connect next-generation WDR sensors to ASSPs commonly used in surveillance solutions.

Intel® Video Surveillance Solution

Using Intel's® Cyclone III or Cyclone IV FPGAs, along with our video surveillance camera reference solution, the FPGA can be used to connect directly to the image sensor, implement the full ISP, and connect to any encoder or PHY device. In addition, the FPGA can be optionally used to perform H.264 encoding and to create a full Internet protocol camera pipeline on one chip – all without the need for an external DSP device or ASSP.

Intel® FPGAs are the ideal choice for efficient processing of HD WDR sensor data. Together with Intel's® partner, Apical, a leader in image processing IP, this HD WDR sensor interface solution offers an ISP (Figure 3) with the following functions:

  • Hot pixel removal and noise reduction (spatial and temporal IP cores are available)
  • Advanced per pixel tone mapping (Apical's patented Iridix IP core)
  • Advanced demosaic and color correction

Figure 2. Block Diagram of Apical's ISP
Block Diagram of Apical's ISP

Streamlined Image Sensor Pipeline IP Delivery

The Intel®, Apical, and AltaSens solution provides a low-risk, low-cost method for camera manufacturers to implement the ISP in a simple chipset. This exclusive solution incorporates:

  • Intel® Cyclone IV E FPGA with a pre-programmed security device that supports Apical's HD WDR full image signal processing (ISP) pipeline IP
  • AltaSens' 1080p60 A3372E3-4T image sensor technology

This unique solution streamlines access to Apical's IP and mitigates the customer risk by providing a complete sensor interface solution.

The chipset simplifies the development of a video surveillance camera system by providing a single vendor source for the IP and the FPGA. You no longer have to purchase the IP from a separate vendor and pay extra licensing fees or non-recurring engineering (NRE) charges. The price of the chipset is all you pay.

Your local Altera sales representative can assist you with the easy download of Apical's IP, registration for an evaluation license online, and evaluation of your system before purchasing the chipset. After evaluation, you can obtain and license the IP online and incorporate the IP into your Quartus® Prime FPGA design. This ready-made, yet flexible chipset decreases development time and adds flexibility to the camera design. Because of the FPGA's flexibility, you can customize the device with your specific features to further differentiate your product from competitors.

 

 

Intel® and our partner Eutecus offer a range of video analytics solutions designed to provide the highest performance, lowest cost and lowest power single FPGA-based solutions for your system.

The MVE Core V2.x provides a high-performance, configurable video-analytics solution for a wide range of both standing and moving platform applications.

The MVE Core V2.x combines high performance video analytics accelerators (Eutecus video coprocessors) and embedded software (Eutecus InstantVision SW) to perform (single-channel) video analytics and/or (multiple-channel) fusion analytics functions.  As a result, the MVE Core enables the system to extract meaningful data from single/multiple video streams and detect pre-specified events in real-time, thereby providing actionable intelligence for a broad spectrum of embedded devices.

MVE™ Core Video Analytics Engine Block Diagram

Quad-Channel ReCo™-Pro module

The quad-channel ReCo™-Pro module implements the full HD MVE™-ready video analytics core for immediate deployment in selected application areas. 

Figure 1. ReCo™-Pro vision system with four cameras

The ReCo™-Pro VA modules enable configurability via versatile input-output connectivity and are designed for use on portable and rugged platforms. The IP library includes ISP, video I/O, video analytics, video compression and Ethernet/Serial Host control solutions to support seamless integration.

ODMs, OEMs and integrators who implement the ReCo™ VA modules into their products, benefit from the power of robust image processing capabilities that are hard-coded into their systems while maintaining flexibility through remote firmware/software upgrades and re-configurability. 

 

Figure 2. The quad-channel ReCo™-Pro module

The ReCo™-Pro module is built on the Altera Cyclone V SoC FPGA + ARM back-end to meet the additional processing requirements needed for sensor/camera specific ISP and video compression applications. It also supports multi-channel video analytics capabilities for moving platform applications (ADAS, UAS). Taken together, four miniature wide dynamic range HD cameras supported by real-time ReCo™-Pro-based video analytic processing offers unparalleled performance and scene-understanding capability for today’s challenging applications.

Solutions:

Industrial applications:

For more information, contact your local Intel® sales office or send an email.