AN6211 · Application noteIntroduction to digital camera interface pixel pipeline for STM32 MCUs
Download .pdfOpen in the full document

DCMIPP functional description

Block diagram

As mentioned in Figure 7 DCMIPP architecture, the DCMIPP receives pixels from either a parallel or CSI interface. Once the pixel data enters, it passes through a flow selector (FLOWSEL) that directs the data to the appropriate processing pipelines.

The architecture features three main pipelines for pixel processing:

  • Pipe0-Dump serves as a side dump pipe for legacy functions (similar to DCMI), statistics, and raw Bayer data for still images. Depending on the product configuration, it includes stages such as data decimation, reformatting, frame control, and image cropping.
  • Pipe1-ISP is more complex, it converts raw Bayer into full-resolution RGB. It also adjusts exposure and white balance parameters, with stages such as byte conversion, defective pixel correction, demosaicing, color conversion, and contrast adjustment.
  • Pipe1-PostProc and Pipe2-PostProc handle postprocessing for different applications, adapting full-resolution RGB to specific needs such as capture rate, cropping, resizing, and color conversion, with the capability to convert to YUV420 for JPEG or H.264 encoding. Both postprocessing pipes include stages such as frame control, cropping, decimation, downsizing, gamma correction, and pixel packing. The Pipe2‑PostProc offers an additional option to sample RGB data before ISP processing. The processed data is then sent to the AXI interface for further use.
Note:

STM32H7R3/7S3 includes only Pipe0-Dump pipeline in the DCMIPP architecture.

Figure 7. DCMIPP architecture

Hardware interfaces

The DCMIPP includes:
  • Parallel input interface:
    • Up to 16 data lines (D15-D0)
    • Pixel clock line DCMIPP _PIXCLK
    • DCMIPP _HSYNC line (horizontal synchronization)
    • DCMIPP _VSYNC line (vertical synchronization)
  • CSI-2 camera serial interface:
    • Clock lane
    • Data lanes (Lane 0, Lane 1)
Figure 8. DCMIPP signals
Hardware synchronization mode

In hardware synchronization mode, the system uses two synchronization signals, DCMIPP_HSYNC and DCMIPP_VSYNC. Depending on the camera module and mode, data can be transmitted during horizontal and vertical synchronization periods. DCMIPP_HSYNC and DCMIPP_VSYNC act as blanking signals, as all data received during DCMIPP_HSYNC and DCMIPP_VSYNC active periods are ignored.

To correctly transfer images in the RAM buffer, data transfer is synchronized with the DCMIPP_HSYNC and DCMIPP_VSYNC signals.

When the hardware synchronization mode is selected and capture is enabled (CPTREQ bit set in DCMIPP_PxFCTCR), data transfer is synchronized with the deassertion of the DCMIPP_VSYNC signal (next start of frame).

Then, the transfer can be continuous, with successive frames transferred by the IP-Plug to either the same buffers, succesive buffers or circular buffers.

Figure 9 Frame structure in hardware synchronization mode illustrates an example of data transfer when DCMIPP_VSYNC and DCMIPP_HSYNC signals are active high, and when the capture edge for DCMIPP_PIXCLK is the rising edge.

Figure 9. Frame structure in hardware synchronization mode
Embedded data synchronization mode

The embedded data synchronization mode is available only with the parallel interface capture mode. In this mode, the data flow is synchronized using embedded 32-bit codes with the values 0x00 and 0xFF, which are no longer used in the data.

There are four types of codes, all with 0xFF0000XY format. The embedded synchronization codes are supported only in 8-bit parallel data capture (in the DCMIPP_PRCR register, the EDM[2:0] bits must be programmed to “000”). For other data widths, the synchronization mode generates unpredictable results.

Note:

Camera modules generate up to eight synchronization codes when in interleaved mode, while the DCMIPP reacts to one single code. Therefore, in an interleaved flow with an embedded synchronization, every other frame is discarded because it is not detected.

Mode 1

This mode is ITU656 compatible (ITU656 is the digital video protocol of ITU-R BT.656).

The following reference codes indicate a set of four events:

  • SAV (active line): line-start
  • EAV (active line): line-end
  • SAV (blanking): line-start during inter-frame blanking period
  • EAV (blanking): line-end during inter-frame blanking period
This mode can be supported by programming the following codes:
  • FS ≤ 0xFF
  • FE ≤ 0xFF
  • LS ≤ SAV (active)
  • LE ≤ EAV (active)

Figure 10 Frame structure in embedded synchronization mode 1 illustrates the frame structure using this mode.

Figure 10. Frame structure in embedded synchronization mode 1

Mode 2

The embedded synchronization codes signal another set of events:
  • Frame-start (FS)
  • Frame-end (FE)
  • Line-start (LS)
  • Line-end (LE)

A 0xFF value programmed as a frame-end (FE) means that all the unused codes (the possible values of codes other than FS, LS, LE) are interpreted as valid FE codes.

In this mode, once the camera interface has been enabled, the frame capture starts after the first occurrence of an FE code followed by an FS code.

Figure 11 Frame structure in embedded synchronization mode 2 illustrates the frame structure when using this mode.
Figure 11. Frame structure in embedded synchronization mode 2
CSI-2 overview

The CSI-2 low-level protocol supports the transport of arbitrary data using both short and long packet formats, with packets transmitted over the physical D-PHY interface. The protocol has the following characteristics:

  • Transport of arbitrary data
  • 8-bit wordsize
  • Special packets for frame start, frame end, line start, and line end information
  • Descriptor for the type, pixel depth, and format of the application-specific payload data
  • 16-bit checksum code for error detection
  • 6-bit error correction code (ECC)
Figure 12. Low-level protocol overview

As illustrated in the figure before, the sequence of the CSI-2 frames begins with an exit from the low-power state (LPS), followed by the start of transmission (SOT) signal, indicating the start of the packet.

Initially, a short packet (SP) is transmitted, followed by the end of transmission (EOT) signal, and a return to low-power state. The sequence then transitions again with SOT to transmit a long packet, which includes a packet header (PH), payload data (DATA), and a packet footer (PF) for error detection, followed by EOT and a return to low-power state. This process repeats for subsequent long packets, ensuring efficient and robust communication.

CSI-2 packet description

The CSI-2 low-level protocol consists of short and long packets, each with a distinct role.

Short packet

The CSI-2 short-packet format for a D-PHY physical layer is shown in the figure below .It is similar to the long packet header, except that the word count field is replaced by a 16-bit data field. The ECC 8-bit code is present to secure headers having potential bit corruption.

Figure 13. CSI-2 short-packet structure based on D-PHY physical layer

Long packet

The figure below presents the long packet format. It includes a data ID that consists of a virtual channel number, associated with the data packets, and the corresponding data type.

The word-count field allows the LLP to determine the data-payload width before detecting the packet footer and the EOT. This information is used to detect shorter-than-expected packets, and to trigger a corresponding error

The ECC corrects a single erroneous bit in the packet header in the case of data corruption.

The packet data corresponds to the data payload for a given virtual channel and for an identified data type. To complete the long-packet reception block, a packet footer that contains a 16-bit CRC checksum is received and compared to that which is internally calculated by the LLP, to check the data integrity of the received data payload.

Figure 14. CSI-2 long-packet structure for a D-PHY physical layer

DCMIPP interrupts

The DCMIPP handles the following interrupts for both the parallel interface and all pipes (Pipe 0, Pipe 1, and Pipe 2) :
  • Parallel interface interrupts
    • IT_ERR indicates the detection of an error in the embedded synchronization code sequence (only in embedded synchronization mode). This interrupt is indicated by bit 6 in the DCMIPP_PRSR register.
  • General pipe interrupts:

    Specific to Pipe 0

    • IT_LIMIT: This interrupt is triggered when the received volume is larger than the maximum allowed dump volume. It indicates unexpected behavior in Pipe 0, which usually does not occur if there are no issues.
    • IT_LINE: This interrupt is for stripe-based operators and is triggered after every 1/2/4/8/16/32/64/128 dumped lines and the last line. It is measured and extracted at the end of the pipe, close to the output to memory.
    • IT_FRAME: This is a secondary interrupt, serving as a backup for fast software. It is triggered after the last data dump of this pipe and is inactive if no capture has occurred. It is typically used to quickly reconfigure non‑shadow registers (during only vertical blanking) and to trigger the usage of the previously captured frame.
    • IT_OVR: This interrupt is triggered when there is a data overflow, meaning the memory was too slow compared to the received pixels. This indicates unexpected behavior in Pipe0, which usually does not occur if there are no issues. The recommendation is to skip the impacted frame.
    • IT_VSYNC: This is the main interrupt, where most software can sit. It is triggered at VSync (mid blanking) and is permanent, even if no dump is active. It is typically used to slowly reconfigure the pipes (at least shadow registers) for the next frame and to trigger the usage of the previously captured frame.

Each pipe has a dedicated interrupt local register that controls the fourth interrupt event (OVR, VSYNC, FRAME, LINE), as indicated in the DCMIPP_P0SR, DCMIPP_P1SR and DCMIPP_P2SR registers for Pipe0, Pipe1, and Pipe2.

DCMIPP events

The DCMIPP contains four events to alert the software to react to certain application situations. These events are linked to the pipeline. The table below mentions these events:
Table 3. DCMIPP events
PipeEvent
Pipe0HSYNC
VSYNC
Frame
Line
Pipe1HSYNC
VSYNC
Frame
Line
Pipe2HSYNC
VSYNC
Frame
Line

Capture modes

The DCMIPP captures only the frames required by the application optimizing memory access, AXI bus occupation and power consumption. The applicative software may request to get continuous frames from the camera sensor, or only frame in snapshot mode. These two different modes are selected thanks to the bit CPTMODE inside the DCMIPP_PXFSCR register.

Snapshot mode
In snapshot mode, the software triggers the bit CPTREQ to launch the acquisition from the next start of frame for a single frame only. The bit CPACT remains high only for one frame. It is deasserted at the end of the active frame. At the same time, the bit CPTREQ is cleared by hardware to be ready for future requests, other snapshot acquisitions, or even continuous acquisition after modifying CPTMODE bit in such case.
Figure 15. Snapshot capture mode
Continuous mode
In continuous mode, the DCMIPP processes the frames continuously. The software needs to arm the bit CPTREQ = 1, and the next VSYNC triggers the data acquisition until the deassertion of this bit. The bit CPTACT remains high, indicating this continuous mode operation.
Figure 16. Continuous capture mode