From the perspective of practical application, this paper uses FPGA as the main control chip to design a digital video interface conversion device. Conversion, resolution conversion and other operations have completed the conversion from ITU-R BT.656 format data to DVI format data, so that the BT656 data format image of the MT9M111 digital image sensor can be 1280×960 (60Hz) and 1280×1024 (60Hz). ) The two Display formats are displayed on the monitor of the DVI-I interface, and it also has the image still function. The standby state when the system is idle realizes the low power consumption of the whole machine, which is suitable for industrial sites using mobile devices.
Overall scheme design
The collection and display process of the real scene is shown in Figure 1. After the image sensor MT9M111 collects the real scene, it sends the generated ITU-R BT.656 data stream to the video conversion interface through the ITU data output port. The video conversion interface converts the ITU-R BT.656 data stream sent from the ITU data input port into a TMDS data stream and sends it to the display terminal for display through the DVI-I port. In this design scheme, the resolution of the output image of MT9M111 is 1280×960.
Figure 1 System acquisition and display process
In the process of collecting and displaying the real scene, the realization of the video conversion interface function is completed through the following steps:
1) Deinterleave the received ITU-R BT.ITU656 data stream;
2) Perform color space conversion on the deinterleaved data stream;
3) Write the converted RGB value of each pixel into memory;
4) Read out the RGB value of the pixel from the memory and convert it into a sequence of TMDS symbols;
5) Read out the RGB value of the pixel from the memory and convert it into a VGA analog signal value.
Hardware Architecture Design
The hardware framework block diagram of the system is shown in Figure 2. The ITU signal output by the image sensor (including YCbCr data stream, horizontal and vertical synchronization signal and pixel clock) is sent to the FPGA main control chip through the ITU input interface. The FPGA main control chip performs de-interleaving and color space conversion on the ITU signal, and then writes the converted RGB value of each pixel into the SDRAM memory. Then the FPGA main control chip reads out the RGB value of the pixel from the SDRAM memory according to the requirements of the output resolution, and sends the RGB value of the pixel to the TMDS sending chip and the D/A chip according to the VGA timing standard, and the TMDS sending chip Provide the digital channel of video data, and provide the analog channel of video data by D/A chip, collect together to DVI-I output interface, transmit to digital display or analog display to display.
Figure 2 Hardware Architecture Block Diagram
The resolution of the output image requires the data transmission bandwidth between the FPGA and the TMDS sending chip to be above 100M (pixels/second), so the speed of the FPGA is required to be fast enough. At the same time, because there are many interconnections between the FPGA and peripheral devices, the number of pins of the FPGA is required to be sufficient. At the same time, because the clock frequency provided by the crystal oscillator is 50MHz, which cannot meet the transmission speed of more than 100M, it is necessary to have a phase-locked loop inside the FPGA. In addition, in order to realize the offline work of the system, the FPGA is required to support the configuration chip. Finally, considering the area occupied by the system and the upgrade of later versions, the internal resources of the FPGA are required to be as abundant as possible. To this end, the system finally selected the Cyclone series FPGA of Altera Corporation.
Considering that the storage and display of video data are carried out at the same time, and the SDRAM memory is a single-port device, the writing and reading of data cannot be carried out at the same time, so two SDRAMs are required to perform ping-pong operations at the same time to complete the continuous reading and writing of data. Finally, the SDRAM memory of MT48LC2M32B2TG-6 of MICron Company was selected; the SiI164CT64 model of Silicon Image Company was selected for the TMDS sending chip. Since the resolution of the output image requires that the bandwidth of data transmission between the FPGA and the TMDS sending chip be more than 100M, this data stream must be sent to the D/A chip to complete the digital-to-analog conversion, so the conversion rate of the D/A chip is required to be Above 100MHz. At the same time, since the data widths of R, G, and B are all 8 bits, a dedicated image D/A chip needs to be selected. It needs to have three data channels of R, G, and B, and the width of each channel is at least 8 bits. According to the above requirements, the system finally selects the image D/A chip of the CSV7123 model from CSEMIC.
FPGA functional design
As the main control chip of the system, FPGA is the core of software design. According to the design idea of the overall scheme, the working process of the FPGA main control chip is as follows: firstly, it receives the video data stream in ITU-R BT.656 format sent by the image sensor. Serial YCbCr values are resolved into independent parallel YCbCr values. Then perform color space conversion on the deinterleaved YCbCr values and convert them into corresponding RGB values. This RGB value is then stored in a piece of SDRAM memory. At the same time, the RGB value of the pixel is read out from another SDRAM memory, and sent to the TMDS sending chip and D/A chip. After passing through the digital channel and the analog channel, it is sent to a DVI display or a VGA display for display. According to the working process of the FPGA main control chip, the designed software function block diagram is shown in Figure 3.
Figure 3 Software functional block diagram
In Figure 3, there are two working clocks inside the FPGA, which are bounded by the dotted line in the figure. The clock used in the left part of the dotted line is the 54MHz pixel clock of the image sensor; The clock is multiplied into a clock after 108MHz, and the 108MHz clock is determined by the resolution of the output image. The two clock domains are connected by an asynchronous FIFO. The whole system is divided into 6 modules: de-interleaving module, YCbCr to RGB module, asynchronous FIFO module, ping-pong operation module, SDRAM controller module and VGA sending module. In addition, the system can also achieve image still, system standby, mode selection and other functions.
Image display effect
Figure 4 is the display effect when the output image resolution is 1280×960 mode, the video image resolution detected by the display in the figure is 1280×960
Figure 4 Display effect in 1280×960 mode