“The global COVID-19 pandemic and the need for improved safety and efficiency are driving companies across industries to integrate intelligent embedded vision technologies into their systems to support people detection, touchless human-machine interaction, and enhanced AR/VR capabilities. Simultaneously use intelligent machine vision technology to improve manufacturing levels and yields. According to Allied Market Research, the global machine vision system market size was US$29.7 billion in 2019 and is expected to reach US$74.9 billion by 2027, with a compound annual growth rate of approximately 11.3% from 2020 to 2027.
The global COVID-19 pandemic and the need for improved safety and efficiency are driving companies across industries to integrate intelligent embedded vision technologies into their systems to support people detection, touchless human-machine interaction, and enhanced AR/VR capabilities. Simultaneously use intelligent machine vision technology to improve manufacturing levels and yields. According to Allied Market Research, the global machine vision system market size was US$29.7 billion in 2019 and is expected to reach US$74.9 billion by 2027, with a compound annual growth rate of approximately 11.3% from 2020 to 2027.
As a collection of Lattice low-power embedded vision system solutions, mVision provides customized performance and flexible Interface Interconnect (MIPI CSI-2, LVDS, PCIe, GigE) and designed for low power (150mW-1W), small form factor (2.5 x 2.5mm) in smart factory, machine vision, smart city and smart home applications -10x10mm) design is optimized to address various design challenges such as sensor interconnection, bridging, aggregation and image signal processing.
A small test
In March 2020, the mVision 1.0 solution was launched, providing a highly flexible, small and modular solution for consumer embedded vision designers who need to rapidly build prototype systems. Its core features include:
Video Interface Platform (VIP) – Modular hardware development board that supports a variety of video and I/O interfaces commonly used in embedded vision applications (including MIPI, LVDS, DisplayPort, HDMI, USB, etc.). VIP board support includes CrossLink™, ECP5™, and CrossLink-NX FPGAs based on the Lattice Nexus technology platform. The VIP provides easy input and output board interconnection through simple plugging and unplugging, and two 60-pin high-speed board-to-board connectors reduce physical wiring and ensure that designers can reuse existing proven hardware and software builds module.
Comprehensive IP Libraries – mVision’s collection of solutions includes ready-to-use IP cores for connecting MIPI and LVDS image sensors, ISPs, common connectivity standards (e.g. USB, Gigabit Ethernet) and displays such as HDMI, DisplayPort, GigE Vision, etc. standard.
FPGA Design Tools – Two easy-to-use FPGA design tools are available, Lattice Diamond® and Lattice Radiant®. These tools speed and simplify programming of Lattice FPGAs by automating many common design tasks.
End-to-End Reference Designs – To further accelerate system development, Lattice mVision provides complete reference designs for common embedded vision applications, including sensor bridging, sensor aggregation, and image processing.
Custom Design Services – For customers who need assistance in bringing their embedded vision systems to market, Lattice has established a network of design services partners that can handle everything from developing individual functional design modules to providing one-stop turnkey solutions and more A range of customer needs.
mVision 2.0: From Consumer to Industrial
A year later, in March 2021, Lattice launched mVision 2.0 on the basis of mVision 1.0 version. The solution adds some important updates, including support for the mainstream new image sensors used in industrial and automotive systems, as well as new image signal processing IP cores and reference designs to help developers design intelligent vision applications at the network edge. The solution collection also now supports the Lattice Propel design environment, which simplifies the development of vision systems using embedded RISC-V processors.
In addition, mVision 2.0 ISP Pieces Library (Drop 1.0) also has a complete ISP pipeline function, supports up to 150MHz pixel clock (720p120/1080p60, etc.) and up to 96dB (16-bit) scene dynamic range, providing flexibility for custom ISP solutions interconnect to balance performance and cost. The debug kit features real-time image capture of external DDR memory and supports the Lattice SSP registration access kit, which can be tuned for specific environments.
Major updates to the mVision 2.0 Solutions Collection include:
Expanded support for embedded vision applications in automotive, industrial and medical
Different from consumer-level embedded vision solutions, embedded vision solutions used in industrial, automotive and medical fields require low power consumption, small size, and extremely strict frame rate accuracy requirements, which do not allow frame loss. Therefore, Lattice has added a new development board in mVision 2.0, in addition to the original CrossLink, CrossLink-NX, ECP5, it also includes the mainstream image sensors Sony IMX464/IMX568 and ON semiconductor‘s AR0234CS that support industrial and medical applications , developers can configure and operate the camera sensor through I2C, SPI or other interfaces.
Supports Lattice Propel design environment and RISC-V architecture
Propel is a design environment for accelerating embedded processor-based development on low-power, small form factor Lattice FPGAs. The tool includes a complete set of graphical and command-line tools to create, analyze, compile, and debug hardware and software designs for FPGA-based processor systems.
The main purpose of adding support for the RISC-V architecture is to simplify the configuration of the entire solution by using C code instead of RTL; support. In the future, Lattice plans to gradually provide RISC-V soft core support on other devices.
New Lattice Image Signal Processor (ISP) Reference Design
In the past, sensor products all have their own ISP drivers. However, with the increase of sensor complexity and resolution, it is difficult for sensor manufacturers to mobilize a driver with a wide audience. Therefore, ISP drivers are often no longer provided, and most of them only provide transmission. Sense or raw data, this part of the work is gradually transferred to the solution developers. Lattice has added a dedicated ISP this time to expand the choice of mVision solutions for customers, which can help users quickly deploy applications such as embedded vision in product design.
Added image signal processing and bridging aggregation IP
The IP cores included in mVision 2.0 are mainly divided into four categories: Display processing, image signal processing, bridging and aggregation, and interconnection. Among them, image signal processing, bridging and aggregation IP are newly added items, and users can directly introduce these modules to reduce the process investment of developing from scratch. Also, all of these available IP blocks support CrossLink-NX.
From industrial display systems, Industry 4.0 solutions in M2M applications, to advanced driver assistance systems (ADAS) and in-vehicle infotainment systems, to DSLR cameras, drones, robotics, virtual reality (VR) systems and medical devices, The number of camera applications in all industries today is growing, and more intelligent applications such as object recognition, depth perception, collision avoidance and decision making are becoming a reality.
In order to build a flexible and intelligent learning environment and turn ideals into reality, Lattice, together with partners such as Softnautlcs, LogicFruit, Bitec, TATA, Helion and others, has developed dozens of flexible solutions to meet the needs of today’s embedded vision design engineers. Requirements such as changing interfaces, low-power image signal processing, and hardware acceleration.
Accelerate time to market with Helion IONOS ISP
Lattice has partnered with Helion Vision to provide a comprehensive solution, including a family of image signal processing products, from basic (ECO HD-ISP) to those supporting advanced high dynamic range image processing (HDR HD-ISP). Among them, Helion Vision’s IONOS IP series provides more than 100 independent IPs through IP libraries to achieve plug-and-play processing solutions, and on the basis of the previous optimization for ECP3™ and ECP5™ architectures, the new CrossLink -NX support, which can significantly reduce the required FPGA resources and latency.
mVision ISP Reference Design Based on Pipeline Architecture
The figure below shows a complete Lattice mVision ISP reference design based on a pipelined architecture, supporting Sony IMX464/IMX568 and ON Semiconductor’s AR0234CS image sensor, and shipping to early customers in November 2020. The earliest hardware platform used is ECP5, which is expected to support CrossLink within the first quarter of 2021. As planned, Lattice’s initial ISP reference design will focus on industrial, medical and automotive applications, and Lattice will offer a variety of EVDK modules including popular sensors ideal for medical, industrial, aerospace, automotive and consumer applications.
The new Sony IMX464 VIP sensor input board includes a Sony 1/1.8″ 4MP MIPI NIR enhanced rolling shutter sensor module, and an adapter cable to connect to the EVDK. Seamless connection with the Embedded Vision Development Kit (EVDK), 2688(H) support x 1520(V) resolution, 4 M pixels, up to 90 FPS, HDR and other performance indicators, suitable for surveillance, factory automation, industrial camera applications.
Left: Image without ISP processing; Right: RGB image after ISP processing
Embedded vision system design engineers face many challenges, image signal processing is just one of them, and the examples above are just a few examples. Image sensors come in all shapes and sizes, and have different levels of performance in terms of pixel granularity, signal-to-noise ratio, light sensitivity, and color depth. In addition, environmental factors such as lighting and color contrast of objects further complicate system design. These situations often pose serious problems for system design engineers as they perform image preprocessing in real time. To address the issues of prolonged product design cycles and rising risks, Lattice’s expertise in image processing will help them meet these challenges.