Renesas autonomy — our open platform for sharp vision

The automotive smart front camera is a unique sensor when it comes to ADAS and Automated Driving. The big difference to other sensor technologies like Radar, Lidar or Ultrasonic is that a smart camera can identify road markings, traffic signs, and traffic lights. Additionally, it can also recognize objects, provide data that facilitates their identification i.e. as cars or pedestrians, detect their motion path, and calculate their distances. This makes the smart camera a sensor that enables – under good visibility - features like automated emergency braking or provides the information necessary for automatic car steering.

Thus, front cameras are playing an important role for carmakers in complying with NCAP (New Car Assessment Program) standards, which are instrumental in reducing accidents with other traffic participants i.e. vehicles and pedestrians.

 

Smart cameras are expected to be used in high volume in future cars, which implies that they should be available at a low cost.

 

The challenges for a front camera includes requirements for high-performance computer vision and high level of functional safety, moreover, at low power consumption. The latter is a stringent requirement, as the small smart camera housing including the sensor itself and the SoC is mounted next to the windshield. At this position, the camera is exposed to direct sunlight, which raises the temperature in addition to the heat being generated by the components itself. Therefore, the power dissipation by the components of smart camera has to be as less as possible.

 

 

V3M and V3H SoC's, innovative solutions for NCAP Smart cameras

The V3M and V3H from the Renesas autonomy platform™ is designed for NCAP compliant smart cameras, providing high-performance computer vision, low power consumption, and high functional safety.

Block diagram of V3M

The V3H SoC is optimized for use in stereo front cameras and achieves five times more the computer vision performance than its predecessor R-Car V3M.

Block diagram of V3H

High-performance Computer Vision

V3M and V3H implement a top-notch performance computer vision heterogeneous platform using different accelerators: a versatile pipeline engine (IMP) and a computer vision engine (CVE) supplemented by DMA to control memory transfers. Furthermore, both SoC's coming with an integrated IP for convolutional neural network (CNN). This makes the R-Car V3M and V3H suitable for computer vision algorithms, like object detection & classification, deep neural networks, and convolutional neural networks. Additionally, to the mentioned hardware accelerators, V3H has dedicated IPs for Dense Optical Flow, Dense Stereo Disparity and Object Classification.

 

High level of integration

The V3M and V3H comes with integrated ISP (Image Signal Processor) that enhances the quality of raw image from the camera sensor and makes it thereby ready for machine vision. The integration eliminates the need of an external ISP component in the front camera or in the sensor itself. The system has a single DDR3L memory channel, which reduces the cost for memory components compared to other solutions for front cameras.

 

The V3M integrates a dual Cortex A53 and V3H a quad Cortex A53 for the application software and both have a Cortex R7 Lockstep cores to run AUTOSAR.

 

This reasoned architecture reduces the need for external components and keeps the bill-of-material to a minimum, still offering high performance and ensuring safety at all times.

 

 

 

Open platform

The V3M follows the principle of the Renesas platform: it is an open platform supported by Renesas and its ecosystem of partners. The ecosystem of partners is further increasing and will complement Renesas’ offer with additional hardware, operating systems, and partners for computer vision and on the application level. This gives customers the freedom to choose where to start development:  based on the differentiating features of Renesas, or using a solution from Renesas or its ecosystem partners.

 

 

This choice allows Tier1 to maximize the creation of value in-house, or shorten time to market by working with partners - a new option in the market for smart cameras.