Market and technology trends

Surround view is a vehicle camera system that provides different views like top view, rear view and panorama view to assist the driver when parking, either manually or automatically. It offers the driver the view of the exterior of the vehicle to aid in manoeuvring the vehicle to park and to alert the driver to obstacles in its path that may not be immediately visible. A display on the vehicle’s interior control panel shows surround view, which is typically composed of four wide-angle cameras that are mounted in the wing mirrors, at the front and rear of the vehicle.

Currently, surround view provides a view to assist the driver as he or she parks. The next levels of surround view will be paired with object detection for auto park, which is where the vehicle moves from the traffic lane to a parking area while the driver is still inside the car. The next evolution will be remote park, followed by valet park.

The simplest implementation today is a 360 degree view from above in 2D perspective and is called “top view”. The different views are stitched together with the correct geometric alignment of the cameras. The brightness and colour of the different cameras’ videos are adjusted to enable a harmonised view.

Top View

3D View

This top view already assists the driver when parking, and can be enriched with additional information such as 2D overlays or a specific rear view using only the rear camera.

The next stage is to improve the representation with a 3D view and arbitrary viewpoints. This allows driver to choose any viewpoint around the car and get a more detailed representation of the car’s surroundings.

It gives the driver a better judgement of the distance from objects when parking. Automated object detection, in addition to the visual representation, will increase safety as it can give automated warnings to the driver. It will also be possible to let the car brake automatically during parking, if an object is too near.

Auto park, remote park and valet park are the next evolutionary steps. Auto park, with the driver still in the car, will provide automated parking based on object recognition, including other cars and pedestrians as well as confining elements like kerbs. The following step is remote park, where the driver can stand outside the car while it moves automatically into a tight parking spot such as a garage. Valet park will go beyond this – the driver will be able to leave the car at the entrance of a car park, and the car will find a space autonomously. This, however, will require investment in the infrastructure of the car park, as well as the sensors in the car.

Renesas Surround View Offer

Renesas Electronics has pioneered automotive SoCs with the R-Car automotive computing platform. The latest series, the R-Car Gen3, is based on the Arm® Cortex®-A57/A53 cores, which use the Arm 64-bit CPU architecture. They provide the performance capability to process the considerable amounts of data from multiple sensors around the vehicle.

There is a trade-off in terms of graphics and computer vision when developing entry level or high end systems. Entry level systems are typically less demanding, whereas high end systems provide better graphical capability. Most OEMs will want devices that can support both of these key areas and which are scalable. The R-Car Gen3 SoC family ranges from the R-Car V3M, targeting entry level vehicles, to the R-Car H3 for the premium segment.

Best User experience with H3

Targeting the high end of the automotive market, R-Car H3 achieves 40,000 Dhrystone million instructions per second (DMIPS). For best-in-class GPU performance, it also includes the Imagination Technologies PowerVR™ GX6650 3D graphics engine.

The R-Car H3 supports the multiple cameras inputs (up to 8) and multiple displays of high end vehicles (up to 4K).

Building on its proven track record in the automotive space, the R-Car Gen3 has an on-chip IMP-X5 parallel programmable engine with advanced image recognition technology. The automotive computer vision hardware is optimised for parking applications that require optical flow and multi-object detection (see Figure 4).

Figure 4: The R-Car H3 SoC supports up to eight cameras for best-in-class 3D graphics rendering

BOM cost optimized with V3M

Reducing the cost of the Bill of Materials (BoM) is a significant factor in the competitive automotive market. System integration can result in space and cost savings.

Figure 5: The entry level R-Car V3M reduces the bill of materials for surround view systems.

Image Signal Processing (ISP) integration processes the data from the sensor, which – together with on-chip automotive network support for CAN-FD (Controlled Area Network – Flexible Data-rate), Flexray and Ethernet buses – can reduce the total BoM cost. The R-Car Gen3 SoC system architecture introduces functionality and interfaces to the SoC to reduce the system cost.  R-Car V3M is the first of R-Car Gen3 that integrates all these features.

Development Support

For seamless development from PC to embedded systems, migration solutions can support the developer from the pre-development to the final development phase, with research and advanced engineering support, evaluation boards, simulation, or software in the loop, and the final embedded operating system (OS).

Figure 6: Development from R&D to final implementation

Starter kits, such as the H3 Starter Kits available from Renesas Electronics, can be used for tests in-car, and to check algorithms from the sensors to the processing unit. The R-Car Starter Kit Premier for the R-Car H3 SoC has both OpenCL and OpenGLES 3.1 support, as well as a powerful computing library.

The Surround View Solution Kit is an extension of the Premier Starter Kit to access to up 8 cameras, enabling complex scenarios as surround view, mirror replacement and driver monitoring.