John Li
Lagos — The Society of Automotive Engineers defines 6 levels (0 to 5) of autonomy available to vehicles, ranging from level 0 (no automation) to level 5 (full automation). The jump from level 2 to 3 is a large one that has seen limited deployment, as automotive OEMs assume greater liability, and eyes can be taken off the road in some situations. From level 4 onwards, vehicles are completely driverless in some conditions, with full automation in level 5 eliminating the title of ‘driver’ as the human becomes a passenger only. To achieve this, automotive OEMs and tier-one suppliers are required to expand their sensor suites beyond conventional cameras and radar, accompanied by reliable software to perceive the vehicle’s surroundings. In IDTechEx’s new report, “Infrared Cameras for Automotive 2025-2035: Technologies, Opportunities, Forecasts”, IDTechEx evaluates and forecasts the potential of long-wave infrared (LWIR) cameras for autonomous driving (levels 4 and 5), including both private vehicles and robotaxis.
2024-2030: SAE level 2+
For most passenger cars on the market today, the implementation of thermal LWIR cameras would be for increased AEB performance, either under regional regulations such as that set by NHTSA in the USA, or generally to improve overall safety. In such implementations, thermal cameras can provide detection ranges greater than 100m, allowing sensing beyond headlights in low visibility conditions. LWIR cameras are especially effective at pedestrian detection, which aligns well with initiatives such as the EU’s Vision Zero, aiming to achieve zero road deaths by 2050. Current AEB systems reduce the chance of fatal collisions by approximately 25% and are tested as part of overall vehicle safety in regional New Car Assessment Programs. For SAE level 2 cars, it is likely only one LWIR camera will be required in a vehicle for forward detection, combined with other sensors around the vehicle.
LWIR cameras for autonomous driving
The jump from ADAS (advanced driver assistance systems) to autonomous driving is considerable. In ADAS, for the most part, sensors are required to detect presence. For example, in adaptive cruise control, sensors scan the road ahead to detect the vehicle in front, feeding this back to the vehicle to maintain a safe distance. However, in autonomous driving, this sensing has to be operational at all times, in all directions, with much superior object classification required to respond to the different scenarios a driver would face on the road. To this end, the sensor requirement increases both in number and variety. The average number of cameras, radar, and ultrasonics sensors per vehicle goes up dramatically from level 2 (most common on today’s roads) to level 4 (autonomous driving). Furthermore, the need for excellent depth perception, combined with operation in a variety of conditions, opens the door to other sensors. IDTechEx believes that SAE level 4 is unachievable without the implementation of currently less conventional sensors, such as LiDAR or infrared cameras.
IDTechEx believes that the majority of level 4 vehicles will use LWIR cameras. In addition to its potential in passenger AEB, LWIR cameras are likely to be used in level 4 private vehicles and robotaxis for the same general reasons: detection of humans and other objects through heat signature and the ability to see in low visibility conditions such as night or fog, where RGB cameras would struggle, while radar alone provides no actual image data. A key theme in all autonomous vehicles is the necessity of redundancy. In case of the failure of one sensor, multiple sensors of the same or different type are required to provide redundancy data, such that the vehicle remains operational. To this end, even if LWIR cameras aren’t the main sensor for image classification or depth perception, their ability to take on these responsibilities temporarily makes them an attractive option.
Furthermore, with increased responsibility on the vehicle to perceive surroundings and make sensible decisions in the absence of an actual driver, edge cases have to be covered by the sensor suite. An example of such an edge case would be at night, with a pedestrian crossing the road from behind an obstacle. A camera could maybe pick up the person’s head above the obstacle, but an LWIR camera could perceive the danger by detecting the characteristic heat signature of a human before that, as LWIR cameras can work at ranges above 100m. Visible light cameras have a limited range, depending on the headlights and ambient lighting.
Performance requirements at higher SAE levels
Performance-wise, there are some differences between the LWIR cameras likely to be used in ADAS vs autonomous driving. While AEB is most likely to use only one LWIR camera for forward detection, level 4 vehicles could use multiple to cover both sides and the rear or in a stereovision setup, where the input from two cameras is combined to achieve a superior depth perception compared to single camera imaging. An example of this would be in the technology used by Foresight Automotive, where both visible light and thermal cameras can be used as part of a stereovision setup for autonomous driving. While most automotive LWIR cameras are at a maximum of approximately 0.3Mp (much lower than the tens of Mp achievable with a typical camera sensor, e.g., Sony IMX), LWIR cameras for autonomous driving are also likely to demand greater resolutions for object detection. Furthermore, most commercially available thermal cameras require a shutter system. This means there are split seconds when the camera is not capturing image data as it needs to recalibrate. Shutterless thermal cameras, such as from Valeo (partnered with Teledyne FLIR), in series production by 2027, and AdaSky, will be an attractive option for level 4 vehicles.
IDTechEx considers the technical requirements and number of cameras required for each LWIR-enabled function in its forecasts for the automotive infrared camera market. It also considers competing technologies and the current market landscape to get a holistic view of how thermal cameras could increase overall vehicle safety, and barriers to entry into a market with mature, cost-effective solutions in the form of cameras and radar. Forecasts and technical analysis can be found in IDTechEx’s report, “Infrared Cameras for Automotive 2025-2035: Technologies, Opportunities, Forecasts”.
*John Li, Technology Analyst at IDTechEx
