The range of driver monitoring systems –

source: Mouser Electronics

According to a 2021 study by the National Highway Traffic Safety Administration (NHTSA), more than 38,000 car accidents occurred in the United States in 2020 — and approximately 95% of those accidents were caused by human error. Advocates of automated driving systems have long argued that automated driving technology will be central to reducing the impact of human error and making driving safer. However, experts agree that fully automated driving systems are still years, if not decades, away, making the human element still an essential part of the equation. As semi-automated driving systems become more ubiquitous in passenger vehicles, a relatively new form of human error has reared its ugly head – distracted driving. Concerns about distracted drivers with their eyes on their phones, tablets, etc., not paying enough attention to avoid preventable collisions and other accidents continue to grow as automated driving systems become more common . Although confirmed malfunctions in automated driving systems are sporadic, there are still valid concerns about their safety, mainly regarding the element of human error.

How do car manufacturers and regulators ensure that drivers operating vehicles with automated control systems pay attention to the road? One method is by using a technology that has been implemented in commercial trucking fleets for years, Driver Monitoring Systems (DMS). DMS uses infrared sensors to monitor the driver’s facial position and eye movements to determine if they are alert and looking through the front windshield. If the system detects that the driver is distracted or inattentive, it will beep or flash interior lights to warn them of potential road hazards and apply the brakes if the driver is unresponsive. Legislation has been introduced in the United States, the European Union and China to make DMS a mandatory feature on new cars by the end of this decade. Let’s take a look at the current state of DMS and where it might be headed.

DMS – past and present

As mentioned earlier, DMS is used in the management of commercial vehicles in the US, parts of Europe and Asia. DMS has been found to reduce accidents in the commercial shipping and transport industry and has helped drivers use their time more efficiently and confirm their adherence to the strict safety protocols that driving large commercial vehicles entails. Until recently, however, these systems relied primarily on human control of the drivers, which meant that the video captured by the DMS had to be uploaded to the cloud and then reviewed by human guards. The lag time created between the driver showing signs of distraction and the human monitor activating the driver alert system was a significant shortcoming in the functionality of these early systems. Using AI, DMS can alert the driver in real time if their attention is perceived as being distracted.

Most DMS technologies work in a similar way. A camera equipped with infrared LED sensors is mounted on the steering column that monitors the position of the driver’s head (is he looking up at the road or down at his phone?). The movement of the driver’s eyes and the frequency with which they blink (people blink more when they are sleepy than when they are awake and alert). When the DMS detects that the driver is not reacting sufficiently to possible road hazards or unsafe behavior, such as entering and leaving lanes, it activates the driver alert system. The DMS then allows the automated driving system to take the appropriate action if the driver does not intervene in time (Figure 1). Lexus was the first automaker to implement DMS in passenger vehicles in 2006, quickly followed by Lexus parent Toyota and Volvo. American car manufacturers such as Ford and Cadillac and European companies such as BMW have also developed and implemented DMS in some of their models. Expectations are that almost all automotive companies will invest in DMS in the next decade.

Figure 1: The driver warning system is activated

DMS: System Implementation and Safety

The main concern when implementing a driver monitoring system is safety. The system architecture must provide real-time responses to various environmental signals that indicate potential danger or unforeseen obstacles. These signals are observed and recorded by multiple sensors, which are combined and/or compared to identify outlier measurements in a process often referred to as “sensor fusion”. A dedicated processor with the necessary computing power and key features is an excellent place to start implementing this kind of system. The NXP Semiconductors S32V234 Vision and Sensor Fusion Processor provides the necessary platform to build the rest of the state-of-the-art DMS solution.

The NXP Semiconductors S32V234 Vision and Sensor Fusion Processor provides a fully integrated approach to safety-critical designs such as DMS by using a heterogeneous combination of general-purpose CPUs, special-purpose processing via 3D GPUs, dual APEX-2 vision acceleration cores enabled by OpenCL™, APEX-CV and APEX graphics and embedded image sensor processing (ISP) for HDR acceleration and color processing. The CPU includes a quad-core 1GHz Arm® Cortex®-A53 for compute-intensive algorithms and a 133MHz Arm Cortex-M4 microcontroller that offloads more minor compute-intensive system management functions.

As an additional safety and reliability feature, all processors have Error Correction Codes (ECC) to protect data and instructions in the processor’s cache memory, internal 4MB scratchpad memory and DRAM memory. The S32V234 also features a built-in Crypto Security Engine (CSE) with 16KB of on-chip secure RAM and ROM, as security is quickly becoming a critical factor in protecting internal IP and protecting against external intrusion attacks. The S32V234 processor is part of the NXP SafeAssure™ program. This program supports ISO 26262 ASIL B ADAS applications and includes several key DMS elements: pedestrian detection, object detection, lane departure warning, intelligent headlight control and road sign recognition. The S32V234 processor is also designed for automotive-grade reliability, functional safety and security to support automotive and industrial automation.

DMS: The current and future regulatory environment

The transition to a mandatory DMS in new vehicles is already underway in the EU. In 2019, the EU Council of Ministers adopted a common safety regulation that requires all new cars available in the EU to include enhanced safety features, including driver monitoring technology. The new rules take effect this year. Although they currently only apply to new cars with some level of automated driving capability, they will apply to all new cars in the EU by 2026. The Council estimates that this legislation will prevent over 100,000 car accidents over the next decade. Legislation is not the only way the EU is investing in DMS. The EU’s new car assessment program, also known as Euro NCAP, now requires new cars to include driver monitoring capabilities to receive a five-star safety rating.

In China, where more than 4 million cars were sold in 2018 before the pandemic reduced sales, DMS is also taking hold as a mandatory safety feature, albeit more gradually than in the EU. Jiangsu became the first Chinese province to make DMS mandatory for long-haul trucks and vehicles carrying hazardous materials in 2018 — a similar national mandate is expected soon. With the US and European markets leading the way, it makes sense for the most populous country to make significant investments in driving safety.

Congress is also taking legislative steps to make driver monitoring systems mandatory in new cars. In July 2020, the House of Representatives passed the Moving Forward Act, a bill to make driving safer by investing in infrastructure and road safety. The bill includes a provision similar to EU safety regulations that make driver monitoring systems mandatory in all new vehicles sold in the US. Although the PROGRESS Act must still be approved by the Senate and the President, similar legislation has been introduced in the Senate by members of the Senate Commerce, Science and Transportation Committee. The Be Informed For All Act, or SAFE, directs the US Department of Transportation to study the effect that automated driving systems have on driver distraction and the efficacy of DMS in reducing car accidents related to distracted driving. The result of the research may require regulators to introduce and enforce rules that require car manufacturers to include DMS technology in their new model. SAFE would order the study to be completed within two years of the bill’s passage and determine the effect of DMS on driver safety within four years.

If DMS is determined to make a significant difference in driver safety, manufacturers will have two more model years to meet the mandate. The legislation is designed to be “agnostic,” according to its supporters, “meaning it will not play favorites with certain technologies or developers. Both pieces of legislation have been held up in the Senate due to the pandemic and other factors, and while no firm timelines have been set for passage of either bill, both are expected to receive bipartisan support when they come up for a vote sometime this year.

The future of DMS

Given the international interest in introducing checks and balances in the mass deployment of (semi-)automated driving systems and improving driving safety, DMS seems to have a bright future in tomorrow’s automotive market. The implementation of AI has now made the DMS more efficient and provided improved functionality. The next stage of DMS development is likely to be the inclusion of facial recognition software – Subaru already uses FR in its DriverFocus system. The system alerts the driver if they take their eyes off the road for more than three seconds at a time, while also putting the car’s automated driving system on high alert in case the system needs to take over from the driver. There are privacy concerns associated with cameras that constantly record video while vehicles are operating. These concerns will likely need to be addressed before facial recognition software becomes a widely accepted DMS feature. Despite these concerns, if EU safety regulations and legislation in the House and Senate eventually pass in some form, DMS is expected to spread in passenger vehicles at a significant rate over the next decade. Get ready for your car to pay more attention to you when you’re not paying attention to the road.

To learn more, visit www.mouser.com