I’ll explain how a car thinks when driving autonomously

A pedestrian approaching the crosswalk, a bicycle suddenly changing direction, a car cutting in at the last moment. For a driver, these are everyday situations that often need to be handled in just a few seconds. For an autonomous driving system, however, they represent a series of problems that must be solved using cameras, sensors, calculations, and artificial intelligence. But how does a car interpret what’s happening around it and decide what maneuver to make?

To find out, I interviewed Dong Li, CTO of QCraft, a Chinese company specializing in developing advanced autonomous driving systems for mass-market vehicles. At the heart of our conversation was QPilot PRO, a technology designed to assist drivers in both urban and highway conditions, even when traffic presents complex situations.

Li explained to me how the system works, what strategies it uses to predict the moves of other road users, and what happens when it encounters a scenario it cannot interpret with sufficient certainty.

Cameras, sensors, and artificial intelligence: how the car makes decisions

QPilot PRO is part of QCraft’s family of driver assistance systems, which also includes the AIR and MAX versions. It is designed to help the car navigate in cities and on highways thanks to the Navigation on Autopilot (NOA) feature. Its computing power ranges from 100 to 200 TOPS, a unit that indicates how many mathematical operations it can perform per second.

QPilot PRO is part of QCraft’s family of driver assistance systems.

Photo by: QCraft

The system uses 11 cameras, supplemented by a LiDAR sensor, to reconstruct the surrounding environment in three dimensions. The cameras identify cars, pedestrians, traffic lights, and lanes, while the LiDAR helps determine the position and shape of obstacles. By combining this data, artificial intelligence attempts to understand where the car can go and how other road users are moving.

The next step is deciding how to react. “Our main innovation lies in integrating safety requirements throughout the entire process, from model learning to trajectory selection,” explained Dong Li, CTO of QCraft, during the interview. The system combines road interpretation and event prediction with safety rules defined by humans, thereby making decisions more controllable.

To learn how to handle different situations, QCraft also uses world models, which simulate the functioning of the physical world, and reinforcement learning, a technique that helps artificial intelligence improve its decisions through learning.

QCraft, how artificial intelligence interprets the road

Photo by: QCraft

This is how the system attempts to identify obstacles and predict possible outcomes. As Li explained, if a ball rolls on the road, the system should consider that a child might run to get it and slow down in advance. Similarly, if there is water near pedestrians, it can reduce speed to avoid splashing them. This is the difference between simply seeing what happens and interpreting the context to anticipate potential consequences.

What happens when the car is in trouble

Naturally, no system can be trained in advance for all possible situations that may arise in real-world traffic. That’s why QCraft uses large-scale simulations and physical world models to create a virtual environment where rare scenarios, extreme weather conditions, and dangerous situations—things that would be difficult to recreate repeatedly on roads—can be simulated.

Dong Li, CTO of QCraft

Photo by: Motor1 Italia visual (AI-assisted)

In addition to these tests, there are the data collected during daily use. According to figures provided by Li, by 2026 QPilot solutions had exceeded one million installations in production vehicles. By the end of July, drivers had traveled a total of over 4.5 billion kilometers using the company’s assistance systems. QCraft also reports an involuntary activation rate for its emergency automatic braking system of less than one occurrence per 500,000 kilometers. This is a figure cited by the company and should be viewed alongside the measurement conditions and criteria used—on its own, it does not allow one to determine how safe a system is compared to human driving in every situation.

What happens, however, when the car encounters a situation it cannot interpret with sufficient reliability?

According to Li, the system evaluates the available information, road conditions, and sensor reliability. If the situation involves too many uncertainties, it can adopt a more cautious approach, such as slowing down, increasing the distance from the vehicle ahead, or avoiding premature lane changes. If the conditions exceed its operational capabilities, it must alert the driver to take control again.

This is a crucial point because QPilot PRO falls under the category of Level 2 driver assistance systems. Therefore, the driver must continue to monitor the road and the system’s performance, remaining ready to intervene at any time. The car can handle certain driving tasks, but human responsibility for control remains intact.

To mitigate risks associated with distraction or over-reliance on technology, QCraft works with manufacturers on driver monitoring systems as well. If, during active assistance, the system detects that the person gets up from their seat or remains unbelted for more than five seconds, it can request manual control and increase the torque threshold required to intervene on the steering wheel, reducing the risk of an unintended touch triggering a reaction.

The driver’s posture is also monitored: if the driver leans back until they are in a reclined position, the system continues to prompt them to return to an upright posture. If prolonged fatigue or distraction is detected, it may request that the driver take over driving again, and if there is no response, it will brake to bring the vehicle to a safe stop within the lane, following the strategies outlined by the system.

Challenges still remain. Extremely heavy rain, glare from lights, obstacles that hide other road users, and extreme situations continue to test even the most advanced technologies. To address these, QCraft trains its models using simulated images from cameras with glare or partial obstruction, as well as scenarios where some sensors do not function properly.

In systems that do not rely on high-definition maps, incorrect mapping information is also introduced during training to reduce dependence on data that may not reflect reality. The company also uses redundant models to mitigate the consequences of any failure in a component.

Then there is the issue of updates. Data collected by vehicles can help identify new problems and improve subsequent versions, but this does not mean that cars learn directly while driving by autonomously changing their driving strategies. Before being distributed, updates must pass verification through simulations, scenario reproduction, regression testing, and road tests. Validation is then completed in collaboration with the vehicle manufacturer.

Arriving in Europe also requires specific training. Roundabouts, traffic priorities, pedestrian crossings, signage, and drivers' habits vary from one market to another. QCraft has already conducted urban and highway tests in various locations in Germany and France, but Li emphasizes that there is no fixed number of kilometers required for each country: training and validation depend on the differences between local conditions and those already familiar to the system.

QCraft: Arriving in Europe Requires Specific Training

Photo by: QCraft

The company’s goal is to make these technologies available in more affordable vehicles as well. QCraft claims it has already introduced urban assisted navigation systems on 128 TOPS platforms and states that QPilot PRO covers 99% of daily commuting scenarios according to its own assessments. The system also supports over 300 parking scenarios, including very tight spaces and multi-level garage parking.

These are results and goals that show how much assisted driving is evolving, but they do not eliminate the core issue: roads are unpredictable environments, and the ability to handle a common scenario does not guarantee success in dealing with an exceptional one. It is here that the maturity of these technologies is measured—not only in their capacity to make decisions when everything works as expected, but also in their ability to recognize when information is insufficient and when it is necessary to give again room for human intervention.

Read also