How Do Self-Driving Cars Work?
A few years ago, Mary Barra the current CEO of General Motors made a statement that did not sit well with the 4-wheel enthusiasts. She claimed she is working on cutting their lineup of cars, eliminating the manual versions of some of their much-beloved models. She stated the Automobile market must move away from manual transmissions and aim towards achieving 5th level automation reasoning an increasing number of new drivers (Millennials and Gen Z) not finding the activity of manual driving “Fun Enough” anymore.
The statement created an outcry on social media but it was quickly pacified when A few days post her statement executives from the rest of American BIG 3 (Ford, GM, F.C.A) and the European Big 3 (BMW group, VW group & Mercedes Benz) reciprocated with her views. The Asian Big 3 (Honda, Nissan, Toyota) followed in agreement a few days later.
This statement might sound controversial for car lovers, but it makes absolute sense. Driving a manual car isn’t as fun anymore given the ever-increasing traffic density and the practically unchangeable road conditions in developed countries & developing countries alike. It’s just not the same it was 20–50 years ago where owning a car meant comfort & freedom. Drive yourself to the office on a 1-hour bumper to bumper commute and you’ll know it. An average commuter spends north of 40 minutes on the road every day in these conditions. This is why Autonomous driving & electric cars are a written in our future. Let’s touch on the working of Autonomous driving now.
Automotive Industry has 6 levels of Autonomous driving,
Level 0 — The cars all way from their inception to the early 90s. Totally manual, not as reliable as modern cars but easy to troubleshoot.
Level 1 — “Hands-On Autonomous” -The cars from the early 90s to early 2010s. Power Steering became the norm & Auto transmission took over. This era saw the emergence of features like Cruise Control, Adaptive Cruise control (ACC), etc but they remained largely manual.
Level 2 — “Hands off -Eyes on Autonomous”. During the mid-2010s the lane correction systems emerged, Collision Avoidance systems & adoption of ACC systems. However, these systems are more on the high-end cars, and it’s only now that they started making their way to regular consumer cars. The Autonomous system here is elementary and needs constant supervision from the driver. It can only maintain the settings input by the driver but cannot think on its own. All the current ‘Driver Assistance’ technologies offered by manufacturers like Tesla’s Autopilot and Cadillac’s Super Cruise fall under this category.
Level 3 — “Eyes Off System” — The vehicles will come with environmental detection capabilities and can make basic decisions. This is the point where they start evolving from “Driver-assistance-systems” to “Driver-replacement-systems” but there’s still a need for the driver’s supervision. As of the moment, Audi’s A8L is the only car in the market to have this system and it takes quite some product cycles to trickle it down to the luxury market and then to the consumer market.
Level 4 — “Mind Off System” — These systems can work by themselves in a controlled environment. They work in conjunction with the environment they exist in with zero inputs from the driver. They cannot replicate the agility and instincts of a human but can accomplish tasks at a steady and unhurried pace. The caveat is that they can only function in an environment that they are programmed for which is called “Geofencing” and must be operated manually in the non-Geofenced areas.
Level 5 — “Fully Automated System”
The hardware side of things:
An array of Cameras — Capture Visual Data
360 degrees LIDAR sensors — Capture the fine intricate details of the environment
Integrated photonics — Helps spread the rapid firing laser beams from LIDAR sensors throughout the environment
IMU — Internal Measurement UNIT sensors — TO get the precise location of the car
GPS sensors — To aid in localization
Internal Altimeters — Measures altitude of the car
Gyroscope — Keeps track of the orientation of the car
CPU/Processors — Process all the data to generate a safe path
The software side of things:
Deep learning
Customized algorithms from manufacturer
High accuracy Map/Navigation software
Stages of Path Generation
Computer Vision + Sensor Fusion + Localization + Path Planning + Control
The Theory:
Autonomous cars use something called “Computer Vision” to identify the objects on the road. Computer vision actively looks for parameters like colors, gradients, and edges through an array of cameras set up on the car. A trained deep neural network identifies lanes and objects on the road. The deep neural network is trained just by feeding it loads of images through which it will learn to distinguish objects. With enough training, they can identify images better than a human brain.
Now that a combination of cameras captures and delivers a series of images to the system this will form the basis of understanding of the environment the car is in like the presence of cars, objects, and obstacles.
This understanding is further enhanced by a 360 degrees assemblage of rapid-firing LIDAR sensors and integrated photonics. This now captures additional data which will complete the system’s understanding of the environment like depth of the objects, the distance between the objects, distance from the objects to our car, the direction of their movement, speed of these objects, etc.
Now that the system has a complete understanding of the environment, the system works on localizing itself in that environment to the accuracy of millimeters. The idea that this works on GPS, is wrong. GPS has an accuracy of a few meters which is not suitable when a vehicle is moving at triple-digit speeds. for such situations need for accuracy up to a few millimeters and this is done by having a highly accurate map system, identifying landmarks on that map, and measuring the car’s distance from these landmarks using the LIDAR. These landmarks could be anything from a building complex to a simple manhole cover.
The car navigation in the safest possible way is done post establishing of localization. The system observes the speed of the surrounding vehicles and predicts their planned direction. Metrics like distance from objects, angulations in the path of the desired maneuver are calculated with constraints like speed limit, the legal distance between the vehicles kept in priority…. a waypoint is mapped evading the obstacles on the road should any exist.
Once the waypoints/path is created the car is prompted to drive in that direction. The entirety of this process is done in a span of nanoseconds and repeats itself in unlimited loops until the car reaches its destination.
